A peer-led intervention to promote sexual health in secondary schools: the STASH feasibility study

School-based interventions

Most sexual health interventions for young people are school based, as this setting facilitates access to the widest range of young people in terms of socioeconomic background and vulnerability to poor sexual health. Schools provide a major context for social development22 and are ideally placed to challenge unhealthy and inequitable norms underpinning sexual risk behaviours. 23 For interventions attempting to diffuse change through social networks, schools provide a ‘captive audience’ and naturally occurring friendship groups. 24

School-based interventions tend to comprise one or both of:

• sexual health education and training to improve knowledge, exploring emotions and sexual norms, developing sexual negotiation skills and self-efficacy, and making informed choices3,25

• providing access to sexual health services either in or out of school, including condoms or access to other forms of contraception (e.g. hormone-based medication). 26–29

Evidence from older systematic reviews suggests that school-based sex education has an effect on sexual health knowledge and attitudes, but only some programmes delay sexual behaviour and increase condom use. 3 Shepherd et al. 25 are cautious about the impact of such programmes in bringing about behavioural change, acknowledging that significant changes in sexual health knowledge and self-efficacy (e.g. knowing how to use a condom) are more common.

More recent systematic reviews suggest that combining education with contraceptives or access to sexual health services may be more effective. For example, Oringanje et al. 27 found that providing education alongside contraceptives increases condom use and may reduce unintended pregnancies. Interventions that combine education with interactive techniques to improve condom use, teaching how to negotiate sex and providing a range of contraceptives can lead to an increase in knowledge and effective use of contraceptives, delay sexual initiation and improve condom use. 28 Owen et al. 26 found that school-linked sexual health services did not encourage sex or risk-taking, and may at best reduce chlamydia infection (males only) and pregnancy; however, the evidence is based on a small number of mainly US studies of variable quality. Studies rarely show an impact on biological markers, such as human immunodeficiency virus, herpes and syphilis, partly because these are relatively rare outcomes and require extremely large samples to detect an effect. 29

A number of contextual factors affect these outcomes. A review of process data from trials30 suggests two key influences: (1) fidelity, influenced by the extent to which the school has a supportive culture, a flexible administration, and enthusiasm and expertise among those delivering the sexual health content; and (2) acceptability and engagement, influenced by enthusiasm, credibility and expertise of intervention providers, and relevance and enjoyment to young people. Another systematic review31 noted the diversity in approaches of effective interventions, suggesting that a range of mechanisms exist.

Peer interventions

Peer education is a method of disseminating knowledge, values or achieving behaviour change through a social network, delivered by someone of a similar age and standing in that network. 32 The theoretical appeal of peer involvement lies in its potential to reinforce positive values and beliefs, thereby strengthening social norms that might influence sexual behaviour. 33,34 The key mechanisms include role-modelling,35 a shared identity between the peer workers and their peer group, and the ability of the peer workers to influence their networks. 32 There is a range of peer-led roles and approaches, including educator or assistant educator,35 peer supporter (PS) and, occasionally, peer leader. 32

Evidence from systematic reviews on the evaluations of peer sexual health interventions for young people demonstrates a weak effect on sexual attitudes, knowledge and intentions, and ‘unconvincing’ evidence for the impact on sexual behaviour. 32,35,36 Methodologically, reviews have highlighted a paucity of peer-based sexual health interventions for young people that are evaluated using randomised control trials. 32

Two notable studies in the UK33,34 engaged peer educators to replace teaching staff, at least in part; however, in the other studies peer educators assisted teachers. Across peer-led interventions, the dominant model is one of a peer educator (teacher) rather than peers as supporters of positive norms and behaviour. 32–36 Involving youth peers as ‘influencers’ within their sexual networks has been under-researched in the field of sexual health. 36 In a review of peer interventions aimed at addressing a range of health and social issues, Harden et al. 37 identified five studies that compared the effectiveness of peer leaders with teachers in delivering the same intervention. In one sexual health evaluation from the USA, peer leaders were found to be generally more effective than teachers. More specifically, peer leaders are thought to be more effective at establishing conservative (non-risky) norms, but less effective than adults at imparting factual information and getting students involved in classroom activities. 34 A formal teaching role may undermine credibility with peers38 and may be less effective than informal social support work. 39

Participation rates in peer interventions vary and further research is required to optimise the most appropriate level of peer participation, which should consider the level of social ease experienced by peer workers, establish how participation affects peer networks and include theoretically informed outcome measures. 32,35,36

How peer educators are chosen and their reputation among peers is important. 32,33 Most studies rely on self-selection or teacher selection, but both these strategies may result in educators who are not particularly credible and find it difficult to reach high-risk students. 40 The Students Together Against Negative Decisions (STAND) study used peer nomination and diffusion of innovation theory in a youth-focused sexual health intervention in schools. 41 Peer leads delivered an education-based programme to ‘opinion leaders’, who were then asked to engage with their networks (although not via social media). Awareness of the STAND study intervention was high. The intervention was effective at improving knowledge and behaviour among those trained, but had mixed success in diffusing risk reduction practices to other non-trained students. 41

Digital media interventions

Digital media offer inspiration for rethinking sexual health promotion. 42 There has been an exponential increase in the use of digital media by young people in the last decade, including social networking, mobile applications (apps), video sharing, podcasts, online games and video or photo communications. 43 Most (95%) young people in the UK use social networking sites at least once per week, most commonly Instagram (URL: www.instagram.com; Facebook, Inc., Menlo Park, CA, USA), Snapchat (URL: www.snapchat.com; Snap Inc., Santa Monica, CA, USA) and Facebook (URL: www.facebook.com; Facebook, Inc., Menlo Park, CA, USA). 7 Technologies such as smartphones give unprecedented mobile access to the internet. 44 New internet-based platforms, including social media, offer greater opportunity for more interactive communication, possibly at the point of health-related decision-making. Digital media can deliver interventions conveniently and anonymously to young people using personalised information. 45

In theory, digital media offers the potential to deliver sexual health messages to those who are already making use of these technologies and are confident in using them. 46 Recent systematic reviews on the impact of digital media on young people found that it can improve sexual health knowledge (of STIs and emergency contraceptives) and self-efficacy (condom use and STI testing), and reduce sexual risk behaviour, but there has been no demonstrated impact on biological outcomes. 42,47–50

The popularity of social networking sites among young people render these as obvious candidates for sexual health promotion, although those in the health and research communities have been slow in using this approach. 51 A review of early interventions indicated that these sites were mainly used to heighten awareness of online sexual health programmes. 52 Few studies have examined the impact of social networking sites on the sexual health of young people. One cluster randomised controlled trial53 found that a peer-led community-based Facebook intervention called ‘Just/Us’ increased condom use over a 2-month period, although the effect diminished by 6 months. Another pilot study54 reported a potential increase in condom use and reduction in chlamydia infection.

Recommendations from systematic reviews highlight the need to involve young people in the design of digital interventions, the importance of using theory to inform intervention development and the importance of using stronger research designs,42,50 including research in the use of social media for sexual health promotion. 45 Few digital media programs provide tailored advice and none assesses social network interventions in school settings. 42,47,48 On a pragmatic level, findings from the evaluation of the FaceSpace project highlight the importance of securing sufficient resources for developing social network platforms and maintaining a high profile, reach and engagement with young people. 55,56

Developing a theory-informed young people’s sexual health intervention

Development work is essential to tailor theoretically grounded programmes to the needs of the target population. 63–65 The STASH study aimed to adapt what was known to work in a successful existing public health intervention to a new problem and context. The planned intervention drew on theories and mechanisms that were effective in ASSIST40 (see Chapter 1), while targeting an older age group (14–16 instead of 12–13 years) and focusing on STI prevention instead of smoking. Therefore, although ASSIST offered a theoretical starting point, the appropriateness of intervention components needed to be reviewed and revised accordingly. We anticipated that these differences would require significant adaptations to the ASSIST content and mode of delivery. Crucially, we also set out to explore how we might harness the potential of social media as a mechanism to communicate messages and, in doing so, modernise ASSIST.

Key intervention components brought forward from ASSIST

We began with the overarching mechanisms of ASSIST:

• Recruitment and training of ‘influential’ students (PSs) to spread and sustain positive norms through informal interactions with peers. Grounded in diffusion of innovation theory59 and recruitment of ‘early adopters’,66 ASSIST found that recruiting 15% of the year group was effective in the context of smoking prevention. 67

• Use of professional trainers to train PSs to increase intervention efficacy and credibility, reduce burden on schools and ultimately increase potential for scalability and widespread adoption.

We intended to use the following components from ASSIST.

Intervention development aims and objectives

The overall aims of the development stage were to finalise the intervention design and refine and test the initial logic model. The broad objectives were to:

• fully articulate the problem [i.e. young people’s sexual behaviour and risk of STI transmission (6SQuID stage 1)]

• explore and clarify causal and contextual factors relating to young people’s sexual behaviour and identify those most amenable to change (6SQuID stages 2–4)

• identify contextual factors that might shape implementation (6SQuID stages 2–4)

• establish how the STASH study might bring about change in young people’s sexual behaviour (6SQuID stage 3)

• identify intervention content, including existing high-quality resources (6SQuID stage 3)

• refine programme theory (overall 6SQuID aims).

Stage 1: evidence collation and synthesis – 6SQuID steps 1–3 (months 0–9)

The objective of stage 1 was to collate and synthesise evidence to inform key intervention content, establish the most effective delivery format and develop the most appropriate theoretical framework to guide this. We explored necessary adaptations to ASSIST, including a shift to social media as an alternative mechanism for delivering content. Key findings and recommendations derived from each activity are outlined in Appendix 1.

Stage 2: intervention co-production – 6SQuID step 4 (months 1–9)

Stage 2 addressed 6SQuID step 4 and aimed to develop and refine proposed intervention format and content, guided by learning from stage 1.

Stage 3: intervention optimisation – 6SQuID step 5 (months 10–21)

The overarching aim of stage 3 was to pilot the intervention in one school and to use learning from the pilot to refine the intervention. Guided by 6SQuID step 5, this stage consisted of six phases, described as follows.

Developing the STASH study programme theory

We conceptualised and used relevant theories at three levels (Figure 3): (1) the social system level (macro theory), (2) the individual level (mid-level theory) and (3) the behaviour level (micro theory). We identified, via literature review, mid-level theories relevant to young people’s sexual health and behaviour. This review, together with stakeholder consultations, informed the choice of behaviour change techniques and training, and website content. We developed our programme theory in line with what we already knew about mechanisms of change in ASSIST and adapted these for the sexual health context.

FIGURE 3.

Use of theory in the STASH study.

A detailed description of the theories and how we used them is given in Appendix 1 (see Table 19). Following the literature review, we excluded the least relevant of any overlapping theories and applied those most relevant at three levels (see Figure 3): (1) the social system of the intervention, (2) individual behaviour change and (3) specifics of intervention content. Together, these theories describe potential mechanisms for change that might be expected to occur as a result of the intervention. 74 Having explored their relevance and utility in the pilot, we integrated these into the revised programme theory (Figure 4) and feasibility trial.

FIGURE 4.

The STASH study programme theory: pre pilot.

Guided by these theories and our consultations (see Synthesising and operationalising development process findings), we established that it would be crucial to prioritise skills development (particularly communication), encourage and support active participation and responsibility-taking (within the intervention and with regard to sexual behaviour), boost intrinsic motivation and self-efficacy, provide SRE information relevant to young people and tailor the intervention to the needs of the target group.

Synthesis and actions relating to key intervention components

Peer nomination

All students in S4 (aged 14–16 years) are asked to complete a peer nomination questionnaire. Each school is given a different combination of nomination questions. Three are the same as those used in ASSIST:

1. Who do you respect in S4 at your school?

2. Who are good leaders in sports or other group activities in S4 at your school?

3. Who do you look up to in S4 at your school?’

Four questions are newly designed for the STASH study:

1. With whom in S4 would you feel comfortable to talk about something personal or sensitive?

2. Who in S4 is good at encouraging and persuading others to do things?

3. Whose opinion do you trust and value most in S4 at your school?

4. Who in S4 is confident at talking to people outside their friendship group?

The 25% of young people receiving most nominations, stratified by gender, are invited to a recruitment meeting.

Peer supporter recruitment

A meeting is held with nominees in each school, in which trainers explain the intervention purpose, the PS role and answer questions. The aim is to recruit 15% of the year group to participate in PS training. If recruitment attendance is poor or role uptake low (or skewed significantly towards one gender), a second recruitment meeting is held.

Two-day peer supporter training

A 2-day PS training session is held in school time, at an external venue, facilitated by Fast Forward and West Lothian Drug and Alcohol Service. The training is intended to equip PSs with the knowledge, skills and confidence required for the role; build motivation and enthusiasm for the role; generate trust and rapport within the PS group and among PSs and trainers; build sexual health knowledge and skills; improve understanding of risks and consequences; build self-esteem and self-efficacy; reinforce social support for healthy sexual norms; and boost intrinsic motivation and autonomy. PSs are required to sign a code of conduct during training and agree a plan to ‘announce’ the project to their year group.

Peer support work

The period in which PSs are active varies between schools: 5 weeks in three schools and 10 weeks in three schools.

Acknowledgement

Peer supporters who fulfil the requirements of the role would be provided with University of Glasgow certificates and, if they complete the online questionnaire, a £10 gift voucher. Schools may also support PSs towards attainment of an accredited Saltire Award.

Inclusion criteria

Secondary 4 students (aged 14–16 years) in state-funded schools who had received, or were receiving, teacher-led sex education, regardless of their sexual experience or individual level of risk, were eligible to participate.

Exclusion criteria

Students attending private or independent schools or state schools not currently delivering comprehensive sex education were ineligible.

The STASH study was designed to augment rather than replace usual sex education. Therefore, to participate, schools had to be already providing SRE. At the time of study, most Lothian schools were delivering a version of the Sexual Health and Relationships Education (SHARE) package85 as part of their personal and social education curriculum. Faith schools were included in the invitation to participate, but none took up the offer.

Purpose of process evaluation

The process evaluation was undertaken primarily to assess whether or not progression criteria were met (see Table 1), but more broadly to evaluate the following:87

• Implementation: what was put in place and exactly how was it implemented?

• Mechanisms of impact: how did the intervention produce change?

• Context: how did contextual factors shape implementation and outcomes?

In line with the study objectives, we sought to assess (1) the extent to which it was possible to recruit and retain PSs, (2) the acceptability of the role to PSs and key stakeholders and (3) the extent of intervention fidelity (i.e. whether or not PSs, trainers and schools were able to carry out the intervention activities as intended). The process evaluation also sought to identify barriers to and facilitators of successful implementation, to contribute to refining and testing the programme theory and to investigate the potential and limits of using social media to diffuse messages. We also evaluated the extent of friendships outside the school and therefore estimated potential contamination in a full-scale cluster randomised trial.

We undertook a SNA as part of the process evaluation. This sought to assess the potential reach of PSs across the S4 year group, by examining their position within the year group network. We wanted to understand the extent to which the nomination and recruitment process had resulted in PSs who could serve as ‘information hubs’ (network centrality) and who, as a group, could span an effective number of friendship subgroups within the year.

We also used SNA to investigate how friendship interactions were shaped and sustained by network properties, as well as individual characteristics, before and after the intervention. The analysis highlighted how perceived friendships interacted with the PS role. We also looked at homophily, that is whether subgroups within each school network were formed around gender (e.g. did young women tend to be friends with young women?) or sexual behaviour (did sexually active students mainly befriend other sexually active individuals?), and whether or not students influenced each other’s behaviours.

Process methods

We undertook basic process evaluation activities in all six schools and enhanced data collection in four ‘case study’ schools. One school was excluded as a case study school, as it had recently been part of a highly intensive research study, whereas another was specifically included because we were interested in exploring intervention feasibility in a very small school. In selecting two case studies from the remaining four schools, we sought balance in terms of urban/periurban location, school size and proportion of free school meals (FSM) (see Chapter 4).

Evaluation activities for all schools comprised:

• A monitoring log (spreadsheet) of all intervention activities (e.g. attendance at training and follow-up meetings by gender, field notes from school visits and informal conversations between research team/teachers).

• An anonymous evaluation questionnaire on PS training, completed by PSs (n = 104) and school staff (n = 12). It was completed immediately following training and capturing views on training activities, students’ perceptions of what they gained and staff views of content and organisation.

• An anonymous online questionnaire for all PSs (n = 88, 85% response rate). It was completed in a computer room during the final follow-up session. PSs received a £10 voucher on completion. Data were captured on the specifics of PS activity and engagement (see Measures used in process evaluation).

• Interviews with the STASH study trainer (n = 3). Conducted separately with each trainer, these addressed perceptions of intervention components (including what worked and what did not work, and why), their role and potential future improvements.

• Reflective record of all activity sessions (recruitment, training and follow-up) completed by trainers (n = 48). These captured student attendance and engagement, and any modifications made to the session plan.

• PS activity log (spreadsheet) recording Facebook group membership (by gender) and posting activity. Anonymous data were extracted via the STASH study trainer account (with PS consent obtained at the intervention’s outset). The number of face-to-face conversations self-reported at follow-up sessions was also logged (extracted from trainer reflective records).

• Items on a follow-up questionnaire assessing engagement in the STASH study intervention activities, views on acceptability of format and content.

• Items on friendship ties from baseline and follow-up questionnaires for SNA (see Measures used in process evaluation).

Enhanced process evaluation activities comprised:

• Structured observation conducted by the research team at recruitment (n = 4), PS training (n = 8) and follow-up (n = 8). A non-participant observer sat in a discreet place and, using an observation guide, recorded data on fidelity (activities carried out as intended), acceptability (apparent responses of/interaction with students, challenges arising), exposure (degree of student engagement) and any relevant contextual information.

• Semistructured interviews with PS and non-PS students (11 groups/pairs, n = 42 participants). PS interviewees were selected to give a mix of gender and engagement (whether using Facebook or face-to-face interaction only). Non-PS students were recruited via PSs (with support from the contact teacher) as we were interested in understanding views of students who had some exposure to the intervention activities. Interviews addressed awareness of the STASH study, acceptability and engagement with messages and perceptions of impact.

• Semistructured individual or paired interviews with school staff and the STASH study contact teacher plus a member of senior management team with pastoral responsibilities and some awareness of the STASH study intervention (seven interviews, n = 8 participants). Interviews focused on perceived value and impact of intervention, and potential barriers and facilitators from the schools’ perspectives.

Measures used in process evaluation

The PS questionnaire (see Report Supplementary Material 2) focused on PS experiences regarding the feasibility and acceptability of the role. It comprised a wide range of items, measuring PS activity, time spent on the role, how easy or difficult they found the role, extent to which they were able to meet challenges and difficulties (including technical issues), views on the role and the STASH study website and training content, perceptions of parent and teacher views on their involvement, how their peer group responded and interest in ongoing involvement.

The process evaluation also used measures from the baseline and follow-up questionnaires (see Piloting of outcome measures). These focused on measuring exposure to the STASH study activities among all students; the acceptability of intervention content and activities among those exposed; the use of Facebook; whether the STASH study was more or less relevant, helpful or interesting compared with usual classroom SRE; whether or not it was age appropriate; and the likelihood of using the STASH study website or talking to a PS in future.

For the SNA, students were asked to name up to six friends and for each to give information on gender, if in year group, if connected on Facebook, if they would share private information with the person, and if time spent together was in school, outside school and/or online (yes/no options).

Analysis of process data

The process evaluation framework details data sources for all process research questions, whereas the qualitative analysis plan gives more detail on the analysis (see Report Supplementary Material 3). Quantitative data from the follow-up questionnaire (see Report Supplementary Material 4) and PS questionnaire (see Report Supplementary Material 2) were summarised descriptively and stratified by gender, as relevant. A descriptive summary was produced from the PS activity log (Microsoft Excel^® file; Microsoft Corporation, Redmond, WA, USA), collating group membership numbers, total posts, total reactions to posts and reported face-to-face activity. Information on Facebook group membership was documented by school for the SNA.

Project monitoring data were entered into a different Excel document and subsequently summarised descriptively. Data included specifics on PS nomination, recruitment, training, attendance at follow-ups and completion, broken down by gender and school; contemporaneous events, such as relevant news stories; significant communication with schools; and key school information.

Transcribed qualitative data from observations (recruitment, training and follow-up) and interviews (teachers, trainers, PSs and non-PSs) were entered into NVivo 11 qualitative analysis software (QSR International, Warrington, UK) to manage data retrieval. We undertook a thematic analytic approach informed by the framework method. 88 A coding framework was developed and applied to transcripts, based on the key aims of the process evaluation [i.e. fidelity, acceptability, exposure, reach, contextual factors (including barriers to and facilitators of implementation), plus researcher reflections on the process of implementation]. From this descriptive stage, data were interpreted further to establish links between themes, develop potential explanations around such links and to better understand the functioning of the intervention. Quotations were selected both to exemplify and illuminate key findings, and to illustrate variation in participant experiences and perspectives. The primary aim was to assess the overall feasibility of the intervention, based on analysis of these constituent themes.

Social network analysis

We conducted a SNA study to evaluate the potential reach of PSs across their social networks. From a SNA perspective, a network is composed of a set of nodes and a set of ties connecting some of the nodes. In this case, nodes corresponded to students in school and a tie appears each time there is a reported friendship relationship among them.

In principle, it can be assumed that each node is capable of connecting with anyone else in its network. However, individuals tend to have preferred patterns of connections because they tend to enjoy some ties more than others. These preferences, usually reciprocated, generate a specific pattern among all members of the network.

In general, SNA measures and routines help in the rigorous and systematic visualisation of the distribution of preferences represented by a tie among pairs of individuals and in testing and exploring the properties of the network (e.g. closeness, cohesion and clustering).

In the present study, students were asked to name up to six friends and for each to give information on gender, if in year group, if connected on Facebook, if they would share private information with the person, and if time spent together was in school, outside school and/or online (yes/no options).

First, we used a direct reach measure, that is the number of students directly connected to a PS (% of the total) to evaluate the potential reach of PS through their direct friendship ties. Second, we measured target reach, corresponding to the number of friendship clusters containing a PS, expressed as a percentage, and revealing the distribution of PS within friendship subgroups. In this case, we applied the algorithm developed by Girvan and Newman89 based on the identification of ties whose removal contribute to the breakdown of the network in small subgroups. Third, we applied standard SNA centrality measures to compare (1) students who were nominated but declined the PS role, (2) students who were nominated and took up the PS role and (3) other students. In particular, we used indegree, two-step reach90 centrality and eigenvector centrality. Indegree corresponds to the number of incoming ties, that is the number of times an individual was named as a friend. This can be interpreted as a measure of popularity, as high indegree scores indicate students who have been named as a friend by many others. The two-step reach centrality measure indicates the potential of PSs to spread the STASH study information with other students within two steps (e.g. via word of mouth). This measure examines how close each student is to all the others. Eigenvector centrality indicates how well connected an individual is by considering how well connected their friends are. This measure can also be considered as an indication of popularity: students with high eigenvector centrality are connected to other students who are themselves well connected. Table 2 summarises the measures and their visual representations.

TABLE 2 - Social network analysis centrality measures

SNA centrality measure	Definition	Visual representation
Indegree	The number of ties going towards a node
Two-step reach	The number of nodes a given node is able to reach in k steps; in our case k = 2
Eigenvector	Each node’s centrality is proportional to the sum of the centrality scores of the nodes that are connected to it

To analyse changes in ties from baseline to follow-up, we applied a stochastic actor-oriented model for network dynamics. 91–93 These classes of statistical simulations are commonly used to test hypotheses regarding the co-evolution of behaviours and networks related to a set of individuals. In our case, we used stochastic actor-oriented models to analyse the formation of ties among students considering individual characteristics (gender, being a PS or not being a PS), behavioural characteristics (sexual experience, self-assessed confidence to refuse unprotected sex, self-assessed knowledge of STI risk) and the influence of PSs in promoting preventative healthy sexual behaviours.

Each network was analysed separately and then a Fisher’s test (which combines independent p-values of parameter estimates and standard errors across the networks) was used to determine the presence of significant differences among schools. We used meta-analysis to aggregate the resulting values of the parameter estimates for each school and compare them on a common scale (see Appendix 3). Finally, we used the goodness of fit test to examine whether or not our data samples fit the simulated networks. 94 These analyses were conducted in the simulation investigation for empirical network analysis package (version 1.2–14, released 5 December 2018) in the R statistical software (The R Foundation for Statistical Computing, Vienna, Austria).

Purpose

A key study objective was to identify key trial design parameters for a future evaluation, including candidate primary and secondary outcomes, ICC and sample size. Since setting this objective, the DELTA2 guidance has been published, which advises against using estimates of effect from pilot studies to calculate the sample size for a full trial because of their imprecision. 95 We therefore did not seek to fully meet this objective. The purpose of collecting these outcomes was to pilot our outcome measures and analysis methods to assess the utility of these for a future definitive trial. The quantitative analysis was not intended to offer any guide to the intervention’s effectiveness due to lack of statistical power.

Outcome methods

The control questionnaire was administered first (March–June 2017) to the S4 year group above the intervention cohort. We administered the baseline questionnaire in August/September 2017 to the intervention cohort, who had just moved up to become the new S4 year group. We administered the follow-up questionnaire to this same S4 group post intervention (March–June 2018). Compared with the control questionnaire, the baseline questionnaire had a reduced set of outcome measures and the follow-up questionnaire had additional exposure measures.

Outcome measures

Aim

The aim of the economic evaluation was to determine the key components of a future cost-effectiveness analysis and test data collection methods.

Measures

To calculate STI-related health-care costs, we assessed the feasibility of collecting self-reported health-related health-care resource use (STI testing and treatment, contraception, including out-of-pocket costs) at follow-up. This included contacts with school specialists, health-care professionals in STI services and primary care, STI treatment and prevention, and pregnancy-related costs. Control and intervention students were asked at the end of S4 to report the number of times they had used each service in the past 6 months. We report descriptive statistics, including missing data, to evaluate the feasibility and acceptability of collecting this information.

The National Institute for Health and Care Excellence recommends that evaluations of new interventions include a cost-effectiveness analysis: the incremental cost per unit gain in an effectiveness measure. Quality-adjusted life-years (QALYs) calculated using the EuroQol-5 Dimensions (EQ-5D) are the preferred effectiveness measure in cost-effectiveness analyses. 106 There is no preference-based tariff for the EQ-5D that can be used to calculate QALYs in children. Instead, we explored the Child Health Utility 9D (CHU-9D) to calculate utility scores,107 completed at baseline and at the end of S4 in control schools, and at the end of S4 in intervention schools.

Quality-adjusted life-years are not always the most suitable effectiveness measure in economic evaluations of public health interventions. 108 We evaluated if the CHU-9D was sensitive to changes in the potential primary outcome, to determine its suitability in an economic evaluation of the STASH study. We also report descriptive statistics for the CHU-9D and its relationship with (1) SWEMWBS and (2) self-esteem, to provide a picture of how it functions within this population group.

Analysis

Ethics approval

Ethics approval was granted by the University of Glasgow Medical, Veterinary and Life Sciences College Ethics Committee in two stages:

1. intervention development/pilot, project number 200150088

2. feasibility study, project number 20160002.

Written consent was sought from all participants taking part in research activities, using information sheets and oral explanations by researchers. The exception was observations made at PS activities; here, participants were given a verbal explanation of the researcher’s presence and advised that no identifying information would be recorded. All students and their parents in the S4 intervention group were sent information on the STASH study. Parents and carers were given the opportunity to opt out their child from the survey and qualitative interviews. PSs and their parents gave written (opt-in) consent to take on the PS role. PSs were advised that the research team would have access to activity in secret Facebook groups to log activity (such as the number of posts and ‘likes’), but that no personal data would be recorded. At all study stages, participants were informed that they could withdraw from any research component at any time without prejudicing their experience at school. Research participants were reassured that their answers would be treated in confidence. In qualitative interviews, participants were advised that the researcher would only break confidentiality if they made a disclosure suggesting that they might be at risk of serious harm or of harming others. The intervention website included links to local and national referral services for sexual health and well-being, as well as child protection services.

All researchers and fieldworkers working in schools had Protecting Vulnerable Groups clearance.

Assessment of harms

The intervention was low risk to participants. The team anticipated that the main risk would be psychological harm resulting from online bullying and breaches of private information by peers. The issue of ‘harm’ was investigated in the process evaluation. Procedures were developed for reporting trial protection concerns in schools (see Report Supplementary Material 6 and 7). The former was designed in consultation with the child protection lead for West Lothian Health Education Authority and the latter in consultation with the TSC.

A number of steps were taken to mitigate the possibility of harm: PSs were required to sign up to a code of conduct (the STASH study charter); social media use was confined to private (‘secret’, i.e. non-visible, invite-only) Facebook groups, of which a STASH study trainer was a member; trainers undertook monitoring ‘spot checks’ and PSs were encouraged to report inappropriate posting to the trainer and/or STASH study contact teacher; PSs had the option of privately messaging the trainer with any concerns; and trainers were advised to pass on any concerns to the contact teacher without delay. Procedures for dealing with inappropriate online behaviour followed the information and communication technologies code of conduct and discipline code of participating schools.

Peer supporters were trained on how to respond to any sensitive disclosures from their peers (such as respecting privacy) and on when to share a disclosure with the staff contact teacher (or school teacher designated to handle child protection issues). They were advised that any disclosures that made them feel worried or uncomfortable should be reported without delay to the contact teacher/designated member of staff. After this point, the school procedure for handling disclosures relating to child protection would be followed.

In terms of reporting ‘harms’ to the TSC and funders, the procedure set out that if alerted to a harm, the principal investigators would triage according to severity and report to the TSC and funders as appropriate. If a harm was reported for which causality by intervention was deemed possible, probable or definite, the principal investigators would communicate to the TSC within 15 days.

Governance

Patient and public involvement

The study was co-produced with young people and professional stakeholders via an extensive series of consultations and development workshops (see Chapter 2). Working with a local youth organisation, we established a Young People’s Advisory Group to advise on study design. The group met once in the initial phases of the project. A subsequent meeting was planned, but proved too difficult to arrange because many of the original members were no longer in contact with the youth group. Instead, the study sought input from teachers and students involved in the pilot to optimise the feasibility study stage. In terms of study oversight, our TSC included a lay member who had a strong professional interest in young people’s sexual health but no training in the evaluation of complex interventions.

Undertaking the control, baseline and follow-up surveys

We opted to investigate the feasibility and acceptability of web-based computer-assisted personal interview over paper-and-pencil interview. Computer-assisted self-interviewing enables more complex skip patterns, improves internal consistency and reduces missing items,117 but they are rarely used in school surveys. 118

The control, baseline and follow-up web-based computer-assisted personal interview questionnaires were completed by students using school computers, with paper versions available in case of technological difficulties. Dates for data collection were arranged 4 months in advance and contact teachers were asked to book computing suites, apprise relevant staff and provide class lists. Reminders were sent 3 weeks prior to data collection. At the data collection sessions, a teacher was usually present to take the register and help with queries. Two fieldworkers were allocated to each computer room, with two or three rooms used at each session. A lead fieldworker moved between rooms, to ensure that participants had arrived in the correct room and to support the fieldworkers in answering any questions. Once the class was seated and the teacher had completed the register, the fieldworker would read a short script to the class. The script provided information about the study and emphasised that data would be confidential. Participants were given an envelope containing their username, password and a web slip with a link to the questionnaire website. Participants were asked to complete the questionnaire as if they were doing an examination, with two fieldworkers per classroom ‘invigilating’ this process. Most schools completed their data collection with three visits per wave of data collection. Occasionally, additional mop-up sessions were scheduled if absence was particularly high at one of the data collection sessions.

Timetabling fieldwork

Adverse weather in March 2018 had an impact on fieldwork at follow-up in schools 1 and 4; both schools were closed as a result of snow on the scheduled week for data collection, and rescheduling of fieldwork at short notice proved challenging because of increasing proximity to public examinations (April/May). In one school, maths revision classes were scheduled after the data collection date had been set and were given priority over the survey; the research team were keen to respect these decisions. Fieldwork was also occasionally disrupted by school activities, such as field trips, work experience or special events. Sometimes these were arranged after the data collection date had been set or the STASH study contact teachers were not made aware of them. Extracurricular activities were generally held during less-pressured times in the academic calendar, such as post examinations (June). There was therefore no ‘ideal time’ to collect data, as periods away from examinations were also those in which more extracurricular activities were scheduled.

Workload for schools

The STASH study contact teacher was required to organise booking of computer suites, arrange movement of students from class to the suites to complete the questionnaire and herd any stragglers or students who had not received the communication about the survey. According to fieldworker reports, school information technology infrastructure was a limiting factor, with the number of available personal computers and computer rooms dictating the number of data collection sessions required. Fieldwork staff reported that network connectivity was acceptable in five schools, but problematic in school 6. There was also a small number of technical issues associated with the web survey, including recurrent login issues.

Ease of data collection varied between schools, but was always smoother if the STASH study contact teacher was present or available. When the contact teacher was absent or called to deal with another issue, fieldworkers were reliant on other teachers to access booked rooms and herd students. Schools varied in their capacity to cope with the absence of the STASH study contact teacher, with those assigning a clear stand-in or depute contact teacher managing more easily. In two schools, the STASH study teacher was rarely available: one through absence and one through lack of capacity. Schools also varied in the extent to which the contact teacher was able to secure ‘buy-in’ for the survey from other teaching staff, and this became increasingly difficult in the run up to examinations.

In interviews (see Chapter 3, Process evaluation), teachers highlighted logistical issues with the evaluation, referring to it as ‘a lot of work’ and ‘a bind’, particularly around organising the computer rooms. This was particularly so when schools felt that they were ‘quite limited in [. . .] resources when it comes to computers’. For this reason, teachers in two schools (schools 3 and 6) suggested that paper questionnaires would be more manageable. Another teacher (school 5) agreed that ‘a lot of organisation was needed’ to co-ordinate questionnaire sessions, but that they had all gone ‘according to plan’.

We used 53 paper questionnaires due to various logistical or technological difficulties. It was notable that data quality in the paper versions was lower. There were 62 instances of non-adherence to filtering or question instructions (skip patterns and prompts prevented these in the web survey).

Confidentiality

Adherence to examination conditions for survey completion varied between schools and classes. Fieldworkers tended to allow some general chat, unless they picked up that students were influencing or observing the responses of others. Fieldworkers reported that, owing to the layout of the computing suites, there was often little space between students. In addition, fieldworkers were required to walk around the room to assist students with queries or technical difficulties. This meant that privacy was not as easy to ensure as with a paper questionnaire (in which students can cover their answers with an arm).

Group interviews with friends of PSs (see Chapter 3) touched on the acceptability of questionnaire completion. Some students raised concerns about privacy arising from the potential visibility to fieldworkers, teachers and other students:

I just found it more weird that there was people walking around, just looking at your computer, looking what you were doing as well sometimes.

Like, sitting next to people, they can easily just look at what you’re doing.

Oh, did you feel like you didn’t have . . . ? [Privacy.] . . . Did that have any impact, do you think, on what you were filling in or what answers you gave?

Nah, not really. I just answered it.

It was mostly the people around you . . . Could just easily look at your . . .

So, were friends looking at each others?

I don’t think they were but it was just the feeling that they could.

And then like the teachers an’ all that walking around an’ you were just kinda like ‘OK’ . . .

School 3, friends of PSs, mixed group

These students were not deterred from answering questions, which may have been in part because of confidence in the confidentiality of the survey:

As long as your name’s not gonna go anywhere [on it] it doesn’t really matter.

School 3, friends of PSs, mixed group

At the end of all the questionnaires, participants were asked ‘Is there anything at all you would like to tell us about the STASH project or this questionnaire?’ and the responses were coded post hoc. Of 1979 completed questionnaires, six participants made comments on confidentiality in the open-ended ‘additional comments’ section, including ‘Nothing is stopping anyone from looking at my screen’ and ‘let the pupils complete this survey in a more private manner so they don’t feel embarrassed or lie about their answers because others can see’.

Questionnaire comprehension and sensitivity

Most students interviewed for the process evaluation said that they thought the questionnaire was ‘pretty straightforward’ and ‘easy to understand’. Teachers were able to comment across a broader range of students, with one teacher (school 4) highlighting that the questionnaire was difficult for students with literacy issues:

There was a couple that we had to sit with ’cause their English isn’t great, which probably wasn’t the best environment to do it with, sitting in front of the class, ‘cause you were having to read out things that they possibly didn’t want to share in front of everybody.

School 4, contact teacher

Fieldworkers also noted that issues of literacy, English as a second language and learning support limited participation for a few students (schools 2–4).

The nature of the ‘quite personal questions’ raised concerns for some students, primarily focused on sexual behaviour questions. Concerns were also raised in relation to the SNA friendship module, in particular the fact that names of students appeared from a pre-populated class list from which respondents could select their friends. This was done to minimise errors in naming students but was perceived by students as ‘weird’:

What did you think of the questionnaire?

It was all right.

Yeah.

It was weird how you could, like, put in people’s names and then click it.

Kinda weird, yeah. And then just came up.

Yeah, and it just like came up.

Aye, just some of the questions were weird. Like, it was just kinda random. We weren’t really told what the questionnaire was about and then like you just see like all these questions, like weird. [What do you mean by weird?] Unusual, like. I wasn’t expecting it.

School 3, friends of PSs, mixed group

This sense of the unexpected was reiterated in interviews with teachers. One teacher said that some students ‘were a bit taken aback by some of the questions’, but attributed this to the fact that ‘they’ve never had that question put to them before’ and that they, nevertheless, ‘got their head down and got on with answering’ them (school 5, DHT). Another teacher commented that some students may have felt ‘timid, or a bit shy’ about the questions, but that the boys they supervised were ‘quite confident and wouldn’t be shy about those kind of things. So they filled it in . . . fine without any issues’ [school 6, contact teacher (CT)].

In the open-ended ‘additional comments’ question that concluded the questionnaire, there were 15 positive responses, mostly brief comments on having enjoyed the experience, such as ‘I enjoyed this’, ‘it was good’ and ‘it was interesting’. Negative comments were more numerous. Eleven comments were general in nature (e.g. ‘pointless’ and ‘weird’). Thirty comments expressed discomfort with the personal nature of the questions, for example ‘This is none of your business’, ‘It’s very personal’ and ‘. . . some questions are uncomfortable to answer’. Five comments were made relating to the questionnaire being insufficiently inclusive, noting for example the equivalence of ‘penis in a vagina’ with intercourse and a lack of options for those identifying as asexual. Five comments expressed concern about the inputting of postcodes (for calculation of the SIMD category) and friend names, querying whether or not the questionnaire could really be anonymous. Six students commented that the questionnaire was too long or that they needed more time to complete it. Ten queries or suggestions were made that were specific to questions (e.g. more ‘not relevant’ options or use of simpler language).

Fieldworkers reported some of the common queries they received. These included requests for meaning of some words (optimistic, self-esteem), queries about why they were being asked to give their postcode and some confusion about the question on data linkage.

Response rates at control, baseline and follow-up

Table 5 outlines the response rate for the control group, intervention group baseline and intervention group follow-up.

The target response rate was met with ease in all but one school (school 4). This school differed from the other schools in being much smaller, with the highest proportion of students eligible for FSM.

It became apparent that class lists (which provided the denominator) were not always up to date, as they were often received a few weeks into the school term. This meant that a school leaver might appear on the class list and a new student might be missing. Contact teachers explained that there are students who appear on class lists, but do not attend school on a regular basis, either because of truancy or because they are engaged with other local authority education or social services. Comparing Tables 4 and 5, it is notable that the three schools with indices indicative of higher socioeconomic deprivation tended to have response rates below the average across schools. This pattern is possibly explained by a higher proportion of non-attenders in these schools.

Recruitment and retention of peer supporters

Students receiving the most nominations from their peers (roughly the top 25% of year group, stratified by gender) were invited to the STASH study recruitment meeting (see Chapter 2). Across schools, an average of 87% of those nominated attended the meeting and 52% of those nominated (63% of those attending the meeting) were trained. Of those trained, 97% fulfilled the requirements of the role, defined as posting three or more STASH study messages on Facebook or having two or more face-to-face conversations about the STASH study, and attending two or more follow-up sessions (Table 8). In relation to progression criterion 1, > 50% of nominees were trained in only three of the six schools, so the green and amber targets were not reached. However, 97% of trained PSs fulfilled the role – 100% of those trained in four of six schools – meaning that the green target for progression criterion 2(a) was far surpassed.

As progression criterion 1 was not met, we did a sensitivity check to see if students accepting the role differed from those who did not, in terms of the number of nominations they received. We found that students accepting and completing the role received 15.6 nominations from their peers, whereas those who did not take up the role received an average of 13.5 nominations. This suggests that students receiving more nominations were slightly more likely to take up the role and means that had we invited the top 21% (as opposed to the top 25%), we may have performed better on this progression criterion.

Role uptake and completion was slightly higher among young women. There were slightly more young men at the recruitment meeting (52.4%), but slightly more young women trained (54.8% young women, 104 students) and completing the role (56.6% young women, 99 students). In terms of acceptability across schools, observational and project monitoring data suggested that variation primarily reflected differences in levels of school support and buy-in to the intervention (e.g. HT presence at recruitment meeting, provision to students of clear information about where and when recruitment meetings would take place). There were small differences in engagement by gender, with slightly more young women than young men reporting all the key PS activities.

Motivation to participate

The most commonly cited reasons for attending training (student evaluation form, n = 104) were curiosity (27%), benefit to CV (21%) and fun (19%). The main reasons for remaining engaged throughout the intervention (PS questionnaire, n = 88) were learning useful information (44%) and viewing the project as fun (34%). These motivations were echoed in interviews, for example ‘I thought it’d be good to learn about it as well, but it’d also be quite fun’ (school 4, PS boy). In one group interview, PSs suggested that some students ‘went for the free food but . . . then everybody, I think, enjoyed it, like, so it was an experience’ (school 5, PS girl, mixed group).

When asked why they thought that some friends had decided not to take up the role, PSs cited ‘feeling uncomfortable’ and simple disorganisation and forgetting to ‘hand their letter in fast enough’ (school 4, PS boys). In terms of the latter, schools played a vital role in helping ensure that student disorganisation (e.g. in not returning parental consent forms) did not preclude participation. Particular schools were observed putting in extra effort, such as phoning parents and carers and obtaining verbal consent if a PS had forgotten their form.

Reasons for reducing involvement or withdrawal from the STASH study

Twelve PSs said that their level of involvement tailed off towards the end (PS questionnaire, 15%). The most common reasons were not wanting to miss classes (five students) and frustration with technical problems (three students). These students had nevertheless fulfilled the role’s minimum requirements. Although we were unable to obtain PS questionnaires from the five students who did not complete the role, informal conversations with PSs, trainers and teachers suggested that one felt uncomfortable with other PSs and the remainder were discouraged by timetable conflicts.

Factors associated with peer supporter engagement in the STASH study

Follow-up meetings and peer supporter activities

The six in-school follow-ups took place in class time and typically comprised trainers checking in on weekly or fortnightly activity and troubleshooting, plus an interactive activity on a STASH study topic (see Chapter 2). Attendance averaged 80%, varying slightly by week and school (Figure 7). The school with highest overall attendance had the smallest group of PSs (school 6 with 10 PSs) and the lowest overall attendance came from one of the two largest groups (school 1 with 26 PSs), suggesting that smaller groups may be simply easier to manage. School support (e.g. in reminding PSs about the meetings) appeared another potential explanation for the variation across schools.

FIGURE 7.

Peer supporter attendance at follow-up sessions. FU, follow-up.

Most PSs found follow-ups ‘quite good’, worth missing classes for, useful as a reminder of their role and a top-up for key information:

It was a good refresh, ‘cause if we hadn’t had the follow ups then we [. . .] would’ve forgotten quite a lot and not know as much.

School 3, PS boy, mixed group

However, some felt that they were less interesting than the training, possibly affecting attendance:

I think everybody would be more involved if there was more activities at the follow-ups. [. . .] I think that’s maybe why people haven’t turned up so much ‘cause it’s . . . not as interactive.

School 5, PS girl, mixed group

In designing the feasibility study, we deliberately set follow-ups either weekly or fortnightly to explore any difference this might make. This was originally for three schools apiece, but changed to four schools weekly and two fortnightly when some sessions were rearranged at one school’s request. Qualitative and monitoring data suggested that students were satisfied with either follow-up, that regular spacing and reminders were important and that sufficient time between follow-ups also facilitated engagement, giving PSs time to share messages:

It was all so much more information and [. . .] it was always spread out so we had enough time to post things.

School 4, PS girl

A practical consideration related to the use of Facebook by trainers to communicate reminders to PSs. A lack of scheduling clarity – particularly for Facebook non-users and when schools had requested timetable amendments – had an impact on attendance. Some students did not know when follow-up sessions were or only found out at the last minute, particularly in school 3, which had the lowest proportion of Facebook users (and relatively low follow-up attendance).

Observational data also highlighted a range in trainers’ ability to maintain engagement and rapport, including using students’ names and apparent familiarity with session materials, which led to some variation in what was delivered and how (see Chapter 6).

In schools 5 and 6, PSs suggested that conflict with popular examination subjects may also have contributed to attendance drop-off, particularly when follow-up sessions were scheduled repeatedly against the same academic subjects. Although timetable conflicts made it challenging to participate in follow-ups, this did not always equate to lower overall engagement. For example, a school 4 PS who reported a timetable conflict was nevertheless one of the highest engagers on Facebook and contributed significantly to the follow-up sessions that he did attend.

Extent to which peer supporters liked the role

The intervention was highly acceptable to PSs, with 85% of PSs reporting in the PS questionnaire that they liked the role [94% girls, 71% boys; range across schools 75% (school 1) to 100% (schools 3 and 4)]. Progression criterion 2b was therefore met with ease.

Overall, the PSs interviewed were positive about the STASH study, with common comments including ‘it was quite a good experience’ and ‘I think we’ve been well informed’. There was a sense in interviews that the role became more comfortable as they gained skills and experience. Although they had initially expected to feel ‘awkward speaking to other people about it’, they eventually felt that ‘it wasn’t as if it was something totally new that we were talking about, so, it wasn’t really a big jump . . .’ (school 4, PS girls). Engagement was also facilitated by group cohesion, which emerged in all schools to some degree. This was typically about familiarity and a sense of a shared experience, rather than friendships with other PSs. Boys from school 6, for example, felt that if the training had been ‘with 12 total strangers it wouldn’t have been that good a laugh’ and that being ‘with . . . I wouldn’t say pals, but folk that you kinda get along with, then . . . you’re a wee bit more laid-back in what you say and you know that it’s not gonna offend anybody’ (school 6, PS boy).

Peer supporters described the broader benefits of their involvement in the project as including an increase in their comfort levels in discussing sex and relationship issues, the additional learning around these topics, and the information and skills – ‘mental preparation’ (school 4, PS girl) – that this provided them with for the future. PS questionnaire data supported these findings, with 86% of PSs saying that the role would benefit their CV and 94% saying learning from the STASH study would help them protect themselves in the future. Some also noted that participation in the intervention had made them ‘more sociable’ and that being a PS was a ‘good life skill’ (school 3, PS mixed group), whereas others highlighted a broad benefit of showing what information and resources are ‘out there’ (school 6, male friend), highlighting a potential contribution to young people’s online sexual health literacy.

Acceptability of time commitment

Around half of the PSs (52%) reported spending < 30 minutes per week on STASH study activities (posting Facebook messages or talking about the STASH study, excluding follow-up meetings); 29% spent 30–60 minutes and the rest spent > 1 hour (with two PSs spending > 3 hours). The majority (84%) found the amount of weekly time they spent on the STASH study to be acceptable, with 7% saying that it was too much and 9% saying that it was too little. Most (61%) reported the number of weeks they spent as a PS to be ‘just right’. There was no clear pattern in views on intervention length according to whether the PS had weekly or fortnightly follow-ups.

Acceptability of online and social media components to peer supporters

For the most part, PSs reported accessing the STASH study website at home alone (51%) or in school with others (44%). Most (84%) PSs liked how the website looked and found it easy to navigate (64%). The majority (70%) found it easy to post messages from the website to Facebook and 87% found it easy to direct friends to the website. Eight per cent (n = 7) reported that their friends had disliked the STASH study messages or website.

The use of social media use in the STASH study was broadly acceptable to PSs, once they knew that posts would be made in a closed group. This was indicated by general comments like ‘it was actually alright’ (school 6, PS girl). There were mixed views on the relative merits of social media compared with face-to-face communication. Some felt that it was ‘a good way to get across to other people in our year’ (school 4, PS girl) and that there was enough flexibility to allow them to feel comfortable about who was in their Facebook group. Others felt that the use of Facebook, specifically, had constrained the project by limiting its perceived kudos (see Chapter 6 for associated limitations to intervention reach). A benefit of using social media was that it generated interest in sex and relationships information on the internet and social media, potentially enhancing digital literacy and use of the internet as a source of advice:

. . . before STASH I wouldn’t really pay much attention to it. But now I would, after learning about it. [. . .]

It’s because we’ve been trained . . . [. . .] If I wasn’t a PS and I got a notification, I probably would read the first couple of lines and then just swipe off. [. . .] Whereas, ‘cause I’ve been told about it face to face in an actual training session, I’d probably be more likely to like read it now. Be more interested.

School 3, PS mixed group

In general, interviewees seemed to take the view that although online communication bypassed awkwardness or embarrassment, face-to-face conversations (when engaged in) allowed them to expand on or explain in more detail about the posts they had shared.

Acceptability compared with usual sex and relationships education

Peer supporters found the STASH study more relevant (71%), helpful (76%) and interesting (75%) than their usual classroom SRE. Relative advantages included depth of information, with PSs saying that ‘it was good for us to have that knowledge, ‘cause it’s not really something that’s massively spoken about in school’, beyond the basics of ‘use protection and be safe’ (school 4, PS girl). PSs also felt that it bypassed the awkwardness of classroom SRE:

I think the teachers find it quite awkward, talking about it.

Yeah. They find it more awkward than we do. [. . .] And it’s like you’re not embarrassed to ask these questions [in STASH study], because you don’t have a bunch of people from your class that you’re maybe not entirely comfortable with.

School 4, PS girls

Peer supporter activity in the STASH study Facebook groups

Table 9 gives an overview of PS activity. Figures reported on the number of Facebook group members exclude the PSs who created that group and the STASH study trainer. Of 104 trained PSs, 87 used Facebook to share STASH study messages. The remaining 17 PSs used only face-to-face conversation. Facebook groups comprised an average of 12 members. Group membership remained largely stable over the intervention period (four groups gained and two lost one member). Most groups (65 of 87) were mixed gender, whereas 16 were all male and seven were all female. Seventy-five groups included non-PS friends (i.e. other S4 students from the year, hereafter ‘friends’) with an average of nine friends. This suggests good potential for messages to reach beyond the PSs.

Peer supporters posted 1343 STASH study-related messages over the intervention period, averaging 15 posts per Facebook-using PS. There were 746 recorded reactions to posts – ‘likes’, comments and shares, excluding flippant or unrelated comments, and comments from the STASH study trainers – averaging nine per Facebook-using PS. The majority of PSs (77%) posted at least three messages. Relatively few (n = 15) direct messages were sent to STASH study trainers by PSs, and only two of those were sexual health specific (the majority related to technical issues).

A Facebook Analytics report (using data from the bespoke API, which allowed users to log in to the STASH study website via Facebook) indicated 146 unique visitors to the website via the API. As only 87 PSs were using Facebook to log in to the website, this suggests a number of repeat logins by PSs. PSs who logged in via Facebook made a total of 408 site visits.

Figure 9 illustrates the temporal spread of message-sharing activity by PSs who were logged into the website via the Facebook API. Activity was relatively consistent over time, with spikes particularly around follow-up sessions. The graph does not capture all posting activity, because technical problems prompted many PSs to screenshot messages and post without using the API’s copy and share function. The PS questionnaire data (n = 88) suggested that 18% had problems logging into the website (via the API) and 37% had difficulties trying to share from the website to Facebook. Although this highlights a functionality issue, and a potential barrier to full participation, it also demonstrates the capacity and willingness of PSs to adapt and get around technical issues. Most PSs (57%) disagreed that they preferred to put messages into their own words rather than use words from the STASH study website, suggesting a preference for using the provided messages, rather than creating their own.

FIGURE 9.

The STASH study website message-sharing activity (by PSs logged in via Facebook API).

Peer supporters reported that Facebook may have served as a barrier to fidelity (and exposure). Of PSs using Facebook, 13% said that some of the people they wanted to join their group were not on Facebook (ranging from zero in school 4 to 21% in school 5) and 30% of all PS participants agreed that ‘people I wanted to join my group hardly ever use Facebook’ (ranging from zero in school 4 to 83% in school 3). This was more commonly reported by young men than young women (39% vs. 24%, respectively).

Facebook activity varied by school and by individual PS (see Table 9). For example, school 3 had the fewest PSs who primarily used Facebook (42%), but the highest average (self-reported) face-to-face conversations (12 per PS, twice the average for all schools). In interviews, PSs at school 3 said that their friends were ‘hardly using’ Facebook. Some PSs at school 3 and their friends said that they had mobile app notifications switched off, and thus were not aware of new posts. Conversely, PSs at school 4 reported relatively few face-to-face conversations, but had a higher proportion of PSs using Facebook (90%), higher Facebook group membership (including more non-PS members), more posts per PS and no PS reporting that their friends did not use Facebook. The reasons underlying variation in social media use by school were difficult to determine in this small sample, but could relate to emergent subcultures within schools, in which friendship groups go with one platform or another.

Although friends interviewed knew that the STASH study Facebook groups had been set up, and some knew that there was a website, not all said that they had engaged in-depth with the topics or messages (or were not willing to say so in a group interview context). This varied slightly among interview groups, with some friends more aware of some STASH study topics and able to discuss specific messages around, for example, ‘consent’ and ‘kinds of infections’ (school 4, female friends). These participants also noted ‘links to YouTube videos and stuff’, although typically said that they had ‘never really looked’ at these or other external links, a finding echoed elsewhere by male and female participants. We had initially expected signposting to external sources to be useful (see Chapter 2), but these data suggest that the ‘ask’ of non-PS participants (following a link from a STASH study message to an external site) might be too much. The process evaluation data confirmed that we were right to replace much of the signposting and text-heavy messages with primarily visual messages (memes, infographics), as these were more positively received, with participants noting that these stood out and ‘. . . were cheesy but alright . . .’ and ‘intrigued me more’ (school 6, friends mixed group).

Peer supporter engagement in face-to-face conversations

Most of the PS questionnaire participants (86%) said that they had talked with friends about STASH study topics, with 85% of these participants reporting at least three conversations (see Table 9). There was some variation across schools, with 100% of PSs in school 3 (low level of Facebook use) reporting at least three conversations, but only 67% of PSs reporting at least three conversations in school 6. Overall, more young women (86%) than young men (83%) reported having three or more STASH study conversations with friends. This included 76% of PSs (80% of girls, 72% of boys) who had advised a friend on where to seek help on sex and relationships.

Interviews explored in more detail how PSs engaged friends in face-to-face conversations. One PS described his conversations as follows:

[I was] just saying how they can see everything on the website, and I obviously showed them [. . .] and then it was just about, like, sexual health and keeping yourself safe in all these different situations, and not just about having sex. It was about the stuff behind it like consent and all that.

School 3, PS boy

Most friend interviewees also said that the conversations they had with a PS were quite general, about the fact that they were attending STASH study meetings and not necessarily much about STASH study topics per se:

Every now and then there was a ‘we done this, it was like this, and then this was said’ [. . .] But there wasn’t really that much [discussion].

School 6, male friend

Friends of PSs did seem aware of what the PSs had been asked to do, ‘Talking to people. If people are, like, worried about something’ and most had a general sense that the STASH study was about STIs, ‘diseases and stuff’ and ‘safer sex’, suggesting some diffusion of the STASH study messages.

Describing the school networks (pre intervention)

Four of the six school networks formed a single component; that is, every student in these schools had a path of ties connecting them to every other student in the network (S4 year group). Schools 4 and 6 had disconnected components, meaning that some students had no links to the rest of the year group. On average, students were separated by five or six steps (degrees of separation), with the exception of school 4, in which this average was three steps. Students averaged seven friendship ties (SD 3.8, range 1–19). This included both ingoing ties towards the node (i.e. others that had named that student as a friend) and outgoing ties from the node (friends that student had named). Network density – the proportion of actual ties from all possible ties in the network – ranged from 2% to 5%. This suggests a fairly low level of cohesion,90 although that may also reflect that students were able to name only up to six friends, and could include non-school friends in their list.

We used a stochastic actor-orientated model (see Chapter 3 and Appendix 4) to analyse change in the year group networks over time. Analysis was undertaken on students completing the friendship module at baseline and follow-up, and used data from five schools. School 4 was excluded because of a low Jaccard coefficient. This coefficient can be considered as a measure of stability: a low value means that the turnover in the network is too high to consider the data as evolving, limiting the possibility to apply RSiena (simulation investigation for empirical network analysis). For all other schools we found good values of Jaccard coefficient, ranging between 0.27 and 0.35.

Results showed that friendship ties tended to be reciprocated and two friends were likely to share the same friends. ‘Popular’ students (those most frequently named as friends) did not necessarily receive or make many more friendships over time. However, students who named more friends at baseline were more likely to add more friends at follow-up. We found that young women tended to name more friends than young men (Figure 10a) and, overall, there was a preference for same-gender friends (see Figure 10c). Level of sexual experience did not appear to be related to friendship ties (i.e. students did not tend to name friends with similar levels of sexual experience). Finally, friends were more similar at follow-up than at baseline in their assessment of STI risk with inconsistent condom use, but not in their self-assessed confidence to refuse sex without a condom.

FIGURE 10.

Gender influence in the selection of named friends. Values are log-odds (95% CI). (a) The difference in number of friends named between boys and girls; (b) the gender difference in ‘popularity’; and (c) the similarity (homophily) between friends by gender. RE, random-effects model.

Peer supporter distribution in friendship clusters

We investigated the distribution of PSs across friendship clusters. The presence of clusters represents a potential barrier to message diffusion because they suggest a level of segregation that those outside the cluster can find difficult to access. Therefore, PS presence across clusters is desirable. In Figure 11, nodes (circles) represent students and links among them indicate friendship ties. Friendship clusters (identified by using the Girvan–Newman cluster detection algorithm; see Chapter 3) are highlighted in different colours and represent groups of students with many reciprocal ties (dense connections). PSs are highlighted blue. The extent to which PSs were distributed across friendship groups varied between 33% and 80%. PSs were present in one-third of subgroups in school 4, in 45% in school 6, in half of subgroups in schools 1, 5 and 3, and in the majority of subgroups in school 2. This is consistent with findings from ASSIST40 (in which PSs were present in 50–60% of clusters) and suggests that PSs had reasonable potential to reach across the network, with some variation by school.

FIGURE 11.

Peer supporter distribution in friendship subgroups by school. (a) School 1; (b) school 2; (c) school 3; (d) school 4; (e) school 5; and (f) school 6. Nodes (circles) represent students; links among them indicate friendship ties. Friendship clusters are highlighted in different colours; solid blue circles are PSs.

The direct reach measure indicates that, overall, one-third of students (34%, n = 302) were directly connected to a PS. Schools 1 and 3 had the largest proportion of students directly connected to a PS (52%), whereas the smallest proportion was in school 6 (7%).

Peer supporter reach by Facebook groups

We also examined PS friend connections in the STASH study Facebook groups. In Figure 12, PSs are indicated in orange and friends who were at least in one Facebook group in light blue. In schools 1 and 5, 53% of students (excluding PSs) were members of a STASH study Facebook group (n = 90 and n = 71, respectively). In school 2, 37% (n = 37) of students were linked on Facebook, whereas in school 6 this was 25% (n = 31). Few students were connected through Facebook in school 3 (9%, n = 16) and 4 (2%, n = 1). As described in Peer supporter activity in the STASH study Facebook groups, PSs in school 3 tended to use face-to-face interactions rather than Facebook. School 4 encountered additional challenges in implementing the STASH study, which seems to be reflected here in the minimal reach beyond the PS.

FIGURE 12.

Peer supporter distribution in Facebook memberships by school. (a) School 1; (b) school 2; (c) school 3; (d) school 4; (e) school 5; and (f) school 6. PSs are indicated in orange and friends who were at least in one Facebook group in light blue.

Comparison of peer supporter network position with nominees who did not take up the role and other students

Sample characteristics by level of exposure to the STASH study

Table 11 shows baseline characteristics by exposure. PSs and exposed students were more likely than unexposed students to be female. There were no differences across exposure group by area deprivation (SIMD) or residence type, but PSs had higher FSM eligibility than the other two groups. There was a gradient in academic attainment, with a greater proportion of PSs studying for National 5 qualifications, followed by exposed and then unexposed students. There was little difference between exposure groups in terms of ethnicity, age, religiosity and sexual orientation. There was some suggestion of a gradient in terms of sexual experience, with PSs more likely to report sexual experience, followed by exposed and then unexposed students. This suggests that the intervention may have reached the groups for whom STASH study information was most immediately relevant.

We examined student perceptions of school climate (e.g. perceived fairness, student–teacher trust, encouragement of participation) and compared across exposure groups to examine whether or not the peer nomination process selected students with a more positive perception of their school. Differences were small and seemed unlikely to be significant. In Table 11, we highlight two key measures: (1) perceived connectedness to others at school and (2) trying hard at school. Perhaps as expected, given the peer nomination process, PSs reported feeling closer to others at school than other students, but if anything they were less likely to say they tried hard at school.

Exposure to STASH study-related conversations

We examined data on students’ conversations about STASH study-related topics as a means of assessing any change between baseline and outcome that may have reflected diffusion of STASH study messages. Table 12 shows the proportion of students at baseline who reported talking about STASH study-related topics, by exposure, and the change in the proportions at follow-up. At baseline, PSs were more likely than other students to report conversations about all STASH study-related topics, and had greater increases in discussions about body and looks and STIs than other students. However, conversations about almost all of the topics increased regardless of exposure.

Principal component analysis on potential primary outcomes

Principal component analysis was carried out on the full range of potential primary outcomes on follow-up and control data sets (as listed in Primary outcomes). The main items that loaded onto the first principal component were the number of different partners in the last 3 months and number of partners with whom a condom or dental dam was not used, explaining around 20% of the variability in the sample. Main items loading onto the second principal component were use of a condom at last sex and how often a condom was used in the last 3 months. These explained 18% of the variance in the sample.

The first principal component did not make good sense as a composite outcome, particularly as it was restricted to those who reported having had oral sex or vaginal sex. The second was a more sensible composite, as it contained a greater amount of information on an important outcome, condom use, but it explained < 20% of the variance in the sample.

In general, the PCA of the potential primary outcomes did not provide any strong indications of a data-driven choice of primary outcome. The results were not used further in defining the primary outcome to take forward in the definitive trial.

Descriptive statistics for primary outcomes

Table 13 presents the summary statistics for all of the potential primary outcomes, overall and for the three study groups (baseline, follow-up and control). The rates of ever having had oral or vaginal sex would be expected to increase from baseline to follow-up (due to maturation effects), but would be broadly comparable in the follow-up and control groups, as group members completed the questionnaires at approximately the same time in their school education and were the same age (see Chapter 4 and Table 7).

Table 13 shows that the rates of ever having had oral sex were very similar between the control and follow-up groups. A lower proportion of those in the follow-up group than of those in the control group had oral sex for the first time in the last 6 months. The rates of condom and dental dam use were similarly low in both groups, at 12–13%, with the most common reasons being that students did not think it was necessary or they did not have one to hand. Most students who had oral sex did so with only one person, rendering the measure ‘number of oral sex partners’ uninformative as an outcome.

The rate of ever having had vaginal sex was 2% lower in the follow-up group than in the control group, with an increase of just under 6% from baseline. Almost 80% of those who reported having had sex had done so in the last 6 months in both groups, the proportion having had sex for the first time in that period being slightly lower in the follow-up group (31%) than in the control group (34%). These observations suggest that the STASH study may be potentially associated with a slightly lower rate of first-time sex among S4 students. Among those who reported having had vaginal sex in the last 6 months, use of a condom at most recent sex was slightly lower in the follow-up group (40.4%) than in the control group (44.1%). The main reasons cited for not using a condom were that students did not think it was necessary, they got carried away and forgot, or they had both been tested for STIs and had another form of contraception. The follow-up group were 3% more likely to have always used a condom when they had sex in the last 3 months, although the percentage who always or often used a condom was similar across the follow-up and control groups, at around 40%. Most students who had had vaginal sex had done so with only one person.

Suitability of primary outcomes

There was much discussion between the TMG and TSC regarding the identification of a suitable primary outcome for any future evaluation. Sexual activity and condom use are clearly important outcomes, which is why they were included as indicative outcomes in this study. However, when considered separately, they provide only partial coverage of the potential impact of the intervention on safe sexual behaviour. Condom use on its own is of limited utility as it is asked only of the subset of the sample who have had sex. The TMG recommended that both dimensions of sexual behaviour (sexual activity and condom use) should contribute to the primary outcome.

A further point of discussion concerned anal sex. This was not originally included in the list of primary outcomes, but it was noted (see Table 13) that the rate of anal sex in the follow-up group was 3.4% lower than in the control group, and the follow-up students were also less likely to have had anal sex in the last 6 months. As this outcome had a low incidence, this was felt to be a considerable drop. The TMG agreed that a reduction in anal sex would be a positive outcome of the intervention, as it is the riskiest practice in terms of STIs120 and as anal sex in young people is associated with coercion. 121 The TMG, in discussion with the TSC, also agreed that oral sex should be dropped from the composite measure because it is much less risky in terms of STIs120 and there is significant resistance to condom and dam use in oral sex. 122 Our data (see Table 13) confirmed that condom and dam use is much less common for oral sex (14.6% at last occasion of oral sex vs. 44% for vaginal sex). Although messages about condom and dam use for oral sex would be retained in the future intervention, we agreed that both intervention and outcome should focus on the practices with highest risk. This supports simpler messaging, which we found key to PS role effectiveness. In addition, it focuses the intervention on outcomes with the strongest potential for improvement.

The TMG therefore agreed to take forward a composite ‘always practising safe sex’ variable to the definitive trial. This would be defined as, ‘no condomless (vaginal or anal) sex in last 6 months’.

This means that students who were not having sex would also be considered to be safe from STIs (i.e. no sex equals no risk), as would those who were having (vaginal or anal) sex but always used a condom. To ensure consistency and inclusion of both heterosexual and same-sex relationships, it was agreed that the inclusion of anal sex in the ‘safer sex’ outcome should be on the basis that it could be happening, but would always involve condom use. We did not ask students in the current study whether or not they used a condom in anal sex, only whether or not they were having anal sex at all. We also use a 3-month period for reporting condom use in vaginal sex. Therefore, the composite primary outcome variable reported in this study is ‘always practising safer sex’ but is defined as ‘no condomless vaginal sex in the last 3 months and no anal sex in the last 6 months’.

The results from the current study suggest that this composite measure is likely to have greater utility than any of the individual outcomes, as it allows inclusion of everyone in the sample, and as both delaying sex and ensuring that sex is always protected are the safest ways to avoid STIs and unplanned pregnancy. The composite primary outcome is likely to be more sensitive to change because it is measured in more people than contraception use, which is relevant only to those who are having sex. The time marker of 6 months is consistent with the individual outcomes that were measured in this study and allows assessment of the impact of the STASH study intervention, as the study could not have affected behaviour prior to its start.

Therefore, a row is included at the bottom of Table 13 to show the rates of students ‘always practising safer sex’ in the current study and this is the variable that will be taken forward to the formal primary outcome analyses. The rate of this outcome is about 2% higher in the follow-up group than in the control group.

Primary outcome by study group

The difference in the primary outcome ‘always practising safe sex’ between the follow-up and control groups was estimated using a logistic regression model, adjusting for the clustering by school. The overall OR for the rate of ‘always practising safe sex’ for follow-up compared with control students was 1.18 (95% CI 0.85 to 1.64; p = 0.33). In other words, the prevalence of safe sex was 18% higher among students in the intervention year than among students in the control year. The difference was not significant, as the study was not powered to show a difference in outcomes, but the results are suggestive of a potential positive outcome for the intervention.

As there were more young men in the control group than in the follow-up group, the model was fitted separately for males and for females. Among males, the OR was 1.07 (95% CI 0.67 to 1.73; p = 0.78), whereas among females the OR was 1.37 (95% CI 0.85 to 2.21; p = 0.20). Therefore, there was a larger impact on the safe sex practices of young women than of young men. The interaction between gender and study group was statistically non-significant (p = 0.48), again because the study was not powered to show this. The difference in magnitude of the OR was considered scientifically significant.

In our statistical analysis plan (see Report Supplementary Material 5), we proposed analysis of whether or not there were any differences in primary outcome according to exposure to the intervention (PSs, students reached by PSs and unexposed students). The subgroup sample size for all outcomes was too small to offer a reliable picture. We present the analysis in Appendix 4 for information, but suggest a high level of caution in any interpretation of the data.

Information on the proposed composite primary outcome relevant to a future sample size calculation

The ICC was estimated from models fitted to the composite primary outcome with a random effect for school and no fixed effects. Owing to the small sample size, these ICC estimates were imprecise, with wide CIs. The ICC estimated from the model including all students in the follow-up and control groups was 0.0198, whereas the individual models for the follow-up and control groups estimated the ICC as 0.0066 and 0.0332, respectively. Although set out as a study objective, we opted not to propose a sample size calculation for a definitive trial based on our estimate of the effect on primary outcome from this study. This is consistent with recent guidance,95 as (1) our estimate is highly imprecise and yet any sample size calculation would be highly sensitive to it and (2) other sources of evidence should inform the specification of a difference to detect, including the value of the effect relative to intervention cost.

Knowledge

The knowledge questions asked in the questionnaire were as follows (with correct answers in brackets, note ‘do not know’ was considered an incorrect answer):

1. With a condom on, a guy should wait until the penis is soft before pulling out after ejaculation (false).

2. You can make an effective dental dam by cutting up a condom (true).

3. If you get chlamydia you will know because you will have symptoms (false).

4. It is against the law for two 15-year-olds to have sex with each other (true).

5. If someone aged < 16 years goes to the doctor for a STI test or pregnancy test, the doctor has to tell their parents (false).

6. All STIs can be treated with antibiotics (false).

7. A girl cannot get pregnant if the guy pulls out (withdraws) before ejaculation/cumming (false).

8. If you use a lubricant with a condom it should be oil based [e.g. Vaseline^® (Unilever UK Limited, Surrey, UK), massage oil] (false).

9. A girl’s clitoris is inside the vagina (false).

The ‘where to go’ knowledge questions were as follows:

1. where to get free condoms

2. where to get emergency contraception

3. where to get free contraception

4. where to get advice on sex.

The Cronbach alpha of the knowledge question bank as a whole was 0.70, whereas for the ‘where to go’ question bank it was 0.86.

Study group

The response rates to the knowledge questions were similar across the study groups, ranging from 90% to 98% at baseline, from 88% to 96% at follow-up and from 91% to 97% among the control students.

Figure 13 shows the percentage of the control and follow-up groups with the correct answer for each of the knowledge questions listed above. Lines between control and follow-up group are drawn for ease of comparison and do not indicate change over time. The graphs show that knowledge was generally higher in the follow-up group for both knowledge and knowing where to go for contraception and advice. Of particular note is knowledge question 2, which 8% of control students answered correctly, compared with 25% of follow-up students, and knowledge question 5, which 51% of control students answered correctly, compared with 64% of follow-up students. Both of these relate to key intervention messages. Correct responses to question 4 showed a decrease from the control group (72% correct) to the follow-up group (66%), which may have been due to some confusion around the term ‘sex’. Specifically, PSs were trained to think more broadly about what counts as sex (i.e. it is not just vaginal intercourse) and this may have led to confusion about which types of sex were against the law. This will be addressed in the terminology used for sex in the definitive trial questionnaire.

FIGURE 13.

Rates of correct answers to (a) knowledge and (b) ‘where to go’ questions for the control and follow-up groups. Lines are drawn for ease of comparison and do not indicate change over time, each line represents an individual question.

Across the nine knowledge questions, the proportion of students getting at least six questions correct was 25% in the control group and 36% in the follow-up group.

Attitudes and norms

A comparison of attitudes and norms across follow-up and control groups is shown in Figure 14. The items were as follows (the positive attitude responses are in brackets):

1. It is OK for someone who is not your boyfriend or girlfriend to ask you to send a nude pic of yourself (disagree).

2. The sex shown in pornography is similar to sex in real life (disagree).

3. Keeping a partner happy is a good reason to have sex (disagree).

4. It is not OK to have sex with someone who is very drunk (agree).

5. As long as you use a condom some of the times you have sex, your risk of getting a STI is very low (disagree).

FIGURE 14.

Percentage of (a) control and intervention students reporting positive sexual attitudes and (b) students reporting a positive change in attitudes by intervention exposure. Each line represents an attitude. Lines are drawn for ease of comparison and do not indicate change over time.

The response rates to the attitude and norm questions were similar across the study groups, ranging from 96% to 98% at baseline, from 95% to 96% at follow-up and from 96% to 97% among the control students.

The rates of adherence to positive attitudes and norms were slightly higher among follow-up students than among control students, with the exception of question 1, which was lower for follow-up students. The Cronbach alpha of the attitude question bank, as a whole, was 0.52.

We found no difference between follow-up and control students with regard to overestimation of the proportion of sexually active students in their year group (half of both follow-up and control students thought that more than one-third of students in their year were sexually active; the actual figure was 20%).

Confidence

The confidence question bank asked, ‘How confident are you that you could:’

1. ‘get condoms of your own?’

2. ‘put a condom on yourself or a partner?’

3. ‘refuse to have sexual intercourse if they won’t use a condom?’

4. ‘recognise when a partner doesn’t want to do something sexual?’

5. ‘tell a partner that you don’t want to take part in a sexual activity?’

6. ‘tell or show a partner what sexual activities you like or dislike?’

Response options were very confident, quite confident, unsure, not very confident, and not at all confident. These were recoded as a binary variable: confident (very, quite) versus other. Follow-up students were asked questions 1–3 at both baseline and follow-up, whereas the other questions were asked only at follow-up. The response rates to the confidence questions were high across the study groups, with 100% completion of the baseline questions, 86–98% completion at follow-up and 98–99% completion among the control students. The Cronbach alpha for the question bank, as a whole, was 0.77.

Generally, confidence was similar or slightly lower among students at follow-up than among control students. An exception was question 3 – refusing to have sex if a partner will not use a condom – for which follow-up students were more confident than control students (73.4% of follow-up students confident vs. 67.7% of control students confident).

Relationship well-being

A bank of items asking about relationship well-being was designed specifically for the STASH study, after we failed to identify any suitable measures of relationship well-being in young people. PCA was carried out on the seven relationship well-being indicators to assess whether or not a summary score could be constructed. The relationship indicators were (with the answer representing a healthy relationship in brackets):

1. They get very angry with me (not often/never).

2. We do sexual activities that I do not feel comfortable with (not often/never).

3. They get jealous and want to know what I’m doing and who I’m with (not often/never).

4. I tell them how I really feel (often/all the time).

5. I feel happy when we are together (often/all the time).

6. I feel comfortable talking about sexual things with them (e.g. whether to do something sexual, using condoms, sexual activities I like) (often/all the time).

7. They respect my opinions and ideas (often/all the time).

Only students who were in a relationship were asked to answer the questions. The response rates for the relationship indicators were similar across the study groups, with a 97–100% completion rate at baseline, a 98–99% completion rate at follow-up and a 98–99% completion rate among control students.

The PCA showed that there were two clusters of questions: questions 1–3 and questions 4–7. Together, these two clusters explained > 63% of the variability in the data and this was consistent whether the analysis was carried out on the control data alone, the follow-up data alone or both combined. The Cronbach alpha for the first measure (questions 1–3) was 0.66, whereas for the second measure (questions 4–7) it was 0.79. The question bank overall had a Cronbach alpha of 0.66. Future validation work on the measure could consider dropping the first subscale (items 1–3) to give a four-item measure of positive relationships. This four-item measure correlated as expected with distress about sex life [those reporting higher well-being were less likely to report distress about their sex life: OR 0.882 (95% CI 0.788 to 0.987); p = 0.029]. Interestingly, those with higher relationship scores were less likely to report safe sex (OR 0.869, 95% CI 0.786 to 0.961; p = 0.006), perhaps because they were in a longer-term relationship and were using another form of contraception.

As well as summarising the individual indicators, we included these two subscales of relationship well-being. These were calculated as a sum of the constituent question scores, with the question options ‘all of the time/often/sometimes/not often/never’ being given scores of 0–4 for the first (negative) measure and 4–0 for the second (positive) measure. The range for each could therefore be from 0 (unhealthy) to 12 (healthy) for the negative measure and 0 (unhealthy) to 16 (healthy) for the positive measure.

Generally, the rates of the indicators of a healthy relationship among follow-up students were similar to, or slightly higher than, rates among control students. The subscales of relationship well-being were very similar between the groups, with the control mean for the negative scale being 9.1 (SD 2.5), the follow-up mean being 9.1 (SD 2.7), and the control and follow-up means for the positive measure being the same, at 13.5 (SD 2.8).

Other secondary outcomes

Scores on the SWEMWBS measure of mental well-being101 had response rates of 99% at baseline, 90% at follow-up and 98% among the control students. Well-being scores were similar between the study groups [control mean score 21.0 (SD 4.6), follow-up mean score 21.3 (SD 4.2)]. As one of our hypothesised outcomes for the trial was increased well-being among PSs, we looked at differences in well-being by exposure (PSs, exposed students and non-exposed students) but found no differences in scores [mean total score for PSs 20.8 (SD 4.7), mean total score for exposed students 20.8 (SD 4.6) and mean total score for unexposed 21.2 (SD 4.7)].

The response rate for distress about sex life was 92% at baseline, 91% at follow-up and 95% among controls. The proportion of students distressed about their sex life was low across the study population, with 8.2% reporting distress in the follow-up group compared with 10.3% in the control group.

The response rates for the social media use questions were 90–92% at baseline, 90–91% at follow-up and 94% among controls. Cronbach’s alpha for the question bank was 0.78. Rates of using social media to view sexual images or to send, receive or be asked for nude or sexual pictures or videos were slightly lower in the follow-up group than in the control group. Rates of sharing a nude or sexual picture of someone else were very low and almost identical between the control (5.2%) and follow-up (5.0%) groups. The proportion of students reporting that they had used the internet or social media to find information about sexual health was slightly higher in the follow-up group (18.4%) than in the control group (15.4%).

The response rates to the questions about finding it easy to talk to parents, friends and partners about sex were 90–97% at baseline, 89–96% at follow-up and 93–96% among controls. The Cronbach alpha for this question bank was 0.5, suggesting poor correlation between comfort communicating with parents and communicating with friends and partners. The rates of students finding it easy to talk about sex was very similar in the control and follow-up groups (parents: control 17.6% vs. follow-up 19.3%; friends: control 58.3% vs. follow-up 59.5%; partners: control 30.6% vs. follow-up 28.9%), with students being most likely to talk to friends.

Questions assessing sexual competency at first sex were asked of the control group, but not of the follow-up group. This was an adaptation of a measure used in the National Survey of Sexual Attitudes and Lifestyles. 99 These measures were summarised as being the right time, both partners being equally willing, contraception being used and a positive reason for having had sex. The response rates were around 94% for these items. PCA showed that there were two clusters among these measures, explaining 70% of the variance in the data. The first cluster was the right time and equal willingness, and the second cluster was use of contraception and a positive reason for having sex. The Cronbach alpha was 0.53 for the first cluster and 0.63 for the second cluster. These may therefore be useful for taking forward as overall measures of sexual competence in the definitive trial. At follow-up, we asked the item on willingness and regret, plus a new one on alcohol and drug use. These items were asked about last sex rather than first sex (to increase the potential number of participants for whom this question would be relevant). The Cronbach alpha for these three items was 0.66, suggesting good utility in a definitive trial.

Primary outcome (behaviour)

• The suggested primary composite outcome for the definitive trial is ‘always practising safe sex,’ defined as ‘no condomless (vaginal or anal) sex in last 6 months’.

• There was an increase in odds of the (modified) primary composite outcome of 18% following the intervention (compared with the previous year’s control group), with a larger effect among girls.

Knowledge

• High response rates and high Cronbach’s alpha for the question banks.

• Knowledge generally increased regarding both sexual health and where to go for contraception and advice, over baseline and compared with the control group.

• There seemed to be some confusion around the law regarding 15-year-olds having sex with each other, which needs to be addressed in the delivery of the intervention.

• Likely to be a useful measure in the definitive trial.

Attitudes

• High response rates and moderate Cronbach’s alpha for the question bank.

• Attitudes were generally observed to be more positive following the STASH study intervention than among control students.

• May be a useful measure in the definitive trial.

Confidence

• Generally high response rates but slightly lower among students not exposed to the intervention. High Cronbach’s alpha for the question bank.

Relationship well-being

• Very high response rates and good Cronbach’s alpha, particularly for the ‘positive’ relationship well-being subscale.

• Students had similar or slightly higher rates of healthy relationship indicators in the follow-up group than in the control group.

• The composite relationship well-being scores were almost identical, both between the study groups and across exposure groups among follow-up students.

• The relationship well-being measures may not be useful in the definitive trial, although the positive composite measure, in particular, may be useful more generally as an assessment of relationship well-being in research.

Other measures

• Response rates were generally high for these measures.

• Well-being scores were similar between study groups and by exposure.

• Rates of distress about sex life were low and slightly lower among follow-up students than control students. There was little difference by exposure.

• Rates of using social media or internet for positive purposes were slightly higher following the STASH study intervention than among controls, whereas use for negative purposes was slightly lower following the STASH study intervention than among controls.

• Students found it easier to talk about sex with friends than parents or partners, and there was little difference between the study groups.

• Sexual competency may be usefully assessed by two measures: one combining the right time to have sex and equal willingness, and the other combining use of contraception and a positive reason for having sex.

• Well-being scores, distress about sex life, social media use and ease of talking about sex did not show strong utility for the definitive trial. Sexual competency at last sex may have some utility.

Incremental cost-effectiveness ratio, cost-effectiveness acceptability curve and cost-effectiveness plane

The incremental cost-effectiveness ratio for the intervention compared with current practice from the health-care cost perspective is £1303 (–12.77/–0.010) per 1-point gain in utility. Including out-of-pocket costs, it is £1715 (–16.95/–0.010). This is dominated by the control group if the cost of the STASH study intervention is introduced (additional cost per student of £24.99 and a utility loss of –0.010). The cost-effectiveness plane is reported in Figure 15. As most of the values sit in the south-west quadrant, where the intervention costs less but also results in a lower utility, the resulting cost-effectiveness acceptability curve (Figure 16) is difficult to interpret. There is a 93% probability that the intervention is cost-effective for very low values (close to £0) of willingness to pay for a utility, but a 2% probability for a willingness to pay > £6500. The probability of cost-effectiveness decreases as willingness to pay increases because, as willingness to pay for utility increases, the reduction in utility for the intervention year plays a greater role in the probability of cost-effectiveness than in the cost savings.

FIGURE 15.

Cost-effectiveness plane for all costs, including the cost of STASH study, out-of-pocket costs and NHS costs for intervention year compared with control year.

FIGURE 16.

Cost-effectiveness acceptability curve of NHS costs, out-of-pocket costs and intervention costs for intervention year compared with control year. Int, intervention; OOP, out of pocket.

If the proposed primary outcome of ‘always safe sex’ was used instead of the CHU-9D, the cost per additional person having ‘always safe sex’ would be £1315, assuming that 85% of students had ‘always safe sex’ in the control group and 87% of students had ‘always safe sex’ in the intervention group, and a cost per student of the intervention of £24.99 compared with the cost savings.

Objective 1

Finalise the design of a school-based STI prevention intervention, in which influential PSs use online social networks and face-to-face interactions to influence norms, knowledge, competence and behaviour, and promote use of sexual health services.

We met the first objective via reviews of literature and sex education resources; intensive consultation with young people and professional stakeholders (patient and public involvement); co-production and optimisation of a website of curated resources and a training manual; and an intervention pilot. Intervention development work identified key motivations for schools in terms of meeting national targets for health and well-being, and providing leadership opportunities for senior students. Our consultations and pilot affirmed that the peer nomination process used in ASSIST would be palatable to students and staff, but that we might need to overinvite students (25% of the year group rather than 17.5%) to the recruitment meeting, to train 15% of the year group as PSs (the proportion deemed critical in ASSIST). 67

In response to discussions with stakeholders regarding the sensitivity of the topic, we developed additional nomination questions focused on trust and listening skills to pilot alongside the original ASSIST questions (focused on respect and leadership). Most development activities contributed to the design of a curated set of shareable digital resources (housed on the STASH study website) and a bespoke training manual. Over various iterations, the STASH study website became increasingly visual and decreasingly reliant on text and signposting to other websites (as students in the pilot rarely clicked on these links).

The PS role crystallised incrementally in response to ideas from young people and stakeholders, as well as learning from the pilot. The role was eventually clarified as information giving, attitude influencing, promoting the website and signposting to sexual health services. Professional stakeholders were clear that the PSs would require training on handling sensitive disclosures and that they should be advised to avoid a counselling role. A PS charter was created to signal to PSs the responsibility inherent in the role.

The STASH study sought to adapt to contemporary modes of social interaction by introducing a social media option for dissemination of messages. A review of current platforms identified Facebook as the only one to meet the project requirements of closed and private groups and ability to interface with a website. Consultations with young people established that Facebook was not a first choice of platform for young people, but nevertheless it was widely used and broadly acceptable. A key design feature, initiated in response to concerns of teachers, was to stipulate that the STASH study trainers would monitor and support online activities via membership of each PS Facebook group. Development activities, and particularly the pilot, highlighted the need to increase awareness of the STASH study across school staff and year group students, but suggested a flexible approach, allowing each PS team to decide a course of action appropriate to their school.

Objective 2

Assess the recruitment and retention of PSs, as well as feasibility and acceptability of the intervention among PSs, participants and key stakeholders.

We anticipated that role uptake would be lower than in ASSIST57 and the ASSIST extensions128 because we were delivering the intervention on a more sensitive topic to a year group undertaking public examinations. Across the six schools, 52% of nominated students (63% of those attending the recruitment meeting) were trained. Variation across schools (a high of 77% in school 5 and a low of 33% in school 6) was consistent with process evaluation findings of variable levels of school buy-in and support of the intervention, as well as variation in the quality of the ‘pitch’ given by trainers at the recruit meeting. Role retention was very high, with 97% of PSs fulfilling the role (defined as posting three or more STASH study messages on Facebook or having two or more conversations and attending two or more follow-up sessions). The intervention was highly acceptable to PSs (85% said they liked the role), more so to young women (94%) than to young men (71%).

The majority of PSs liked the STASH study website and found it easy to navigate and share messages. The STASH study appeared to achieve a good balance between allowing flexible engagement and ensuring trainer monitoring and support.

Over 70% of PSs said that they found the STASH study more relevant, helpful and interesting than their usual SRE lessons. Of the other students engaging with STASH study messages via the PSs (exposed students), 74% thought that the way the STASH study was run was acceptable and 78% thought that the information provided was acceptable. Again, acceptability was higher for young women than for young men. We did not interview parents directly, but only one student reported that their parent was unhappy about them being a PS.

The eight staff members interviewed were positive about the ethos of the STASH study, the leadership opportunities it afforded, the rapport between trainers and PSs, and the high-quality training. Staff acknowledged the additional workload involved in delivering the STASH study intervention and the challenges for students in terms of missing classes for training and follow-up sessions. One teacher raised concern about school capacity to deliver the intervention. His concerns related as much to the burden of the evaluation as the intervention, and partly reflected that the school was undergoing a major transition at the time.

Objective 3

Assess the fidelity and reach of intervention delivery by trainers and PSs, including barriers to, and facilitators of, successful implementation.

The STASH study was implemented with excellent fidelity. All nomination exercises, recruitment sessions, external trainings and 35 of 36 follow-up sessions were delivered as planned. Attendance at follow-up sessions was high (average 80%), particularly given that this was an examination year. The PSs were highly active on social media, sharing 1343 messages from the STASH study website to their Facebook groups over the course of the intervention period (5 weeks in three schools and 10 weeks in the other three schools). On average, PSs reported six face-to-face conversations about the STASH study with their friends.

Social network analysis showed that students trained as PSs were well positioned across their year group friendship networks. They had greater potential to reach across their year group than students who were not nominated (5.3 incoming friendship ties vs. 3.5 for non-nominated students). PSs were also well distributed across friendship clusters, ranging from presence in one-third of friendship groups in one school to four-fifths in another. Across all schools, PSs had direct friendship ties to one-third of students.

The intervention had good reach across the S4 year group, with 58% (n = 611) of students at follow-up reporting some level of exposure (trained as a PS, visited the website or member of a Facebook group, or had a conversation with a PS). Young women were more likely to report exposure than young men (63% vs. 53%, respectively). Educational attainment and sexual experience at baseline both appeared linked to likelihood of exposure, but not socioeconomic deprivation, ethnicity, religiosity or sexual orientation. Key facilitators identified were buy-in and support from schools; visibility of the STASH study intervention to the rest of the year group and school; good rapport at training; high-quality training; regular direct communication between trainers and PSs via Facebook, including reminders for follow-up meetings; the use of private and closed groups to give PSs control over message sharing; and well-planned and regularly spaced follow-up sessions that included an interactive activity, as well as progress check.

Key barriers identified were reliance on schools for scheduling of activities; academic pressures, such that follow-ups scheduled during class time were difficult for some teachers and students; technical issues with the website (these were small in number); infrequent use of Facebook among some students; low levels of awareness of the intervention across the rest of the school; low levels of intrinsic motivation of non-PSs to engage with PS activities; and poorer engagement by young men compared with young women, probably due to higher embarrassment and less maturity in discussing sexual health issues.

Objective 4

Refine and test the programme theory and theoretical basis of the intervention.

We used the feasibility trial to test the STASH study programme theory, leading to a refined theory in which mechanisms of change, conditions of success and context were more clearly articulated (Figure 17). We also explored ways to visualise the theory to reflect the dynamic interaction of intervention components in a format accessible to stakeholders and delivery partners (Figure 18). Figure 17 shows the version of the programme theory following the feasibility trial, but prior to the full analysis of the results. Figure 17(a) shows the full logic model from problem identification to outcomes and Figure 17(b) ‘explodes’ the intervention component to detail the conditions of successful implementation and mechanisms of action. In comparison to the post-pilot, pre-feasibility trial programme theory, the version showing in Figure 17 has a number of key refinements. It more clearly sets out the theoretical underpinnings of the trial at micro, mid and macro level. It articulates the mechanisms of change for key actors in the intervention and it more clearly establishes the conditions of successful implementation, linking these to their respective intervention components. In addition, we refined our expectations of the key intermediate and end outcomes. We added competence at last sex (equally willing, no regret, no alcohol) and ease of talking about sex as intermediate outcomes, we moved relationship well-being to the end outcomes, and we added confidence and well-being among PSs as an end outcome. These refinements to the end outcomes emerged from discussion – with the TMG, TSC and project stakeholders – about the importance of recognising the wider potential impact of the trial beyond STI prevention. Although we captured qualitatively the broader psychosocial benefits for PSs, our measures of well-being (at least in this small sample) did not pick this up. Similarly, our new measure of relationship well-being was unable to detect a difference between intervention and control groups (see Objective 6).

FIGURE 17.

The STASH study programme theory: feasibility trial stage. (a) Full programme theory; and (b) details of conditions of successful implementation and mechanisms of action. TDF, theoretical domains framework. Reproduced from Forsyth et al. 62 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

Reproduced from Forsyth et al. 62 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure.

FIGURE 18.

An alternative visualisation of the programme theory designed primarily to explain the STASH study to stakeholders (using an animation to highlight different stages). The visualisation highlights the conditions and mechanisms relevant to each intervention stage.

Our analysis of the feasibility trial data largely confirmed that the programme theory presented in Figure 17 adequately captured the preconditions of success and contextual factors influencing effectiveness. In general, the key conditions were met, with variability across schools. Some conditions were harder to meet than others.

The feasibility trial was able to interrogate the mechanisms of change to some extent (see Figure 17). Table 18 shows the mechanisms in a separate box, alongside key evidence from the study relating to that mechanism and suggested revisions to the hypothesised mechanisms for a future version of the programme theory. The more challenging mechanisms were those dependent on PS delivery of activities (the aspect of the intervention over which we had least control). In particular, there appeared a limit to which the PS felt socially validated in the role by their peers (they appeared neither validated nor disproved of). A key finding was that there appeared a limit to which non-PSs were interested in messages and saw them as relevant. Although the training led to enthusiasm and awareness of sexual health issues among PSs, it appeared difficult for them to generate the same level of interest among their friends via brief messages and conversations. As these conditions were only partially met, the extent to which the intervention supported autonomy and intrinsic motivation (a purported mechanism of change) is unclear.

Diffusion of innovations theorists suggest five stages in the adoption of new behaviours (knowledge, persuasion, decision, trial and adoption). 129 Owing to variation in the pace at which people move through the stages of adoption, considerable time can elapse between first and last adopters, even for clearly advantageous behaviours. 24 The theory suggests that the rate at which a new idea or behaviour is adopted depends, among other things, on the past experiences and needs of potential adopters. 59,129 A challenge for sexual risk reduction messages in mid-adolescence is that only a proportion of students are sexually active [approximately 20% reported vaginal or oral sex in this study (see Table 11)]. The ‘potential adopters’ of new messages on sexual behaviour may be relatively small (particularly as some of those who are sexually active are already practising safer-sex behaviours). The STASH study messages were wide-ranging in recognition of this variation among the target group; even so, students who were not sexually active may have simply decided that the STASH study was less personally relevant to them.

Given that sexual experience and ‘readiness’ for messages about sexual health will always be variable within a S4 student year group, it is worth reinterrogating our programme theory to ask whether or not supporting increased autonomy and intrinsic motivations is (1) realistic and (2) an essential mechanism of change. The architects of self-determination theory, Ryan and Deci,83 describe how social and environmental factors can facilitate or undermine intrinsic motivation (cognitive evaluation theory,83 a subtheory within self-determination theory). In the STASH study, PSs were able to create their own groups, and to choose which messages to share, how and with whom, and non-PSs had full autonomy over if and how they engaged with messages. The practice and ethos of the STASH study was thus autonomy supportive, particularly in comparison with more controlled didactic classroom-based methods. What seems less clear is whether or not this autonomy led to intrinsic motivation. Interestingly, the top two motivations cited for PSs continuing in the project were ‘learning information’ and ‘fun’ (see Chapter 5, Motivation to participate), and not extrinsic rewards, such as benefit to CV, indicating some level of intrinsic motivation among the PS group. Among non-PSs, intrinsic motivation to engage with the topic appeared low, but it is worth questioning whether or not this is a realistic expectation across the year group, and whether or not it is essential as a mechanism of change. It is possible that norms can be diffused across a peer network with relatively low levels of intrinsic motivation to engage with the intervention. Therefore, in the next iteration of the programme theory, increased intrinsic motivation should perhaps be limited to PSs and not the year group as a whole. In Recommendations for future evaluation, we also suggest ways to enhance non-PS engagement in the intervention.

Objective 5

Enhance understanding of the potential of social media, when used by influential peers, to diffuse norm change and facilitate social support for healthy sexual behaviour.

The students interviewed in the STASH study engaged with social media for primarily social reasons, using a range of platforms. Social ‘gossip’ unfolded seamlessly across online and offline spaces, such that not keeping up online could mean missing key developments in social events. Non-PS student reactions to receiving STASH study posts via Facebook ranged from openness and interest to indifference. Some saw the opportunity to engage online as less awkward, whereas others preferred face-to-face conversation or a mix. To the extent that non-PSs registered and responded positively to messages, it appeared important that messages were sent by someone they knew and seen by others in the group, such that there were opportunities for discussion. In other words, the offline social context was key in both legitimising and augmenting online messages.

The process evaluation confirmed findings of the intervention development stage in suggesting a strong preference for humorous, colourful, eye-catching images, with as little text as possible. The opportunity to share messages within a private group rather than on a general profile page was key to feeling comfortable, and PSs were discerning about who they invited. Issues of stigma and unease at being linked to sexual health content, identified in previous social media interventions in sexual health,130,131 did not emerge as particular issues in the STASH study. This is most likely as a result of the use of private groups. A possible cost of this privacy is that STASH study interactions may have been viewed as parallel to, rather than embedded in, PSs’ usual online social interactions.

The presence of a trainer within the private groups appeared reassuring for many, although only a small number felt the need or wished to engage with them in this forum (primarily young women). In terms of practical considerations, the use of social media necessitates hiring trainers who are comfortable using it, and there are challenges in establishing a design that is sufficiently flexible to keep pace with constantly evolving platforms and technology.

Objective 6

Determine key trial design parameters for a possible future large-scale trial, including recruitment and retention rates, strategies, outcome measures, ICC and sample size.

All six schools were retained in the study. Response rates for the student surveys were 80% for the controls, and 82% at baseline and 79% at follow-up. Among the intervention year, 81.2% of students completing the baseline questionnaire also completed the follow-up questionnaire. Response rates were slightly lower than other recent school-based studies. 128,132 As parental opt-out (n = 6) and student non-consent (n = 3) were low, the response rate primarily reflected the number of students who attended school on survey days, and made it to the class in which the questionnaire was administered. There was a suggestion from teachers that the class lists (on which the denominator was based) were not always up to date and included students who did not regularly attend school. Our finding that students who did not complete the follow-up questionnaire were more likely to be male, from more deprived backgrounds and with lower educational entertainment highlights both the importance of conducting the STASH study in schools (where the fullest range of young people can be reached) and the methodological challenges of doing so.

We investigated a number of candidate primary outcomes, identifying a composite measure of ‘always practising safer sex’ as a potential primary outcome for the trial. We defined this as ‘No condomless (vaginal or anal) sex in the last 6 months’. A future intervention would seek to increase the proportion of students practising safer sex either by not having sex (the safest option) or by having sex but always using a condom. The odds of reporting always safer sex in this study were in a direction suggestive of benefit, although the study was not powered to demonstrate a significant difference. Like all self-report measures of sexual behaviour, this outcome is subject to social desirability bias and, to a lesser extent, recall bias. A further limitation is that it does not include unprotected oral sex and also does not account for situations in which two young people in a long-term monogamous relationship may have stopped using condoms following negative STI tests. These behaviours will be measured in the outcome questionnaire.

Delayed initiation or no sex and condom use are the most common outcomes in youth-targeted sex education interventions,3,31 but these are usually reported separately. Some studies use a measure of consistent condom use, whereas others focus on condom use at last sex. 3,31 For instance, the school-based ‘Safer Choices’ intervention had three separate primary outcomes: (1) delayed initiation of sexual intercourse, (2) frequency of sex with a condom among those reporting intercourse and (3) the number of sexual partners with whom students had condomless intercourse in the last 3 months (among those reporting intercourse). 133 Our composite outcome captures the first and second of these dimensions within a single measure. In a subsequent evaluation, we would not propose to measure the third dimension, because the rate of partner change within our time frame for follow-up is too low (an average of one partner) (see Chapter 7, Descriptive statistics for primary outcomes). Reviews of condom use measures in evaluating human immunodeficiency virus prevention show substantial heterogeneity in measurement, with no trend towards standardisation over time. 134 Condom use at last sex is the most common measure, followed by consistency of condom use; however, current recommendations do not suggest clear superiority of one over the other. 134 With respect to youth-targeted interventions, a challenge is that behavioural outcome measures can present an inconsistent picture. For instance, the STAND study found an increase in condom use at last intercourse, but negative changes in other outcomes (e.g. condom use in anal sex, unplanned pregnancy and STIs). 41

In terms of secondary outcomes, our question banks on knowledge and attitudes had high response rates. Reliability was good for the knowledge question bank (Cronbach alpha 0.7) and medium or low response rates for the question bank on attitudes (Cronbach alpha 0.52). Knowledge was higher, and attitudes more positive, in the intervention responses than in the control, suggesting potential benefit of the intervention (again, not powered for significance). The question bank on confidence had good response rates and reliability (Cronbach alpha 0.77); however, we found an inconsistent picture (control year higher on some items and follow-up on others), possibly reflecting that individual assessment may vary according to experience of trying a behaviour (such as negotiating condom use). If taken forward to a future trial, these measures should be limited to those with sexual experience. Our newly designed measure of relationship well-being had high reliability (Cronbach alpha 0.79) for the positive subscale in particular (measuring openness, happiness, comfort and respect) and correlated with related constructs (e.g. distress about sex life), but did not differentiate between control and intervention groups. This needs to be considered in the decision about whether or not to take this forward. Our adapted measure of sexual competence99 (willingness, regret, use of alcohol or drugs) has medium reliability (Cronbach alpha 0.66) and merits inclusion, but condom use should be added to the measure (as a key dimension of competence). The measure should also be asked of most recent sex, rather than first sex, to increase the number of students for whom the question is relevant.

Our analysis of potential effect modifiers suggested several that might affect the primary outcome and should be retained: FSM, ethnicity, educational attainment, religiosity, peer group sexual behaviour and alcohol use. Other modifiers did not demonstrate an association with the primary outcome and could possibly be omitted from the questionnaire going forward. These include self-regulation, parental monitoring, sexual attraction and importance of social media to social life.

Although a study objective, we ultimately opted not to propose a sample size calculation for a definitive trial at this stage. This is partly because the estimate of effect from this small sample is imprecise and yet any sample size calculation would be highly sensitive to it. The recent DELTA2 recommendations state that pilot trials are typically too small to form a reliable basis for a sample size calculation, although they can indicate a realistic target difference. 95 At the stage of designing a full-scale evaluation, we will follow the recent DELTA2 guidelines to calculate the sample size, basing it on a wider range of sexual health interventions and on an assessment of what would be considered meaningful to key stakeholders. 95 Evidence from previous school-based sexual health interventions suggests that any differences in behaviour are likely to be small;3 for instance, the effective and widely adopted ‘Safer Choices’ intervention found estimated OR effect sizes (intervention vs. control) of 0.83 for sexual initiation (non-significant), 0.63 for frequency of sex without a condom in last 3 months (p = 0.05) and 0.73 for number of sexual partners without a condom in last 3 months (p = 0.02). 133 In our case, an OR of 1.2 may be realistic for an outcome of ‘always safer sex’. In terms of what is considered meaningful, a drop in the primary outcome of a risk behaviour may be less important to educational stakeholders than improved well-being, and health sector stakeholders may be happy to accept a small change in primary outcome when other secondary outcomes (such as knowledge about where to seek help and self-efficacy in condom use) improve. Such considerations are key, particularly when only small changes in the primary outcome are realistic. A further consideration is the heterogeneity in measurement of primary outcomes for youth-focused STI prevention. The DELTA2 guidance recommends performing a sensitivity analysis to consider the effect of uncertainty in the estimate of the target difference. This study provides useful information to contribute to our review of existing literature: a control proportion of 0.854 for practising ‘always safer sex’ and an ICC estimate (based on follow-up and control students) of 0.0198.

Objective 7

Determine the key components of a future cost-effectiveness analysis and test data collection methods.

The total cost of the STASH study intervention was approximately £42 per student or £4258 per school for a year group of approximately 100 students. This varied from £23 per pupil to £93 per student, dependent on year group size. This suggests that the intervention may be more costly to deliver in smaller schools, a fact that should be borne in mind for future evaluation and roll-out. The question bank on sexual health-related resource use was completed well, with relatively low levels of missing data. We established that the CHU-9D was unsuited to calculating QALYs in the context of the STASH study because it was not associated with the primary outcome of always practising safer sex. This analysis confirmed our lack of theoretical expectation that the intervention would have an effect on the domains measured by the CHU-9D, making the CHU-9D an inappropriate and insensitive measure for any effects on well-being that might be achieved by the STASH study. Lower CHU-9D scores in the control arm made the resulting incremental cost-effectiveness ratios, cost-effectiveness acceptability curves and cost-effectiveness planes difficult to interpret. A possible approach for the economic evaluation in a future trial of the STASH intervention could be to model long-term outcomes of STIs and unintended pregnancies related to increases in safer sex, accounting for the disutility of a STI and unplanned pregnancy.

Objective 8

Establish whether or not pre-set progression criteria are met and if a larger-scale trial is warranted.

The study set seven progression criteria regarding the feasibility and acceptability of the intervention and evaluation. We adopted a traffic light system (green, amber, red), with green targets as follows:

• In each of four schools, 60% of nominated students are recruited and complete the training.

• In each of four schools, 60% of PSs complete the training, send three or more messages or have three or more conversations, and attend two or more follow-up meetings.

• In each of four schools, 60% of PSs report that they ‘liked’ the role.

• In each of four schools, 60% of students who are exposed to the STASH study agree that the intervention was acceptable.

• No major acceptability issues raised.

• Less than 15% of PSs report that their parents or carers were unhappy about them being a PS.

• In each of four schools, achieve student response rates of > 70% at baseline and follow-up.

We met green targets with ease for all criteria bar the first. This first indicator (red) was discussed at length with the TSC, as stipulated in our protocol. At the trial outset, we agreed that in the case of amber or red, a mitigating strategy should be identified and continuation should be considered only if all other targets were green. Given that the green targets were easily surpassed for all other criteria, the TSC were supportive of continuation.

Although we did not achieve 60% role uptake among those nominated, we were still able to train 13% of the year group as PSs. This is just short of the 15% deemed critical in the popular opinion leader model of peer education on which ASSIST is based. 66 We achieved this partly by overinviting the top nominees (taking the top 25%, rather than the 17.5% invited in ASSIST). Nominees who took up the role received more nominations than those who declined to participate, suggesting that uptake was more likely among the more influential students. This suggests that we may have thrown the net a little wide and that inviting a smaller proportion to recruitment (for example 20%) might still achieve a critical mass of 15%, but with a higher proportional uptake among those invited. The number of friendship ties was broadly similar across those who did take up the role and those who did not take up the role, except that the trained PSs had higher two-step reach centrality (mean of 27 students reached in two steps vs. a mean of 24 students reached for those who did not take up the role). The PSs were well distributed across friendship clusters, with results similar to those in ASSIST. 40 PSs also had higher potential to reach across their network than non-nominated students, and were reasonably representative of their year group, being similar in terms of ethnicity, age, religiosity and sexual orientation, but slightly more academic and more likely to be eligible for FSM (a proxy for low-income household). PSs were slightly more likely to report some sexual experience at baseline, such that role uptake may have been related to perceived personal relevance of the topic and/or readiness to hear messages about sexual health.

The target of 60% was based on the experience of ASSIST and extensions of ASSIST67,128 [in which role uptake was high (> 90%)], but may have been set too high for the STASH study given the topic and age group of the students. Among younger students in ASSIST, the prospect of a 2-day training out of school provided a strong motivation, whereas in the STASH study this element received a mixed response, because of academic pressures. There is little evidence from peer-led sexual health interventions to guide decisions on an appropriate target for recruitment. Of five peer-led studies included in a recent European systematic review,32 none used peer nomination to select peer educators (two employed teacher selection, two asked for volunteers and one did not report the selection method). The USA-based STAND study41 is, to our knowledge, the only other youth-focused sexual health intervention to employ peer nomination. The STAND study41 was also a school-based study targeting 15- to 16-year-olds. Their peer nomination process identified 42 opinion leaders, of whom 21 were trained (50%), a very similar proportion to the STASH study. This suggests that 50% uptake may be a more reasonable target. Nonetheless, it is important to think about how we might improve uptake in the future. The target of 60% was far surpassed in one school (77%) and nearly met in two schools (58% and 56%), suggesting that more intensive work to prepare and support schools may be important. In establishing the appropriate proportion of nominees invited to the recruitment meeting, it will be important to identify the percentage (somewhere between 17% and 25%) that achieves the highest proportional uptake while still recruiting 15% of the year. Other possible mitigation strategies regarding the recruitment process are discussed in Recommendations for future evaluation.

Adaptation to the intervention

• Further strategies to enhance awareness of the intervention in the year group and among school staff should be developed. Strategies employed in this feasibility trial included announcements in registration class, assemblies, bulletins and staff meetings. The PSs were given flexibility to ‘announce’ the project according to their preference. In practice, the effectiveness of these activities depended on the support they received from school staff. Going forward, the intervention might supplement these activities via a standard trainer-delivered SRE session, which would give key STI prevention messages and introduce both the project and the PSs. Such a session would help generate a more supportive context for PS activities, reducing the ‘surprise’ element of sexual health messages appearing in student social media threads. It would also reduce the inbuilt inequality of exposure being dependent on proximity to a PS and increase the coherence60 of the intervention for participants (in understanding why only a proportion of the year received training). The cost implications in terms of additional trainer time would need to be considered.

• Uptake of the PS role could be improved in a number of ways. On average, 13% of nominees did not attend the recruitment meeting and so did not have a chance to consider participation. The importance of the recruitment session needs to be emphasised to schools, along with practical suggestions as to how they can support the process. For larger schools, in particular, it may be prudent to split the recruitment meeting into two sessions, on the basis that students may feel more comfortable in smaller groups. The second session would have an additional benefit of providing a mop-up for those who missed the first. We found that in addition to school support, the ‘pitch’ given at the recruitment meeting is very important. Equipping trainers to deliver a strong pitch, with a focus on the key motivators (identified as curiosity, benefit to CV and fun) may also help to increase uptake. Anonymous PS testimonies given in this study could be integrated into recruitment materials and if the STASH study is delivered in subsequent years within the same school, there would also be the option of inviting previously trained PSs to talk about their experience. This is likely to be both reassuring and motivating.

• The study established social media as effective at diffusing intervention messages when used in conjunction with face-to-face elements. At a practical level, the social media element needs to be ‘future-proofed’ to cope with evolving engagement in social media and technological advances. In addition, the appropriate level of control over social media use requires confirmation. Ideally, PSs would have flexibility to engage with STASH study messages using a social media platform of their choice, giving them greater ownership and potential to tailor to the social media preferences of their friendship group. The absence of any untoward incidents in our study suggests that some loosening of control and enhanced flexibility may be justifiable and beneficial; however, this will require careful consideration and monitoring. On a practical level, updates to the website should aim for simplicity for users and minimisation of technical issues.

• A challenge for the STASH study was the range and complexity of messages required to fully equip young people for positive sexual health and avoidance of STIs. This challenge was intensified by the wide range of sexual experience and existing knowledge across the year group, such that individuals differed in the messages they required and found relevant. Our literature review strongly supported a holistic intervention content (rather than a focus on STIs and risk)11–13 and flexibility for PSs to engage with messages according to their interest. Although beneficial in terms of ownership and engagement, this high level of choice was challenging for some PSs. In future, the intervention might supplement the curriculum with a set of streamlined, simple and memorable ‘basic set’ messages that PSs are encouraged to engage with as a minimum.

• Any future roll-out of the intervention will need to address the gender bias in engagement. A finer-grained analysis of motivations to participate might identify particular motivators for young men that could be used to strengthen the appeal for them.

• In terms of timetabling, the peer nomination process could be moved back to June (prior to summer break) so that the baseline questionnaire could be implemented as soon as schools return from the summer break. This would have an important additional benefit of giving more time for intervention delivery before the Christmas break. It is possible, although unlikely, that friendship networks change over the summer break, so this change may need to be piloted (e.g. by repeating the nomination survey in a single school).

• Our health economics analysis suggests that delivery in a small school is potentially more costly.

• In the longer term, the ideal implementation model for the STASH intervention could be an annual cycle of delivery for successive S4 year groups. Trained PSs moving into S5 and S6 (the final 2 years of secondary school) could support the PSs and deliver additional activities, such as supporting school drop-in clinics, facilitating SRE sessions for younger years and undertaking outreach. A wider implementation model such as this might facilitate embeddedness within schools and contribute to longer-lasting impact. After the study ended, the STASH trainers continued to work with PSs in several schools post intervention (e.g. to support the school drop-in clinic) and informal learning from this work will contribute to future discussion on intervention implementation.

• A significant alternative adaptation to the model would be to train S6 students (aged 16–18 years) as PSs, rather than S4. The S6 students could use social media and face-to-face conversation to diffuse messages to their peers in S6 (and S4 and S5 if desired, as senior students are often in mixed year group classes), as well as facilitating classroom sessions with S4 students. The project could be delivered in a 3-year cycle, in which S4 students initially taught by S6 PSs eventually become trained PSs themselves. This approach could potentially address some of the limitations identified in this study. First, the older age group would be more palatable to schools because there is less academic pressure in S6 (the main public examinations are undertaken in S5) and students tend to have free periods in which follow-up sessions could be scheduled. Second, there would be a greater proportion of sexually active students for whom the STASH study messages feel personally relevant, thereby potentially addressing some of the identified challenges around intrinsic motivation. Third, an older age group would not require such close control and monitoring of social media use, enabling students to engage more flexibly with different social media platforms. Finally, in later school years, young men tend to ‘catch up’ in terms of reaching a level of maturity conducive to full engagement with the role. Although it could be argued that S6 is too late to prevent unsafe or unhealthy first sexual experiences, the fact that the STASH study is intended to augment rather than replace sex education justifies waiting until a greater proportion of the target group perceives the messages as personally relevant. In an English educational context, this would translate to the lower sixth year group. In schools with a separate sixth form, the intervention may need to take place later in the school year, as new friendship groups will need time to establish.

Adaptation to data collection methods

• Administering the questionnaire was burdensome for schools and a future trial should consider ways to reduce this burden. A return to a paper-and-pen questionnaire is not suggested for reasons of data quality. 117,118 One option for a web-based survey, would be for the fieldwork team to carry mobile Wi-Fi ‘hotspots’, which would allow students to remain in class and complete the survey on their mobile phones. A pilot of this approach in another ongoing school-based study within our unit suggests that this is feasible (Equally Safe Research team, University of Glasgow, 2020, personal communication). A small number of tablets could be made available to students without phones or charge. This approach would reduce logistical difficulties and increase privacy. Survey administration may be easier for schools if there is a clearly identified back-up or support teacher for each contact teacher. This person should be present at all early planning meetings to ensure that they understand the study and can stand in effectively.

• The questionnaire was slightly long, both for students undertaking it and for administration within a school period. Reducing the length of the questionnaire should be straightforward to achieve, as analysis of the outcome measures identified a number of items that can be omitted. We have identified specific ways to improve questionnaire wording and filtering. We could also do more to work with teachers to identify language issues ahead of time and put in place measures to support students on a school-by-school basis.

• The questionnaire was experienced as intrusive by some students. This is, to some extent, inevitable in a survey on attitudes and behaviour related to sex. Although we were careful to explain the questionnaire topic in the participant information sheet, and at the start of the questionnaire, we could expand the introduction to the survey on the fieldworker script. A stronger explanation as to why we need to collect the data could be provided, focusing particularly on the questions that raised concern about confidentiality, such as the postcode and friendship module. Reassurances could be repeated throughout the questionnaire. At follow-up, an additional explanation should state the reasons for repeating the same questions, as higher levels of missing data at follow-up may reflect students feeling that they have already answered the questions.

• With respect to the PS questionnaire, judicious use of open-ended questions would allow us to collect further insight into varied perspectives across the PS cadre, augmenting the qualitative data collection.