Modifying the secondary school environment to reduce bullying and aggression: the INCLUSIVE cluster RCT

Article history

The research reported in this issue of the journal was funded by the PHR programme as project number 12/153/60. The contractual start date was in March 2014. The final report began editorial review in June 2018 and was accepted for publication in November 2018. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Russell M Viner is President of the Royal College of Paediatrics and Child Health. Stephen Scott reports grants from King’s College London during the conduct of the study. Chris Bonell is a member of Public Health Research Research Funding Board. Deborah Christie reports personal fees from Medtronic plc (Dublin, Ireland), personal fees from Pfizer Inc. (New York, NY, USA) and personal fees from ASCEND (Watford, UK), all unrelated to the submitted work.

Copyright statement

© Queen’s Printer and Controller of HMSO 2019. This work was produced by Bonell et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2019 Queen’s Printer and Controller of HMSO

Scientific background

The study protocol for this trial is available in full in Trials. 1 It is presented again here. Bullying, aggression and violence among children and young people are some of the most consequential public mental health problems apparent today. 2,3 The prevalence and harms of aggressive behaviours among young people makes addressing these a public health priority. 4–7 The World Health Organization (WHO) considers bullying to be a major adolescent health problem, defining this to include the intentional use of physical or psychological force against others. 5,8 This includes verbal and relational aggression that aims to harm the victim or their social relations, such as by spreading rumours or purposely excluding them. 9,10 Some definitions of bullying11 emphasise that bullying refers to abuse that is committed repeatedly over time and that involves a power imbalance between the perpetrator(s) and the victim. The prevalence of bullying among British young people (at around 33%)12 is above the European average,13 with approximately 25% of young people reporting that they have been subjected to serious peer bullying. 14 Cyberbullying, in particular, is rapidly becoming one of the most common forms of bullying. 15 There are marked social gradients: family deprivation and school-level deprivation increase the risk of experiencing bullying. 16 Bullying most commonly occurs in schools17,18 and prevalence varies significantly. 19–22

Being a victim of peer bullying has been associated with an increased risk of physical health problems;23,24 substance use;25–28 long-term emotional, behavioural and mental health problems;29–33 self-harm and suicide;34–36 and poorer educational attainment. 37,38 Students who experience physical, verbal or relational bullying regularly tend to experience the most adverse health outcomes. 39 There has also been evidence that childhood exposure to bullying and aggression may influence lifelong health through biological mechanisms. 23,29,40

The perpetrators of peer bullying are also at risk of a range of adverse emotional and mental health outcomes, including depression and anxiety. 13,19 Therefore, the prevention of bullying, aggression and violence is a major priority for public health and education systems internationally,3,41 with schools a key focus of policy initiatives to improve young people’s mental health and well-being. 42 In England,43 schools have a legal obligation to prevent bullying.

Bullying is often a precursor to more serious violent behaviours. One UK study44 of 14,000 students found that 1 in 10 young people aged 11–12 years reported carrying a weapon, and 8% of this age group admitted that they had attacked another with the intention of hurting them seriously. By the age of 15–16 years, 24% of students reported that they had carried a weapon and 19% had reported attacking someone with the intention of hurting them. 44 Interpersonal violence can cause physical injury and disability, and has been also associated with long-term emotional and mental health problems. Aggression refers to behaviour that is intended to harm, either directly or indirectly, another individual who does not wish to be harmed. 45 There are also links between aggression and antisocial behaviours in young people and violent crime in adulthood. 46,47 This is thought to result from low-level provocation and aggressive behaviours in secondary schools being educationally disruptive and emotionally harmful, and reducing educational attainments and later life chances, and therefore leading to more overt physical aggression over time. 48–50 The economic costs to society as a whole from bullying and violence are extremely high: the total cost of crime attributable to conduct problems in childhood has been estimated to be about £60B per year in England and Wales. 51

Reducing aggression, bullying and violence in British schools has been a consistent priority in public health and education policies. 52–54 The 2009 Steer Review50 concluded that schools’ approaches to discipline, behaviour management and bullying prevention vary widely and are rarely evidence based, and that further resources and research are urgently needed. There is, therefore, a pressing need to determine which interventions are effective in addressing bullying and aggression in schools, and to scale up such interventions to local and national school networks.

Whole-school-based interventions

A number of systematic reviews have assessed school-based interventions to address bullying and aggression. Interventions that promote change across school systems and address different levels of school organisation (i.e. ‘whole-school’ or ‘school environment’ interventions) are particularly effective in reducing victimisation and bullying in comparison with curriculum interventions. 55–57 ‘Whole-school’ or ‘school environment’ interventions are interventions that modify the systemic operations of schools, and they have been shown to have an impact on a range of health outcomes and risk behaviours. 57 A key element of these interventions appears to be increasing student engagement with school, particularly the most socially disadvantaged students, who are at the highest risk of poor health and educational outcomes. 58,59 Two trials have found that such approaches are associated with reductions in risk behaviours, including violence and antisocial behaviour. 60,61

The effectiveness of such interventions may be attributable to the way that they address bullying as a systemic problem meriting an ‘environmental solution’, rather than an individual student issue. 55 Whole-school interventions are also inherently universal in their reach and are likely to provide a cost-effective and non-stigmatising approach to preventing bullying. 56 This is in keeping with other evidence from the UK and internationally, which shows that schools promote health most effectively when they are not treated merely as sites for health education, but also as physical and social environments that can actively support healthy behaviours and outcomes. 62,63

Thus, these school environment interventions take a ‘socio-ecological’64 or ‘structural’65 approach to promoting health, whereby behaviours are understood to be influenced not only by characteristics of individuals, but also by the wider social context. A recent National Institute for Health Research (NIHR)-funded systematic review on the health effects of the school environment found evidence from observational and experimental studies that modifying the way in which schools manage their ‘core business’ (teaching, pastoral care and discipline) can promote student health and potentially reduce health inequalities across a range of outcomes, including reductions in violence and other aggressive behaviours. 63 Other outcomes that are improved by school environment interventions can include mental health and physical activity and reduced substance use, including alcohol, tobacco and drugs. 63

School environment interventions, therefore, are likely to be one of the most efficient ways of addressing multiple health harms in adolescence owing to their potential for modifying population-level risk as well as their reach and sustainability. 63 Multiple risk behaviours in adolescence are subject to socioeconomic stratification, and are strongly associated with poor health outcomes, social exclusion, educational failure, and poor mental health in adult life. 66 A recent King’s Fund report, Clustering of Unhealthy Behaviours Over Time, emphasised the association of multiple risk behaviours with mortality and health across the life-course, and the policy importance of reducing multiple risk behaviours among young people through new interventions that address their common determinants. 67

The INCLUSIVE (initiating change locally in bullying and aggression through the school environment) trial aims to evaluate the Learning Together (LT) intervention. This has been particularly informed by two international evidence-based school environment programmes. The first is the Aban Aya Youth Project (AAYP),61 a multicomponent intervention enabling schools to modify their social environment as well as delivering a social skills curriculum. This approach was designed to increase social inclusion by ‘rebuilding the village’ within schools serving disadvantaged, African American communities. To promote whole-school institutional change at each school, teacher training was provided and an action group (AG) was established (comprising both staff and students) to review policies and prioritise the actions needed to foster a more inclusive school climate. Among boys, the intervention was associated with significant reductions in violence and aggressive behaviour. 61 The intervention also brought benefits in terms of reduced sexual risk behaviours and drug use, as well as provoking behaviour and school delinquency. Second, the Gatehouse Project60,68 in Australia also aimed to reduce health problems by changing the school climate and promoting security, positive regard and communication among students and school staff. As with the AAYP, an AG was convened in each school, facilitated by an external ‘critical friend’ and informed by data from a student survey, alongside a social and emotional skills curriculum. A randomised controlled trial (RCT) found consistent reductions in a composite measure of health risk behaviours, which included violence and antisocial behaviour.

Restorative justice

The INCLUSIVE trial extends the AAYP and Gatehouse interventions by including a ‘restorative justice’ approach. The Steer Review50 in 2009 called for English schools to consider adopting more restorative approaches to preventing bullying and other aggressive behaviour to help minimise the harms associated with such problems. The central tenet of such approaches is repairing the harms caused to relationships and communities by criminal behaviour, rather than merely assigning blame and enacting punishment. Such approaches have now been adapted for use in schools and can operate at a whole-school level, informing changes to disciplinary policies, behaviour management practices, and the way in which staff communicate with students in order to improve relationships, reduce conflict and repair harm.

Restorative practice aims to prevent and/or resolve conflicts between students or between staff and students to prevent further harms. 75 It enables victims to communicate the impact of the harm to perpetrators, and for perpetrators to acknowledge and take steps to remedy this in order to avoid further harms. Restorative practice can involve methods to prevent incidents (e.g. ‘circle-time’, which brings students together with their teacher during registration periods or other lessons to maintain good relationships) and/or to resolve incidents (e.g. ‘conferencing’, bringing together relevant staff, students, and, where necessary, parents and external agencies such as police or social services). Restorative practice aims to prevent the occurrence or continuation not only of bullying but also of other forms of aggression and classroom disruption. Restorative practice can be delivered instead of, or alongside, more traditional punitive discipline. 76

The theoretical basis for restorative approaches has much in common with the theory of human functioning and school organisation. 59 It is theorised that the process of students coming together, discussing the harm, and working towards a reparative plan develops perpetrators’ competency through accepting responsibility for the actions and contributing to a reparative solution, and develops offenders’ understanding of the realities of others. Victims are also empowered in this process as they become an active participant in the decision-making process, and the acknowledgement of the offenders’ ability to offer some healing to the victim (e.g. via an apology or carrying out a sanction) gives dignity to both parties. This resonates with the ideas of improving relationships as well as promoting practical reasoning and sense of connection to school. By eliciting accountability for the harm caused to the victim and the school community and negotiating a plan for restitution, the young person is encouraged to reclaim an identity as a participant of the school community, not a peripheral outsider. 76 Through this process, the young people involved develop relational competency, and enhance their relationships with staff and other students by improving their ability to empathise and communicate effectively. Restorative approaches might indeed be particularly suitable for ‘alienated’ student offenders as they are given the opportunity to develop the necessary competencies to participate as a responsible member of the school community from which previously they might have felt excluded. 77 It may also be particularly helpful for female young people, as gender theory suggests that female adolescent identity is often based within a framework of relationship and connection. Thus, application of the principles of restorative approaches becomes a natural adjunct to the therapeutic process of self-identity and growth. 77

However, to date, restorative approaches in schools have been evaluated using only non-randomised designs, and systematic reviews have called for more rigorous evaluations of restorative practice in schools. 78 Those studies that have been carried out do suggest that the restorative approach is promising both in the UK79–81 and internationally,82–84 particularly when implemented at the whole-school level. For example, in England and Wales, the Youth Justice Board evaluated the use of restorative approaches at 20 secondary schools and six primary schools, and reported significant improvements in students’ attitudes to bullying and reductions in offending and victimisation in schools that adopted a whole-school approach to restorative practice. Restorative approaches thus appear to have the potential to complement school-environment interventions such as Aban Aya and the Gatehouse Project. They offer a highly promising way forward for reducing aggressive behaviours among British young people. A 2009 Cochrane review85 found no RCTs of interventions employing restorative approaches to reduce bullying in schools and recommended that this be a priority for future research.

Process evaluations report positive results in terms of feasibility and acceptability of restorative practices in schools. In New Zealand,86 case studies of five secondary schools and colleges found that all teachers valued restorative practice and felt that it was a good strategy for managing misbehaviour. In a pilot in London schools,87 students in schools that had implemented restorative practice reported that their school was doing a good job of stopping bullying. Teachers reported that most restorative meetings were effective at addressing bullying, gossiping, and disagreements between students and teachers.

Common challenges reported in process evaluations of restorative practice in schools include consistency in the way restorative practice is implemented. 82,88,89 Evaluations also suggest that restorative practice is implemented most successfully when it is delivered as part of a whole-school approach, when a positive ethos has been established, and when one-to-one problem-solving skills (e.g. listening and responsibility) have been introduced into the curriculum. 90,91

Social and emotional education

Social and emotional education aims to educate students not merely academically but using non-cognitive social skills and emotional self-management skills to enable young people to function at school and in other areas of life and to develop resilience or ‘grit’. There was evidence that classroom curricula that teaches young people the skills needed to manage their emotions and relationships can enhance social relationships, improve mental health and reduce bullying. 92 Many whole-school interventions, including the Aban Aya93 and Gatehouse60 interventions, also include social and emotional education elements, the intention being that these act synergistically with other components, helping students to take part in intervention activities within and beyond the classroom.

Objectives and hypotheses

In 2014, we developed the LT intervention, which aimed to modify the school environment by involving students, building better relationships, using restorative approaches, and developing social and emotional skills to reduce a range of risk behaviours including bullying and aggression. 74 The intervention combined AGs of staff and students to modify the whole-school environment; training for staff to implement restorative approaches; and a social and emotional skills curriculum. These components were intended to address school- and individual-level determinants of bullying and aggression and to be synergistic with one another: the AG ensuring that schools took a whole-school approach to restorative practice and the social and emotional skills curriculum enabling students to participate actively in AGs and restorative practices.

A pilot cluster RCT in eight schools met all of the prespecified feasibility and acceptability criteria. 74 We report here the findings of a full-scale cluster RCT of LT (the INCLUSIVE trial).

We hypothesised that in secondary schools randomly allocated to receive LT there would be lower rates of self-reported bullying and perpetration of aggression among students aged 14–15 years at the 36-month follow-up.

We hypothesised that at the 36-month follow-up student and staff secondary outcomes would be improved in intervention compared with control schools. More specifically, we expected improvements in students’ quality of life (QoL), well-being, psychological function and attainments; reductions in school exclusion and truancy, substance use, sexual risk, NHS use and police contacts among students; and improvements in staff QoL and attendance and reductions in staff burn-out.

We hypothesised that individual-level student socioeconomic status (SES), sex, and school-level stratifying factors (single-sex vs. mixed-sex school, school-level deprivation, and value-added strata) would moderate the effectiveness of the intervention for student outcomes.

We also hypothesised that LT would be cost-effective compared with standard school practice in terms of student quality-adjusted life-years (QALYs) and costs.

Trial design

We undertook a two-arm repeat cross-sectional cluster 1 : 1 RCT of the LT trial with an integral economic and process evaluation in 40 secondary schools in south-east England, with schools as the unit of allocation. 1 Our study population consisted of all students in the school at the end of year 7 (aged 11–12 years) at baseline, and at 24-month (end of year 9; aged 13–14 years) and 36-month (end of year 10; aged 14–15 years) follow-up, as well as school teaching and teaching assistant staff at each time point.

Refinements to the trial informed by the pilot

The changes made to the trial informed by the pilot included:

• identifying the Gatehouse Bullying Scale (GBS) and Edinburgh Study of Youth Transitions and Crime (ESYTC) school misbehaviour subscale as primary outcomes of bullying victimisation and perpetration of aggressive behaviours, respectively

• including validated measures of drug use, sexual behaviour and educational attendance and attainment as additional secondary outcomes

• including all ‘state’ schools in the recruitment pool of schools to reflect the overall population profile of schools in south-east England

• using existing school networks to facilitate timely recruitment

• using revised timetabling – project initiation in February of the preceding school year, surveys of staff and students to be conducted in the summer term each year, timetabling of intervention and staff training to be undertaken prior to September school-year start

• enhancing quantitative data on intervention fidelity, including structured independent assessments of intervention delivery

• undertaking an economic evaluation to use the Child Health Utility 9D (CHU9D) scale and to be supplemented with a cost–consequences analysis.

Refinements to the trial after commencement

During the development of the trial, the following changes to the protocol were made:

• A meaure of bullying perpetration (the Modified Aggression Scale Bullying subscale) was included after the baseline survey as we elected to add a measure of bullying perpetration as well as one of victimisation.

• We included administrative documents (e.g. minutes, attendance sheets, training satisfaction feedback) in our assessment of trial arm fidelity to provide us with a wider range of quantifiable data.

The protocol was amended during the trial to refine the methods used. All amendments were approved by the independent study steering committee and the funder of the trial (NIHR). The only change to trial outcomes was the addition of a measure of bullying perpetration (secondary outcome). All refinements were completed before the 36-month surveys were collected and before any trial analyses were conducted.

Participants

The INCLUSIVE trial was a universal intervention aimed at all 11- to 16-year-olds in participating secondary schools in England. Although the intervention was expected to have effects on the whole school, our study population consisted of students at the end of year 7 (aged 11–12 years) at baseline and at the end of year 10 at the 36-month follow-up (aged 14–15 years), as well as all school teaching and teaching assistant staff. This year group should have experienced the intervention for 3 years, including the classroom curriculum for years 8–10. All students in the school in that year and all teaching staff were surveyed at each time point, not only those who participated at baseline.

Eligibility criteria for participants

Participant eligibility was assessed at the school level. Eligible schools were:

• Secondary schools within the state education system (including community, academy or free schools, and mixed- or single-sex schools) in south-east England. We took the widest definition of a ‘state school’ and excluded only private schools, schools exclusively for those with learning disabilities and pupil referral units. The last two were excluded as the INCLUSIVE study was unlikely to be appropriate for their populations.

• Schools whose most recent school quality rating by Ofsted was ‘requires improvement’/‘satisfactory’ or better. Schools with an ‘inadequate’/‘poor’ rating were excluded, as these schools would be subject to special measures that were likely to impede LT delivery.

Eligible schools were approached initially by letter and e-mail, with a telephone follow-up, complying with good practice and research governance for undertaking studies within the education system.

As the intervention was delivered at the whole-school level, there were no specific eligibility criteria for students, although parents who did not want their child to participate in the surveys were able to opt out on behalf of their child.

Settings and locations

Schools were recruited between March and June 2014 from secondary schools in Greater London and the surrounding counties (Surrey, Kent, Essex, Hertfordshire, Buckinghamshire and Berkshire) that had a maximum travel time of 1 hour from the study centres in London. To aid recruitment, we partnered with existing school networks, such as the University College London (UCL) Partners Schools Network and the Institute of Education Teaching Schools, and schools that are part of our collaborating schools network, Challenge Partners. We approached approximately 500 eligible schools, initially by letter and e-mail and with a telephone follow-up.

The 40 participating schools did not differ from 450 non-recruited schools in terms of school size, population, deprivation, student attainment or value-added education. However, participating schools were more likely to have an Ofsted rating of good or outstanding (see Appendix 1, Table 26).

Intervention

The INCLUSIVE trial involved 2 years of externally facilitated intervention and a final year without external facilitation. The LT intervention was intended principally to augment, rather than to replace, existing activities (e.g. training and curricula) in intervention schools. However, it was intended to replace existing non-restorative disciplinary school policies and practices when the AG deemed restorative approaches more appropriate. The intervention logic model is shown in Appendix 1 (see Figure 6).

Below we describe the intervention informed by the TIDieR (Template for Intervention Description and Replication) checklist for better intervention reporting. 94 Fidelity assessment is described under process evaluation, the product of which is presented in our results.

Materials

Schools allocated to receive the intervention were provided with various resources. School staff were offered training in restorative practices, with participants given written summaries of the material covered in training. Schools were provided with a manual to guide the convening and running of an AG. For the first 2 years of the intervention, schools were provided with an external facilitator for the AG. Schools were sent a report on student needs detailing findings from a survey of students aged 11–12 years about their attitudes to and experiences of school, and their experiences of bullying, aggression and other risk behaviours (see Appendix 3). Schools were provided with written lesson plans and slides to guide delivery of a classroom-based social and emotional skills curriculum.

Procedures

Training was given to all staff, and in-depth training was given to selected staff, including training in formal ‘conferencing’ to deal with more serious incidents.

Training

Staff training was implemented to ensure that teachers understood the necessary skills to engage in restorative practice. Training was provided by trainers accredited by the UK’s Restorative Justice Council. Each school had its own named facilitator, who was a freelance consultant with experience of school leadership or organisational change, co-ordinated by a lead facilitator who trained them in the intervention theory and methods. AGs comprised at least six staff members (including one member of the school’s SLT and one member of the school’s teaching, student support and administrative staff) and at least six students from each school, led by a member of the school’s SLT with support from the external facilitator in the first two years of the intervention but not in the third year. All of these staff attended the all-staff training and some attended the in-depth training. Staff who received basic training in restorative practice implemented this in the form of the use of restorative language and circle time. In addition, 5–10 staff members at each school who received in-depth training in restorative practice implemented this in the form of restorative conferences. The curriculum was delivered by teachers who specialised either in PSHE or in other subjects guided by lesson plans.

Modes of delivery

All intervention components were delivered face to face.

Location

All components were delivered on school premises.

Dose

Training occurred in the first year of intervention, comprising half a day for all staff plus in-depth 3-day training for 5–10 staff members at each school. AGs met six times per year in all 3 years. Restorative practices were delivered as frequently as required in each school. In their curriculum, students received 5–10 hours of teaching per year.

Planned adaptations

The intervention enabled local tailoring, informed by the needs survey and other local data sources. AGs ensured that implementation in their school was appropriate to local needs as identified by members and the survey of student needs. This included ensuring that revisions to policies and rules built on existing work, deciding which curriculum modules to deliver in each year, and implementing locally decided actions aiming to improve relationships and student participation (e.g. cascading restorative practice training to staff who had not attended or to student peer mentors).

Unplanned modifications

There were no unplanned modifications.

Comparator: control schools

Schools randomised to the control group continued with their normal practice and received no additional input. The sample of schools was spread over a wide geographical area and there were no cases in which intervention and control schools were near one another. Head teachers and a small number of staff in control schools were aware that the school was participating in a trial that was described as the INCLUSIVE trial. These individuals were not informed that the name of the intervention was LT and were not informed about the detailed contents of the intervention during recruitment. It is therefore unlikely that schools in the intervention and control arms would have shared information about the intervention.

Assessment and follow-up

Data management

The study centre received class lists for each school in advance of each survey. Participants were allocated a unique identifier (ID) prior to each survey and this ID was recorded on the questionnaire. All questionnaires were anonymous. Questionnaires were completed in classrooms and completed questionnaires were collected in schools on the day of the survey. If a participant was not in school on the day of the survey, a questionnaire was left at the school for them to complete later and was returned to the study centre by post. Completed questionnaires were transported from the school by study personnel to the study centre, where they were stored in a locked room.

Questionnaires were then securely transported for data entry by a third party, where they were double-entered into a database by trained personnel. Each questionnaire was checked at the time of data entry for any handwritten comments. Questionnaires with any additional text, regardless of content, were scanned, and password-protected scans were sent to the study team for safety reporting assessment. Electronic data generated from data entry were transferred via password-protected secure FTP and stored on secure servers at the London School of Hygiene & Tropical Medicine (LSHTM).

Following data entry, questionnaires were securely transported to the LSHTM Clinical Trial Unit (CTU) for archiving. An inventory of all questionnaires was maintained by LSHTM CTU.

Electronic data generated from data entry were transferred via password-protected secure FTP and stored on secure servers at LSHTM. Relevant trial documentation will be kept for a minimum of 15 years after study completion.

Ethics arrangements

The trial was approved by the UCL Ethics Committee (reference 5248/001). Ethics arrangements were informed by recent guidance on ethical issues in cluster RCTs.

Outcomes

Demographic information

Process evaluation

The process evaluation examined intervention implementation and receipt, and possible causal pathways, in order to facilitate interpretation of the outcome data and enable refinement of the intervention logic model. Informed by existing frameworks,110,111 data were collected to examine the following.

Economic evaluation

Economic evaluation of the intervention also took place. In accordance with NIHR guidelines, the methodology and results for the economic evaluation are reported separately in Chapter 4.

Changes to trial outcomes

The following deviations from this plan occurred during data collection.

We were unable to collect school-level data on individual student and staff attendance and school rates of temporary and permanent exclusions, despite multiple attempts to contact schools and obtain these data after the intervention was completed. Small numbers of data on school-level exclusions were provided but these were not sufficient for analysis. In response to requests for data, schools either did not respond or notified us that this was a burden they were not prepared to undertake now that the trial had finished. Data were not available similarly across intervention and control schools. A further offer of money towards school staff funds (May 2018; see above) did not motivate schools to provide these data.

In discussion with NIHR, we came up with the following mitigation plan.

• School attendance data: these data are currently planned to be collected by Manchester University in a planned follow-up to this study in late 2019. These are therefore not reported here.

• School rates of temporary and permanent exclusions: these data are published at school level by the Department for Education, albeit with a significant delay. We plan to examine these in the future when available, although the data are not reported here.

• Staff attendance: individual staff attendance levels are not available through other means and these analyses cannot be undertaken.

Sample size

We proposed to recruit sufficient participants to detect a difference between groups of 0.25 standard deviation (SD) with 90% power and a 5% level of significance. This is considered to represent a moderate size of effect and is in line with the effect sizes seen in the literature. 117

The average English school has approximately 150 students per year group of students, although this varies across schools. Using a conservative intraclass correlation coefficient (ICC) of 0.04118 and an estimate of 150 students per school, a trial involving 20 schools per arm would have provided 90% power to identify an effect size of 0.25 SD with a 5% significance level. If two schools per arm (i.e. 10%) were to be lost to follow-up over the course of the trial, we would still have 80% power to detect an effect size of 0.23. The total student sample size was estimated to be between 4000 (assuming 100 students per school) and 6000 (assuming 150 students per school). As we were surveying all young people in the relevant school year at each follow-up, this sample was likely to remain similar across the study. The total student sample was therefore planned to be approximately 6000.

No stopping guidelines were put in place for this study; any and all schools and students who agreed to take part were included.

Randomisation

Allocation concealment mechanisms

As with most social intervention trial schools, their students, teachers and other staff could not be ‘blinded’ to allocation status. However, field work staff were blinded to allocation, as were data-input staff. Analysis of follow-up quantitative data was undertaken blind to allocation.

Implementation

Randomisation and stratification was undertaken remotely by the CTU at LSHTM.

Blinding

After randomisation, schools, the intervention team and the process evaluators could not be ‘blinded’ to allocation status. However, other field work staff were blinded to allocation, as were the outcome evaluation research team lead (RV) and data input and data analysis staff. Process and economic evaluation researchers could not be blinded.

It was not possible to blind the costing conducted alongside the process evaluation. Researchers remained blinded to the arm and school name while preparing statistical analysis of cost data from questionnaires and the CHU9D until the analysis was completed.

Statistical methods

Participant flow

A total of 6667 students in the 40 participating schools provided data at baseline, with the participation rate being 93.6% of the students on the school roll (intervention arm, 92.9%; control arm, 94.3%). Characteristics of the schools and students at baseline and at 24 and 36 months are shown by trial arm in Appendix 1 (see Tables 26 and 27). All schools participated in the follow-up surveys at 24 and 36 months; the numbers of students who completed the questionnaires at baseline, 24 months and 36 months were similar between the arms (Figure 1). Student and school characteristics and outcomes at baseline were well balanced between the arms.

FIGURE 1.

The CONSORT flow diagram. Reproduced from Bonell et al. 122 © 2018 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY-NC-ND 4.0 license.

Reproduced from Bonell et al. 122 © 2018 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY-NC-ND 4.0 license

Baseline data

Summaries of demographic characteristics of respondents are presented in Appendix 1 (see Table 27). The arms were well balanced in terms of demographic characteristics and the primary and secondary outcome measures.

The data presented come primarily from mixed-sex schools (80%) with a ‘good’ Ofsted rating (60%). There was a good mix of male and female participants, ethnicities and religions. The majority of student participants came from a two-parent family (72%) in which at least one parent was employed (73%).

In terms of primary and secondary outcomes measures, reports of bullying and aggressive behaviour, emotional difficulties, QoL, risk behaviours, truancy, NHS service use and contact with police were similar between the control and intervention arms at baseline (see Table 1).

Staff outcomes in relation to stress and burnout and QoL were also similar across control and intervention conditions at baseline (see Appendix 1, Table 28).

Numbers analysed

A total of 3087 participants were included in analysis from the control condition and 2873 were included from the intervention condition. Analysis was undertaken by original assigned group. Details on missing data are shown in Appendix 1 (see Table 43).

Outcomes and estimation

Staff outcome data

No differences were found in relation to staff stress or burnout of QoL between the control and intervention arms of the study at 24 or 36 months (Table 4).

Ancillary analyses

Analysis of the primary and secondary outcomes by prespecified subgroup (see Appendix 1, Table 29) suggests that the intervention had a greater effect in boys than in girls for many secondary outcomes [PedsQL overall score; SDQ total difficulties score; SWEMWBS total well-being index; bullying perpetration (modified aggression scale); CHU9D overall score; ever smoked; ever drunk alcohol; contact with police]. The intervention was also more effective in students with higher baseline bullying experience, with greater effects on bullying (GBS score) and psychological secondary outcomes (PedsQL overall score; SDQ total difficulties score; SWEMWBS total well-being index; CHU9D overall score). The intervention was more effective in those with greater baseline aggression, with greater effects on both primary outcomes (GBS score; ESYTC overall score), psychological secondary outcomes (PedsQL overall score; SDQ total difficulties score; SWEMWBS total well-being index; CHU9D overall score) and risk behaviours (ever smoked; ever drunk alcohol). There was no suggestion of any difference in the outcomes by level of SES).

Adverse events

A serious adverse event was defined as (1) any death or event requiring hospitalisation in a young person reported to investigators or (2) a response to study questionnaires that prompted significant concerns about mental health, sexual risk or child safety, which were then communicated to the school. There were 15 reported SAEs to date (Table 5). The total number of reported serious adverse events was similar in each arm.

Process evaluation

Fidelity was variable, with a reduction in the fidelity of formal intervention activities in year 3. The median fidelity score for years 1 and 2 (maximum possible score 8) was 6 (interquartile range 5–7), whereas for year 3 (maximum score 4) the median was 1 (interquartile range 0–3). In year 3, 15 schools sustained restorative practice. Interviews with AG members and focus groups with staff in case study schools suggested that, in year 3, schools commonly incorporated what they regarded as the most useful AG functions into mainstream school structures and processes. For example, involving students in decision-making and further review of policies to ensure that these supported restorative practice was integrated into existing school committees. Regarding specific components (training, AGs and restorative practices), these were delivered with good fidelity. The fidelity of curriculum delivery was lower. Increased fidelity of delivery in the first 2 years of intervention was associated with lower rates of bullying victimisation at 24 months, but not with rates of aggression. The fidelity score for year 3 was not associated with either primary outcome (see Appendix 1, Table 30).

Slightly over half of staff in intervention schools were aware that the school had been taking steps to reduce bullying and aggression, with this falling slightly between years 2 and 3 (Table 6). About one-third of students reported being aware that the school had been taking steps to reduce bullying (Table 7). About half reported that, if there was trouble at school, staff responded by talking to those involved to help them get on better. About two-thirds of students reported that teachers and students got together to build better relationships or discuss their views and feelings. Other data on the process evaluation are shown in Appendix 1, Tables 31–35.

Many control schools implemented similar activities to those prescribed in the intervention but with very variable quality. We identified five control schools that delivered restorative practice, social and emotional skills education and consultation with students on policy. A per-protocol analysis excluding these schools showed no discernible differences in the intervention effects compared with the intention-to-treat analyses (Table 8).

Response rates

Some elements of the process evaluation, such as interviews with facilitators, school staff and students, as well as observations of training and AGs, achieved excellent response rates (see Appendix 1, Table 36). However, other elements, such as the surveys of staff delivering the curriculum or restorative practice (RP), achieved poor response rates. This reflected what, in retrospect, was an excessive data collection burden on schools. The low response rates for some elements of the process evaluation in year 3, such as the survey of AG members, also reflected the reality that in this year not all schools delivered all aspects of the intervention, rendering data collection obsolete. However, despite this, our multisource approach meant that we were still able to assess what was happening in most schools, with the exception of a few schools in year 3, where it is reasonable to assume that few, if any, intervention activities were being implemented.

Baseline context of intervention schools

Context of control schools

Intervention implementation

Participation and reach

We drew on facilitator diaries, AG minutes and training attendance sheets to estimate the numbers of staff and students participating in training and AGs. All-staff training on average reached 103 staff members (range 34–162 staff members) per school in schools for which data were available. In-depth training on average reached 6.35 staff members (range 2–13 staff members) per school in schools for which we have data. Fifteen schools sent at least five staff members on this training (see Appendix 1, Table 39).

On average in year 1, 7.4 staff members (range 3–12 staff members) per school participated in AGs in schools for which data were available, with this falling to 5.4 staff members (range 2–9 members) per school in year 2. On average in year 1, 7.8 students (range 5–14 students) per school participated in AGs in schools for which we have data, with this increasing to 8.8 students (range 4–23 students) per school in year 2. We did not calculate figures on participation in year 3 because schools implemented AGs in such heterogeneous ways.

Schools were encouraged to, and most did, select a diversity of students to serve on the AG, including those with a history of misbehaviour (schools AD, AF, AL, AO, AT, AW, AX, BC, BK, BI and BM), those who struggled academically (schools AD, BC, BI, BK and BM) and those who were at risk of disengagement (AK and BC). Other schools focused recruitment on students who were high achievers (schools AK, AS and AW). These schools sometimes dropped lower-attaining students from AGs. Most schools selected students across school years to participate, but some (schools AF, AL and AW) invited only those in the study cohort.

Just over one-third of staff completing the all-staff survey at the end of year 2 reported that students were involved in developing the behaviour policy or school rules, with this rising slightly by the end of year 3 (Table 12). At the end of years 2 and 3, only about one-third of students reported being aware that the school had been taking steps to reduce bullying. By the end of year 2, around one-third of students reported that they had experienced lessons about relationships or emotions, with this falling to around one-quarter by the end of the third year (Table 13). Only around one-quarter of students reported that students had been involved in developing the behaviour policy or school rules. Across both time points, only about one-sixth of students knew what was meant by the term ‘restorative practices’. About half reported that, if there was trouble at school, staff responded by talking to those involved to help them get on better. About two-thirds of students reported that teachers and students got together to build better relationships or discuss their views and feelings. Focus groups with students not involved in AGs suggested that there was broader awareness of the work of the AG among students in some schools (schools AE, AF, AH, AW and BK) more than others (schools AO, AS, AT, AZ, BC and BD).

Even if the AG itself was not well known, students on AGs in some schools reported that their peers had noticed changes being made but were unaware of how these had come about (schools AD, AT and BK):

They’re aware of certain changes. But they’re not really aware of why people are doing this. So they see it as, why have they put up a basketball hoop? Oh, that’s probably for people to play on. They don’t see why . . . what caused it to happen. It’s only people on the action group and some teachers are aware of it. Everyone else is oblivious to the situation.

Male, year 9 student, school BK, interview, year 1

Slightly over half of staff in intervention schools were aware that the school had been taking steps to reduce bullying and aggression, with this falling slightly between years 2 and 3 (see Table 12). In interviews with SLT members in year 3, some reported that although the name of the intervention was sometimes not known by all staff members, the push for increased use of RP was more widely known.

Reception and responsiveness

Among the staff who were aware that the intervention was occurring, the vast majority were supportive overall at both time points (see Table 12). With the increasing expectations for educational attainment, some staff resented the work involved in the intervention given its primary focus on health and well-being rather than attainment:

One of the challenges that I foresee is that . . . I’ve worked at a few schools and I don’t think this school is unusual in that teachers feel responsible for academic progress and they don’t . . . feel responsible for student well-being in the same way. And I think that’s a result of accountability measures being focused around those things and so the challenge is to move that perception so that staff take ownership of all of those things. But also then to help them to understand that the two things obviously are linked.

Assistant head-teacher, school BD, interview, year 1

Commitment to the intervention was notably lower in schools where the pre-existing approach to behaviour was disciplinarian or not supportive of student participation in decision-making (schools AH, AT and BD).

Methodology

Study question, selection of alternatives and form of evaluation

A cost-effectiveness analysis (CEA) was undertaken to assess the relative cost-effectiveness of the LT intervention compared with standard school-based practices for managing aggression. The economic evaluation was undertaken in conjunction with the analysis of the effectiveness of the LT intervention at 24 and 36 months since randomisation. The primary economic evaluation was a cost–consequences analysis with all main outcomes. In addition, a cost–utility analysis was conducted, with health outcomes expressed in terms of QALYs.

Resource use and outcome data collected as part of the cluster RCT were used to report cost-effectiveness at 24 and 36 months since randomisation. The CEA involved estimation of resource use, costs, HRQoL and QALYs at 24 and 36 months. In the cost–consequences analysis, we report incremental costs and incremental effects of all main outcomes. The cost–utility analysis used information on QoL collected at 24 and 36 months’ follow-up to report QALYs at each time point. We compared randomised groups by reporting incremental costs, incremental QALYs and an incremental cost-effectiveness ratio (ICER). The main assumptions of the CEA were subjected to sensitivity analyses. The results are illustrated on the cost-effectiveness planes (Figures 2 and 3).

FIGURE 2.

Uncertainty in the mean costs (£) and QALY differences and their distribution for intervention versus control at 24 months.

FIGURE 3.

Uncertainty in the mean costs (£) and QALY differences and their distribution for intervention versus control at 36 months.

Results

Cost-effectiveness

There was no significant difference between groups in staff QoL as measured with the overall CHU9D or the SF12 (Table 22). However, as noted earlier in the report, there was a significant effect on the primary outcomes of the GBS and ESYTC scores. The mean total costs per pupil at 24 and 36 months for the control school were £493 and £667, respectively. The mean total costs per pupil were higher for the intervention school at both 24 months (£650) and 36 months (£719). The unadjusted incremental costs were £157 (95% CI –£83 to £397) at 24 months and £62 (95% CI –£165 to £288) at 36 months. The adjusted incremental costs were £57 (95% CI –£183 to £297) at 24 months and £71 (95% CI –£73 to £214) at 36 months. Overall, the intervention is associated with higher costs, but the incremental costs are surrounded by statistical uncertainty.

TABLE 22 - Cost–consequences results

These are subscales of the SF-12.

Measure	Trial arm, mean (SD)		Unadjusted effect estimate, difference (95% CI); p-value	Adjusted effect estimate, difference (95% CI); p-value
	Control	Intervention
At 24 months
Total costs (£)	493 (1687)	650 (6203)	157 (–83 to 397); 0.199	57 (–183 to 297); 0.642
Bullying victimisation – GBS overall score	0.42 (0.02)	0.37 (0.02)	–0.02 (–0.05 to 0.01); 0.2198	–0.02 (–0.05 to 0.01); 0.1581
Misbehaviour/delinquency – ESYTC	4.24 (0.28)	3.96 (0.28)	–0.04 (–0.34 to 0.27); 0.8113	–0.06 (–0.35 to 0.24); 0.7206
CHU9D – student HRQoL	0.86 (0.01)	0.87 (0.01)	0.00 (–0.00 to 0.01); 0.4883	0.00 (–0.00 to 0.01); 0.2640
SF-12 – staff HRQoLa physical health score	54.97 (0.32)	55.22 (0.34)	–0.35 (–1.05 to 0.34); 0.3187	–0.26 (–0.99 to 0.47); 0.4825
SF-12 – staff HRQoLa mental health score	44.56 (0.49)	43.71 (0.52)	–0.66 (–1.64 to 0.32); 0.1883	–0.45 (–1.49 to 0.58); 0.3896
At 36 months
Total costs (£)	667 (1829)	719 (3486)	62 (–165 to 288); 0.593	71 (–73 to 214); 0.334
Bullying victimisation – GBS overall score	0.34 (0.02)	0.29 (0.02)	–0.03 (–0.06 to –0.00); 0.0395	–0.03 (–0.06 to –0.00); 0.0441
Misbehaviour/delinquency – ESYTC	4.33 (0.20)	4.04 (0.21)	–0.07 (–0.38 to 0.25); 0.6820	–0.13 (–0.43 to 0.18); 0.4199
CHU9D – student HRQoL	0.85 (0.00)	0.86 (0.01)	0.01 (–0.00 to 0.01); 0.2423	0.01 (–0.00 to 0.01); 0.0795
SF-12 – staff HRQoLa physical health score	55.62 (0.36)	55.18 (0.36)	–0.71 (–1.4 to 0.02); 0.0449	–0.61 (–1.36 to 0.13); 0.1058
SF-12 – staff HRQoLa mental health score	43.76 (0.44)	43.69 (0.45)	0.15 (–0.83 to 1.13); 0.7642	0.31 (–0.76 to 1.38); 0.5691

Overall, the intervention is associated with higher costs (Table 23) but the mean gain in CHU9D score is slightly higher in the intervention arm even though it has not reached a significant level (see Table 22), leading to small QALY gains. The adjusted ICERs are £13,284 (95% CI –£32,175 to £58,743) and £1875 (95% CI –£12,945 to £16,695) per QALY at 2 and 3 years, respectively. These baseline ICERs are well below the threshold at which NICE considers interventions to be cost-effective, which is £20,000 to £30,000 per QALY gained. However, assessment of the CIs around ICERs indicates that there is uncertainty in the results about whether or not the intervention is cost-effective at 2 years, whereas at 3 years it is within the threshold.

TABLE 23 - Cost–utility results

The incremental effects are reported after adjusting for baseline variables.

Measure	Trial arm, mean (SD)		Incremental effect (unadjusted), mean (95% CI)	Incremental effecta (adjusted), mean (95% CI)
	Control	Intervention
At 24 months
Total costs (£)	493 (1687)	650 (6203)	108 (–119 to 335)	96 (–151 to 343)
QALY	1.6833 (0.1710)	1.6834 (0.1710)	0.0106 (–0.0113 to 0.0325)	0.0072 (–0.0043 to 0.0188)
ICER			10,214 (–20,190 to 40,616)	13,284 (–32,175 to 58,743)
At 36 months
Total costs	667 (1829)	719 (3485)	46 (–154 to 246)	28 (–187 to 242)
QALY	2.4858 (0.2496)	2.4937 (0.2473)	0.0240 (–0.0097 to 0.0578)	0.0148 (–0.0057 to 0.0353)
ICER			1905 (–5938 to 9748)	1875 (–12,945 to 16,695)

Graphically we present the uncertainty in cost-effectiveness acceptability curves in Figures 4 and 5. At 2 years, there is a 65% probability that the intervention is cost–effective at a willingness-to-pay threshold of £20,000 per QALY. At 3 years, there is a 90% probability that the intervention is cost-effective at the £20,000 per QALY threshold. Overall, there is some uncertainty at 2 years that the intervention is cost-effective, but at 3 years there is a high degree of certainty that it is.

FIGURE 4.

Cost-effectiveness acceptability curve, reporting the probability that the intervention is cost-effective (at 24 months) at alternative willingness to pay for a QALY gain.

FIGURE 5.

Cost-effectiveness acceptability curve, reporting the probability that the intervention is cost-effective (at 36 months) at alternative willingness to pay for a QALY gain.

In the sensitivity analysis we explored the impact of not including teacher training time, as this was provided as part of routine inset days (Table 24). This had little impact on results and conclusions. Again, there was some uncertainty about whether or not the increased hospital days in the intervention arm were related to the trial, but exclusion of NHS and police costs did not have a significant impact on results. The fidelity analysis shows (Table 25) that a higher fidelity score was associated with lower costs and QALYs in the unadjusted analysis. When other baseline variables are adjusted, higher fidelity score was associated with lower costs but higher QALYs, although the results were surrounded by statistical uncertainty.

TABLE 24 - Sensitivity analysis reporting costs, QALYs and ICERs at 36 months

Measure	Trial arm, mean (SD)		Incremental effect (unadjusted), mean (95% CI)	Incremental effect (adjusted), mean (95% CI)
	Control	Intervention
Excluding staff time training costs
Total costs (£)	667 (1829)	709 (3486)	35 (–164 to 235)	28 (–187 to 242)
QALY	2.4858 (0.2496)	2.4937 (0.2473)	0.0241 (–0.0097 to 0.0577)	0.0148 (–0.0057 to 0.0353)
ICER			1476 (–6393 to 9345)	1175 (–13,470 to 15,820)
Excluding NHS and police costs
Total costs (£)	184 (66)	215 (84)	17 (12 to 22)	32 (28 to 36)
QALY	2.4858 (0.2496)	2.4937 (0.2473)	0.0129 (–0.0035 to 0.0294)	0.0106 (–0.0009 to 0.0221)
ICER			1314 (–371 to 2999)	3008 (–275 to 6291)
Alternative discount rate
Total costs (£)	681 (1862)	735 (3605)	47 (–158 to 252)	29 (–191 to 249)
QALY	2.5509 (0.2562)	2.5591 (0.2538)	0.0247 (–0.0099 to 0.0593)	0.0152 (–0.0059 to 0.0362)
ICER			1918 (–5907 to 9743)	1917 (–12,900 to 16,734)

TABLE 25 - Effect of fidelity score on costs and QALYs at 36 months

Measure	Analysis, mean (95% CI)
	Unadjusted	Adjusted
Total costs (£)	–16 (–92 to 61)	–13 (–73 to 46)
QALY	–0.0007 (–0.0112 to 0.0098)	0.0043 (–0.0017 to 0.0102)

Primary and secondary results

Learning Together reduced student reports of bullying victimisation, both overall and in terms of teasing and rumour-spreading, compared with schools continuing their standard practice. We did not identify a reduction in student reports of aggression across the whole sample. Additionally, LT appeared to have larger beneficial impacts on a wide range of important secondary outcomes among students, ranging from improved psychological function, well-being and QoL to reductions in police contact, smoking and alcohol and drug use. The effects seen for bullying and other continuous outcomes by the third year approximate 0.1 SD, which are potentially highly important at the population level. The impact of LT on risk behaviours at 36 months was also substantial. We found intervention effects both in the whole sample and in those with higher levels of bullying or aggression at baseline, implying that the intervention worked to curtail existing bullying and aggression (secondary prevention) as well as prevent new bullying (primary prevention).

We found a small effect of the intervention on bullying at 36 months (as hypothesised for our primary outcome) but not at 24 months, and we found a similar strengthening of effects over time for most secondary outcomes, suggesting that this was a broad intervention effect. We had anticipated that this would be the case, reflecting the time needed for changes made in the first 2 years to be translated into organisational change within schools. Although many schools did not deliver formal intervention components so well in year 3 as earlier, our process evaluation suggested that by year 3 schools had integrated some of the AG functions into mainstream school structures and processes, enhancing their ability to implement whole-school change. This strengthening of intervention effects with time is consistent with evidence from the Gatehouse Project, in which effects increased among student cohorts subsequently moving through intervention schools after the initial 2-year intervention was completed. 68

We found that although LT was delivered as a universal intervention, exploratory analyses suggest that it was most powerful for students with higher baseline levels of bullying or aggressive behaviours, regardless of the prevalence of bullying or aggression in their school. This is reassuring in that LT appears to benefit those who most need it regardless of the level of problems in a school, and reflects similar findings in other school aggression programmes,93 but is also likely to reflect greater room for improvement among those with higher baseline problems. We also found LT to have greater effects for boys than for girls for secondary psychological and behavioural outcomes, although not for primary outcomes.

There are few other studies to which we can appropriately compare our findings. Our study is unique for the breadth of positive outcomes across bullying, well-being, QoL, psychological problems and substance use. We did not find an effect of our intervention on perpetration of aggressive behaviours, which is contrary to the Ayan Aba study,61 although consistent with the Gatehouse study68 and findings from reviews that school-based studies consistently have stronger effects for victimisation than perpetration. 57 Among comparable trials, the Gatehouse Project identified no impact on bullying or psychological problems, in contrast to our findings,60 although it identified similar impacts on risky behaviours, including substance use. 68 As predicted by the theory of human functioning and school organisation,59 which underpinned our intervention theory of change, intervention effects were concentrated on risk behaviours that may function as overt manifestations of anti-school rebelliousness, such as bullying, smoking and drunkenness, rather than on outcomes that are less manifest, such as sexual health outcomes.

We hypothesised that the intervention would increase the age at sexual debut and increase use of contraception because there is evidence that engagement with school is protective against early sexual debut and lack of contraception use. 138 However, we found no effects on these outcomes, perhaps because our intervention did not explicitly address sexual health, or because, unlike bullying and substance use, sexual behaviours occur off the school site and in private. We did not assess sexual debut or contraception use at baseline at the request of the ethics committee because of the age of the baseline cohort. It is possible that, despite good balance generally in the trial between the intervention and control arms, there were chance imbalances in these outcomes that might have obscured intervention effects on these outcomes.

Intervention cost and cost-effectiveness

The intervention was low cost, falling into the ‘very low cost’ category for school interventions according to the Educational Endowment Foundation guidance. 139 The costs of trainers, facilitators and school staff were an additional £58 per pupil in the intervention compared with control schools over the 3 years, and approximately £47 if LT training costs were regarded as replacing other teacher training rather than being additional. In terms of cost per QALY, at 2 years there was uncertainty about the cost-effectiveness (ICER £13,284, 95% CI –£32,175 to £58,743), but over 3 years the intervention was cost-effective (ICER £1875, 95% CI –£12,945 to £16,695).

Trial context

Schools recruited to the trial were commonly already taking actions to implement RP, student participation in decisions, and/or social and emotional skills education, but these were not being implemented as part of a coherent, guided intervention. Despite reporting the value of RP, most schools used discipline systems focused on traditional punitive responses and only a few schools had staff trained in RP. Schools allocated to the control group did, in some cases, act to implement elements of what they perceived the intervention to involve, in particular RP. However, only a handful of control schools implemented RP, student participation in decisions and social and emotional skills education, and none did so as part of a coherent intervention.

Fidelity and acceptability

The intervention was implemented with variable fidelity and acceptability between components. The training, needs reports and AGs were well delivered and generally acceptable. AGs generally succeeded in reviewing policies and rules, and this was perceived as useful. There were mixed views about the facilitators. On balance, they appeared useful to keep schools focused on scheduling meetings and delivering action, but they may not have needed to attend every meeting. Meetings became less frequent after facilitators were withdrawn, but this may have reflected the mainstreaming of their function (see below). The curriculum was delivered patchily and there were more negative views than positive from teachers. Although data on implementation were poor, interviews and surveys suggest that RP, particularly in the form of conferences, was widely delivered and acceptable. However, time constraints and perceived threats to authority were listed as barriers. The intervention was developed with lower formal fidelity in year 3 but there was some evidence that schools prioritised the components that they most valued and adapted and mainstreamed these components into normal school policies and systems.

Determinants of implementation

The intervention was delivered more completely when led by a member of staff with sufficient authority and support to make decisions and drive delivery. In many, but not all, cases, this required that the staff member was on the school’s SLT. Fidelity, in terms of both success in delivering specific components and ensuring that activity was focused on improving relationships and supporting students’ overall well-being, also appeared stronger when the intervention could build on a pre-existing school culture that was at least starting to address such issues.

Reach and awareness

There was low awareness of the intervention among both staff and students not directly involved in implementation. Staff and students who were involved in delivery did not view this as a barrier to the intervention achieving impacts, as what counted was piecemeal actions to improve the school environment whether or not these were branded as part of the intervention.

Intermediate outcomes

There was some evidence that the intervention did improve students’ sense of a positive school climate, but as this did not manifest until year 3 it was not possible to conclude that this mediated intervention effects on primary and secondary outcomes that also did not manifest until year 3. There was some evidence that the intervention reduced student contact with anti-school peer groups in year 2 but not year 3 and with no evidence that this mediated intervention effects on primary or secondary outcomes. Qualitative data showed how intervention components and processes improved empathy between students and students and staff.

Strengths and limitations

We undertook a well-conducted, large, cluster RCT of an intervention of proven feasibility (LT) in a group of schools that are larger, are more ethnically and culturally diverse, and have notably greater levels of student deprivation than the English average. Participating schools were representative of the approximately 500 schools initially approached and all schools were retained in the trial. Our follow-up was sufficiently lengthy to allow time both for intervention effects to develop and for investigation of persistence of intervention effects after the end of the facilitated intervention. Student participation was high. Our outcome research team and intervention team remained independent throughout the trial and blinding of lead researchers was maintained. Outcomes were assessed using age-appropriate validated instruments. Although self-report outcomes can be open to recall bias, baseline data were collected before randomisation, instruments were used with standardised recall periods and actions at the school level are very unlikely to have biased reporting between intervention and control arms. Analysis used an intention-to-treat approach and appropriate mixed-effects models.

Our data are subject to a number of limitations. In terms of generalisability, our trial was carried out in a representative sample of urban and periurban settings in and around London. Although our study did not include schools in rural England, our process evaluation identified no factors that might suggest that implementation or effects would be different in such schools. The absence of students at some waves may have introduced bias. However, given that non-responders are more likely to have experienced bullying or behaviour problems, this is likely to have underestimated the intervention effect. The large number of secondary outcomes investigated necessitated multiple statistical testing. However, to mitigate this, we restricted ourselves to testing only prespecified secondary and subgroup analyses. A sensitivity analysis allowing for a very stringent Bonferroni correction to the p-values led to little change in our conclusions; the evidence of an effect of the intervention remained for QoL, psychological total difficulties, ever smoked regularly, been offered or tried illicit drugs and number of times really drunk. Subgroup analyses were underpowered. We have not yet assessed the impact of the intervention on staff outcomes or on educational outcomes (attainments, attendance and exclusions), as these rely on routine administrative data which will be available 1–2 years after the end of the trial. We used the GBS, a well-established tool, to measure the occurrence of bullying victimisation in schools. The measure aligns with the WHO definition of bullying described in Chapter 1. 8 It focuses on different forms of abuse committed face to face or online, as well as the frequency of victimisation and the hurt caused. The tool aligns less well with some other definitions of bullying, such as that of Olweus,11 which focuses on repeat victimisation from the same perpetrator(s) as well as power imbalances between perpetrator(s) and victim. We used the ESYTC school misbehaviour subscale to assess broader forms of violence and aggression at school. 72 It should be noted that, as well as assessing forms of behaviour that clearly involve interpersonal aggression, the measure includes items on some behaviours that might best be regarded as anti-social, such as arriving late for classes, refusing to do work, wandering around school in class time and cheating when doing homework or tests.

Self-report of student outcomes, although providing the most useful data for the majority of outcomes, has limitations. There is evidence that peer reports identify a higher prevalence of bullying perpetration than self-reports,140 but we are unaware of any evidence that peer reports identify a higher prevalence of bullying victimisation than self-reports. The use of multiple respondents may have improved the accuracy of mental health data. Parental report or the use of linked routine administrative data is likely to have improved the accuracy of policy contacts and health service use. Our process evaluation was both broad in collecting data from all schools and deep in exploring some processes in depth in case study schools. It also enabled a longitudinal evaluation of implementation across 3 years. Although response rates for process data collection were generally high, some aspects, such as the surveys of teachers delivering the curriculum, had lower response rates. Despite this, our multisource approach meant that we were still able to assess what was happening in most schools, with the exception of a few schools in year 3, when it is reasonable to assume that few, if any, intervention activities were being implemented.

Fidelity of implementation of LT was variable. Some schools in the control arm implemented activities that resembled some elements of LT. However, only five control schools implemented what might be regarded as three of the key elements of our intervention (RP, social and emotional skills education and student participation in decision-making) and implemented these with less support than was the case in intervention schools. A per-protocol analysis excluding these control schools found similar intervention effects. A sensitivity analysis excluding the six case study schools selected for more intensive process evaluation data collection and showed no discernible differences in intervention effects from the intention-to-treat analyses, although CIs were wider as these analyses were underpowered.

The economic evaluation also had some limitations. We have not incorporated the staff outcomes (HRQoL) into the QALY estimates, although we do report physical and mental health scores of staff in the cost–consequences analysis. This was because it is difficult to know how staff and student QoL should be combined. It is possible that we have not captured all of the costs incurred in the control arm, as these were difficult to capture without changing the control arm status. Finally, we have only conducted a within-trial analysis and have not modelled the longer-term impact of bullying on health outcomes, costs and QoL.

Implications for research and practice

Our study adds to the evidence that whole-school approaches to preventing bullying and aggression and promoting student health are feasible to implement and have positive effects on a range of outcomes in a broad range of high-, middle- and low-income settings. 57,60,61,144 LT offers the potential for broad improvements in behaviour and health in secondary schools and, as the first RCT of school-based RP, provides strong support for further development of restorative approaches in secondary schools. The results are important for public health policy in that a single, very low-cost intervention had an impact on a clustered set of outcomes of public health importance including bullying, mental health, well-being and QoL, as well as on the use of tobacco, alcohol and drugs. The findings are also of broader scientific interest in that they provide the first experimental evidence that it is possible to promote multiple health outcomes by transforming the school environment and increasing educational engagement rather than via health services or traditional health education, as we theorised.

The wider value of LT must be examined in further trials in diverse settings, such as nations within the UK that have different education systems (e.g. Scotland). We recommend that some future studies examine the effectiveness of interventions that do not include a curriculum component in order to assess whether or not whole-school elements alone are sufficient to achieve health outcomes. We also recommend that other studies examine the effectiveness of an intervention that incorporates a strengthened curriculum element, where more thought is given to ensuring that lessons are culturally appropriate and can be delivered within busy school timetables. Curriculum components may prove more feasible to deliver in schools or school systems that have dedicated time in school timetables for personal, social and/or health education. We further recommend that future studies explore whether external facilitation is essential to initiate implementation or training for schools on implementing whole-school change would be an alternative. We recommend that, in future studies, training be repeated annually to compensate for staff turnover.

We are currently undertaking further analyses to investigate which elements of the intervention may have been the most beneficial. The poor fidelity for the curriculum element suggests this was less likely to have contributed significantly to the benefits detected, although it is possible that schools were already implementing social and emotional learning in other ways.

Future work will examine staff and educational outcomes, explore the sustainability of the intervention within LT intervention schools once the trial was complete, and examine hypothesised mechanisms by which the intervention had an impact on outcomes and whether or not these varied between schools. Work is also needed to establish the generalisability of our findings. However, given that participating schools were representative of those invited to participate and included a good range in terms of attainment, deprivation and inspectorate ratings, this suggests that LT may achieve similar effects in other schools in England and beyond. Given the extent to which control schools were implementing quite similar activities to those included in the intervention, effects of LT might be greater in settings and countries where schools are less involved in such activities. Future work is needed to examine broader implementation of LT and evaluate any modification of the intervention.

Patient and public involvement

This trial has strongly benefited from service user and public involvement. Our trial involved young people from the National Children’s Bureau young researchers’ group (YRG) in advising on intervention and research methods.

We consulted with the YRG on refinements to the intervention and trial methods during the set-up phase. This ensured that the intervention remained acceptable and appropriate to young people, as they were in the pilot. Three meetings took place with the YRG on these issues. Note that training and support to the YRG was routinely provided by the National Children’s Bureau. At the trial summation, the YRG was again consulted about the meaning of the results to students and schools, and the most effective ways to disseminate the findings to schools and charities. One group’s feedback on this is provided in Appendix 2.

School AGs during the intervention ensured that implementation in their school was appropriate to local needs that were identified by members and the survey of student needs. This included ensuring that revisions to policies and rules built on existing work, deciding which curriculum modules to deliver in each year, and implementing locally decided actions aiming to improve relationships and student participation (e.g. cascading restorative practice training to staff who had not attended or to student peer mentors).

The study student advisory group comprised students and staff drawn from intervention schools in the pilot who had signalled their willingness to undertake this. This was used to consolidate learning from the pilot, particularly regarding implementation issues.

We convened an advisory group of research and service users. This included representatives who were key policy-makers and clinicians. This group met twice during the study and provided an opportunity for the research team to consult about the research design and methods for data collection, choice of outcomes and methods for data analyses. The advisory group also aided in interpreting the initial findings and developing the dissemination strategies.

Research governance

Contributions of authors

Chris Bonell (Professor, Public Health Sociology) co-led development of the LT intervention, conceptualised the study, led design of the trial, led the research and writing of the manuscript. He directed the study, together with Russell M Viner. Chris Bonell conducted the trial’s progress review, including staff and resources. He ensured that the trial was administered in a financially responsible manner. The research team was managed by Russell M Viner with Chris Bonell. Chris Bonell led the process evaluation and intervention teams. Chris Bonell planned and led the process evaluation.

Elizabeth Allen (Professor, Medical Statistics) contributed to the trial design, and was responsible for randomisation of schools, for entry and cleaning of all quantitative data, and for all quantitative analyses.

Emily Warren (Research Fellow, Behaviour and Intervention Use) was responsible for process evaluation.

Jennifer McGowan (Research Associate, Health Psychology) conducted the review of the progress of the trial, looking at staff and resources issues. She reported to the NIHR on study’s progress and achievements. She had day-to-day responsibility for the conduct of the trial and the operations of the research team, and writing of the final report.

Leonardo Bevilacqua (Research Assistant, Developmental Psychopathology) conducted schools recruitment, dissemination of surveys in schools, overall management of the trial, and led the process evaluation activities.

Farah Jamal (Social Researcher, School Environment) conducted schools recruitment, surveys in schools, overall management of the trial, and led the process evaluation activities.

Zia Sadique (Lecturer, Economic Evaluation) conducted the economic evaluation with Rosa Legood.

Rosa Legood (Associate Professor, Health economics) conducted overviews of progress, advice, implementation, publications and dissemination strategy. She was responsible for the economic evaluation.

Meg Wiggins (Research Officer, Education) managed the intervention team at the Institute of Education. She was part of the Scientific Steering Committee.

Charles Opondo (Honorary Assistant Professor, Medical Statistics) conducted the qualitative statistical analysis with Elizabeth Allen.

Anne Mathiot (Trial Manager) conducted the review of the progress of the trial, looking at staff and resources issues. She reported to the NIHR on study’s progress and achievements, before passing the role to Jennifer McGowan.

Joanna Sturgess (Research Fellow, Clinical Trials) was responsible for randomisation of schools, for entry and cleaning of all quantitative data, and for the quantitative analyses. She managed the trial data and contributed to the analysis.

Sara Paparini (Honorary Research Fellow, Social and Environmental Health) contributed to the fieldwork and analysis within the process evaluation.

Adam Fletcher (Senior Lecturer, Public Health Improvement) conducted overviews of the intervention progress, advice, implementation, publications and dissemination strategy.

Miranda Perry (Education Consultant) was responsible for the information required for delivering the intervention in the schools.

Grace West (Research Assistant, Developmental Psychology) conducted schools recruitment, dissemination of surveys in schools, overall management of the trial, and led the process evaluation activities.

Tara Tancred (Assistant Professor, Public Health) contributed to the fieldwork and analysis within the process evaluation.

Stephen Scott (Professor, Child Health and Behaviour) conducted overview of the intervention progress, implementation, publications and dissemination strategy.

Diana Elbourne (Professor, Health-Care Evaluation) contributed to the trial design, was responsible for randomisation of schools, for entry and cleaning of all quantitative data and for quantitative analyses. She also conducted overviews of the intervention progress, advice, implementation, publications and dissemination strategy.

Deborah Christie (Professor, Paediatric and Adolescent Psychology) conducted overviews of the intervention progress, advice, implementation, publications and dissemination strategy.

Lyndal Bond (Professor, Population Health and Evaluation) conducted overviews of the intervention progress, advice, implementation, publications and dissemination strategy.

Russell M Viner (Professor, Adolescent Health) co-led development of the LT intervention, conceptualised the study, led design of the trial, led the research and led the writing of the manuscript. He directed the study, together with Chris Bonell as co-director. Russell M Viner conducted the trial’s progress review, including staff and resources. He ensured that the trial was administered in a financially responsible manner. The research team was managed by Russell M Viner with Chris Bonell. Russell M Viner led the outcome evaluation team.

All authors contributed to writing the manuscript, with the exception of Farah Jamal.

Publications

Bonell C, Allen E, Christie D, Elbourne D, Fletcher A, Grieve R, et al. Initiating change locally in bullying and aggression through the school environment (INCLUSIVE): study protocol for a cluster randomised controlled trial. Trials 2014;15:381.

Hargreaves D, Bevilacqua L, Shackleton N. Association between ethnic minority status at school and bullying and mental health outcomes in early adolescence: cross-sectional analysis of data from the INCLUSIVE study. Lancet 2015;386:S42.

Jamal F, Fletcher A, Shackleton N, Elbourne D, Viner R, Bonell C. The three stages of building and testing mid-level theories in a realist RCT: a theoretical and methodological case-example. Trials 2015;16:466.

Bonell C, Shackleton N, Fletcher A, Jamal F, Allen E, Mathiot A, et al. Student- and school-level belonging and commitment and student smoking, drinking and misbehaviour. Health Educ J 2016;76:206–20.

Bonell C, Warren E, Fletcher A, Viner R. Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Belle et al. Trials 2016;17:478.

Shackleton N, Hale D, Bonell C, Viner RM. Intraclass correlation values for adolescent health outcomes in secondary schools in 21 European countries. SSM Popul Health 2016;2:217–25.

Bonell C, Mathiot A, Allen E, Bevilacqua L, Christie D, Elbourne D, et al. Initiating change locally in bullying and aggression through the school environment (INCLUSIVE) trial: update to cluster randomised controlled trial protocol. Trials 2017;18:238.

Shackleton N, Fletcher A, Jamal F, Markham W, Aveyard P, Mathiot A, et al. A new measure of unhealthy school environments and its implications for critical assessments of health promotion in schools. J Crit Pub Health 2017;27:248–62.

Bonell C, Allen E, Warren E, McGowan J, Bevilacqua L, LeGood R, et al. A multi-component school environment intervention reduces bullying and risky behaviour and improves mental health and quality of life: findings from the INCLUSIVE cluster randomised controlled trial. J Adolesc Health 2018;62:S9.

Bonell C, Moore G, Warren E, Moore L. Are randomised controlled trials positivist? Reviewing the social science and philosophy literature to assess positivist tendencies of trials of social interventions in public health and health services. Trials 2018;19:238.

Fantaguzzi C, Allen E, Miners A, Christie D, Opondo C, Sadique Z, et al. Health-related quality of life associated with bullying and aggression: a cross sectional study in English secondary schools. Eur J Health Econ 2018;19:641–51.

McGowan J, Bonell C, Allen E, Warren E, Aswathikutty A, Bevilacqua L, et al. The relationship between school-level factors and adolescent student well-being: cross-sectional findings from the INCLUSIVE trial. J Adolesc Health 2018;62:S14–15.

Bonell C, Allen E, Warren E, McGowan J, Bevilacqua L, Jamal F, et al. Effects of the Learning Together intervention on bullying and aggression in English secondary schools (INCLUSIVE): a cluster randomised controlled trial. Lancet 2018;392:2452–64.

Bonell C, Allen E, Opondo C, Warren E, Elbourne D, Sturgess J, et al. Examining intervention mechanisms of action using mediation analysis within a randomised trial of a whole-school health intervention. J Epidemiol Community Health 2019;73:455–64.

Bonell C, Beaumont E, Dodd M, Elbourne D, Bevilacqua L, Mathiot A, et al. The effects of school environments on student risk behaviours: evidence from longitudinal study of secondary schools in England. J Epidemiol Community Health 2019;73:502–8.

Bonell C, Allen E, Opondo C, Warren E, Elbourne DR, Sturgess J, et al. Examining intervention mechanisms of action using mediation analysis within a randomised trial of a whole-school health intervention. J Epidemiol Community Health 2019;73:455–64.

Warren E, Bevilacqua L, Opondo C, Allen E, Mathiot A, West G, et al. Action groups as a participative strategy for leading whole-school health promotion: results on implementation from the INCLUSIVE trial in English secondary schools. Br Educ Res J 2019;45:979–1000.

Data-sharing statement

All data requests should be submitted to the corresponding author for consideration. Access to anonymised data may be granted following review.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the PHR programme or the Department of Health and Social Care.

Data sources for England averages

Because recruitment was undertaken in 2014, we used the most recently available data for 2013/14. Data for England averages were taken from Department for Education statistics (www.gov.uk/government/organisations/department-for-education/about/statistics, accessed 24 April 2017) and www.compare-school-performance.service.gov.uk/ (accessed 25 July 2017).