Interventions integrating health and academic education in schools to prevent substance misuse and violence: a systematic review

Design

The project is a multimethod systematic review of known existing research. This chapter describes the review methods. Chapter 3 describes the flow of studies through the review as well as the characteristics of included studies (addressing research question 1), study reports and interventions. This is followed by four chapters presenting our various syntheses.

1. A thematic synthesis of the literature describing the theories of change of interventions included in this review (addressing research question 2).

2. A thematic synthesis of process evaluations exploring what factors facilitate or limit successful implementation and receipt (addressing research question 3).

3. A narrative synthesis and meta-analysis of the effects of interventions on substance use and violence outcomes (addressing research question 4). The evidence reviewed did not enable us to examine how effects varied according to students’ sociodemographic characteristics.

4. An analysis of intervention components assessing key features of interventions integrating academic and health education (this reports a post hoc analyses not included in the original protocol1).

The evidence reviewed did not allow us to examine research question 5, concerning what characteristics of interventions, deliverers, school contexts and students appear to moderate or are necessary and sufficient for the effectiveness of such interventions. Chapter 7 of this report draws on all the component syntheses to develop overall conclusions.

Criteria for considering studies for this review

The criteria and definitions used for considering which studies to include in this review are outlined below. These inclusion criteria were operationalised into exclusion criteria to inform the screening of found studies (see inclusion and exclusion criteria in the protocol1). The results of this screening process are detailed in Chapter 4.

Search strategy

Information management

All citations identified by our searches were uploaded and managed during the review process using the EPPI-Centre’s specialist online review software, EPPI Reviewer version 4.0 (Evidence for Policy and Practice Information and Coordinating Centre, University of London, London, UK). This software records the bibliographic details of each study, where studies were found and how, reasons for their inclusion or exclusion, descriptive and quality assessment codes, text about each included study and the data used and produced during synthesis. The software also enabled us to store and track electronic documents [e.g. portable document files (PDFs)].

Study selection

An exclusion criteria worksheet was prepared, informed by our inclusion criteria (see above) and with guidance notes. For the screening of references identified from electronic databases, this was piloted by four reviewers (CB, TT, GMT and AF) who screened 50 references in pairs on title and abstracts. Pilot screening results were discussed by pairs of reviewers involved in screening to ensure consistency in applying the criteria. This process was invaluable because, despite being guided by clear inclusion criteria, decisions about whether or not an intervention aimed to integrate health and academic education were not always easy. The definition of integration in terms of full and partial integration of health and academic education was also helpful in making clear decisions about which studies to include or exclude. A 90% agreement rate was required before proceeding to screening by single reviewers, with each reviewer screening discrete subsets of the full set of references. Full reports were obtained for those references judged as meeting our inclusion criteria based on title and abstract, or for which there was insufficient information from the title and abstract to judge inclusion or exclusion. These reports were then screened to determine inclusion. The reviewers piloted the procedure by screening full reports and working in pairs screening 50 reports each and discussing any differences in opinion. A 90% agreement rate was required before proceeding to independent screening of the full set of references. The principal investigator (CB) reviewed all studies identified as potentially includable in the review as a final check to determine inclusion, identify which review question they answered and group multiple reports from the same study. Citations identified from websites were screened online based on their title, title and abstract or full text, when available. Potentially includable studies were cross-referenced with the electronic searches imported to EPPI-Reviewer to identify any unique references. As is customary with searching of this type, only the included references, not the excluded ones, were recorded.

Data extraction

Quality assessment

Synthesis of results

User involvement

We conducted two sets of one-to-one consultations to reflect on our findings with young people and with policy and practice stakeholders. Individual consultation was used instead of group meetings because of the impossibility of finding a date when all could attend a meeting. The first set of consultations was conducted from December 2016 to June 2017. As per the protocol,1 we aimed to discuss the validity of our typology of interventions and synthesis of process evaluations and identify the feasibility and acceptability of such interventions in the UK. Our original intention was to then use these discussions to determine which specific interventions should be included in a secondary analysis of outcome data focused on interventions most relevant for the UK. However, it was clear from our typology of interventions and synthesis of process evaluations, as well as from consultation with stakeholders, that it was not possible to identify a discrete subset of interventions that were relevant to the UK. All interventions were potentially relevant to the UK with adaptation. The adaptations required would concern the detail of the intervention materials rather than the overall intervention approaches and theories of change.

Stakeholders’ views were sought about the potential feasibility and acceptance of integrated academic and health education within the UK (see Appendix 14). We asked the following general questions:

1. Could this type of intervention be delivered within the UK?

2. If so, which intervention characteristics (e.g. delivered within primary or secondary schools, delivered within specific academic classes, integration being ‘full’ or ‘partial’, intervention being facilitated by teachers, etc.) would be the most appropriate?

3. Do you think schools would be receptive to this type of intervention in the UK?

4. What factors would enable or inhibit these types of interventions in the UK?

In addition, based on some preliminary findings from the review, as well as on the review team’s previous experiences, we explored the following hypotheses with stakeholders:

1. Interventions of this sort will, in general, be more feasible in primary rather than secondary schools because the timetable is more flexible and the emphasis is on core skills (e.g. literacy) that such interventions often address.

2. Interventions will be more feasible when they target an academic subject that is part of the national curriculum for students of that year group.

3. Interventions that aim to integrate health with academic subjects that not all schools or students will study will have much less reach (e.g. drama).

4. Interventions that focus on students beyond year 9 will be less attractive to schools because they will be perceived as reducing curriculum time for exam preparation.

5. Interventions that aim to fully integrate health into existing academic education will be more feasible than those which provide discrete new curricula that include separate health and academic components.

6. Interventions that are to be delivered in regular, timetabled health education lessons (e.g. PSHE) will be less feasible because many schools, especially at the secondary phase, do not run such lessons.

7. Interventions that include the whole school alongside classroom elements will be feasible in principle but the more complex the school-level components are the less likely it is that they will be implemented with fidelity.

8. Intervention components that aim to reach out to parents will generally not be well delivered, especially at the secondary level.

9. Interventions that require teachers to attend training away from school for > 1–2 days will not be feasible because schools will be reluctant to release teachers for this amount of time.

The second round of consultations occurred in September/October 2017, with the aim of reviewing the validity and usefulness of our syntheses to inform how our research outputs are structured and disseminated. Participants were given a summary of study findings (see Appendix 14), inclusive of outcome evaluation synthesis results. Furthermore, we discussed stakeholder views around next steps in terms of knowledge translation, replication studies and the creation of new interventions.

Consultations involved the following adult policy stakeholders: the Ariel Trust (Paul Ainsworth), Mentor UK (Michael O’Toole), Public Health Wales (Mary Charles), the School Health Research Network (Joan Roberts), Public Health England (Claire Robson), the PSHE Association (Jonathan Baggaley), the Association for Young People’s Health (Ann Hagell and Emma Rigby), London Fields Primary School (Sindee Bass), St. Saviour’s Church of England Junior School (Nick Bonell), Barnhill Community High School (Greig Pilkington) and Newstead Wood School (Jonathan Lewis).

Young people were consulted via the Advice Leading to Public Health Advancement (ALPHA) young people’s public health research advisory group based in the Centre for Development and Evaluation of Complex Interventions for Public Health Improvement (DECIPHer) for Public Health Improvement, a collaboration between the universities of Cardiff, Bristol and Swansea. Ten young people were consulted, half of whom were female and half of whom were male, and the age range was from 14 to 18 years.

Ethics arrangements

This project was approved by the Research Ethics Committee (REC) of University College of London Institute of Education (ethics approval reference number REC 746). The project complied with the Social Research Association’s Ethical Guidelines81 and guidance from the National Co-ordinating Centre for Public Engagement. 82

Rate of report publication

Figure 2 provides a breakdown of rate of publication, based on the 68 reports included in the review and according to which review question they answered. Only six studies were published between 1981 and 2000,100,108,122,125,131,135 and 60 from 2001 to 2015,3,51–57,60,83–99,101–107,109–121,123,124,126–130,132–134,137,139–141 reaching peaks in 2009, 2010, 2013 and 2015, each with six publications per year. An additional two papers were published in 2017. 136,138 Of the 68 studies, 12 (19%) published only on research question 2 (theory),54–56,99,102,105,106,112,119,121,122,124 five (8%) published only on research question 3 (process),89,117,120,123,126 23 (27%) published only on research question 4 (outcomes),51,60,87,90,91,101,104,109,111,114–116,128–132 another 17 (25%) published on research questions 2 and 4 (theory and outcome),3,52,53,83,88,92,93,95–98,100,103,110,113,125,134 10 (16%) published on research questions 2 and 3 (theory and process),57,84–86,94,107,108,118,133,135 and none was published on process and outcomes together. Only one publication127 answered research questions on theory, process and outcomes.

FIGURE 2.

Rate of study publication by research question, by year. RQ, research question.

Geographical location of included studies

Of the 30 empirical studies of 19 interventions indicated in Table 1, 23 (77%) reported on a study conducted in the USA. 3,51–54,56,57,83–87,92,93,95–101,103,104,108–111,113–118,123–125,128–134 There were four studies conducted in Australia (13%),60,88–91,94,119–122 two studies conducted in the UK (7%)107,126,127 and one study each from Canada (3%)107 and Israel (3%). 135 An additional study was carried out in the Democratic Republic of Congo (3%) (Figure 3). 137,138

FIGURE 3.

Spread of studies by country. DRC, Democratic Republic of Congo.

Phase of schooling targeted by interventions

There was a fairly even split between interventions aimed at primary school students (n = 9)3,51–57,93,96–101,103,104,106,109–112,117,123–127,131,134,136–138 and secondary school students (n = 6),60,84,88–91,108,121,122,128,132,133,135,139–141 with four interventions targeting both primary and secondary school students (Figure 4). 55,83,85–87,92,94,95,102,105,107,113–116,118–120,129,130 Note that primary school is defined as encompassing years 1–6 and secondary school as years 7–12. These year groups were given different labels by authors in different settings (e.g. elementary school and high school for studies in US settings).

FIGURE 4.

Intervention target populations.

Providers of interventions

Not surprisingly, teachers of the academic curriculum were also the primary facilitators for 13 interventions. 3,51–57,83,85–87,93,96–104,106,108,112–120,123–125,129–131,133–141 Five interventions engaged teachers and external facilitators together, often with the external facilitator acting in more of a mentoring role with the intention that the teacher might take over fully at a later date. 60,84,88–92,94,95,105,107,121,122,126–128,132 Only one intervention, Youth Matters, was fully externally facilitated. 109–111

Intervention components

Twelve interventions had only a classroom component (Figure 5). 3,51–57,84,94,96,97,102,105–112,119,120,123,128,132–136,139–141 One intervention had classroom and family components,126,127 four had classroom and school-wide components55,60,83,85–91,93,102–104,113–118,121,122,129,130,137,138 and two had classroom, school and family components. 92,95,98–101,124,125,131

FIGURE 5.

Intervention components.

Classroom-only strategies sought principally to deliver content in the classroom context (e.g. 4Rs,3,51–57 the Bullying Literature Project,96,134 I-LST84,128,132 and Youth Matters109–111 were all principally focused on classroom-based activities). Although participants in Youth Matters109–111 presented their work in school-wide fora, it was not clear from the intervention’s theory of change whether or not this was central to the intervention’s outcomes (i.e. the goal was not to effect school-wide change).

Classroom and whole-school change strategies often linked academic and health education to school-level initiatives with the goal of reflecting messages about social–emotional learning and pro-social behaviour throughout the school, by altering the school environment and by providing school-level resources to support classroom-level learning. The Gatehouse Project88–91,122 and Positive Action55,83,85–87,102,113–116,118,129,130 both focused on school climate beyond the classroom. The Gatehouse Project88–91,122 included the development of an adolescent health team in each school. This adolescent health team, which was supported by intervention developers and evaluators, was tasked with the dual foci of enhancing adolescent attachment to school and creating a locally relevant intervention strategy based on school-level profiles of risk and protective factors. Similarly, Positive Action55,83,85–87,102,113–116,118,129,130 included the formation of a ‘climate intervention’ led by the head teacher or a designated co-ordinator to facilitate school-level change in pro-social norms and behaviour. Steps to Respect blended school climate with specific out-of-classroom resources. School-wide bullying policies were audited and developed via a targeted staff committee, and participants in bullying episodes (both aggressors and victimised students) received targeted coaching from a counsellor after an episode was observed or reported. PATHS55,97,106,112,123 included school-level events and materials to support classroom-level learning.

Classroom and external domain strategies sought to influence young people’s development both in and beyond the classroom. A frequent focus beyond the classroom was the family. KAT126,127 included a parents’ evening at the conclusion of the intervention, during which students could display the work they had completed on safe alcohol consumption. The goal of the parents’ evening, as well as the goal of the materials (e.g. DVDs, leaflets) provided to parents, was to involve parents in conversations about how to time alcohol consumption. PATHS55,97,106,112,123 similarly included ‘take-home’ materials in the form of homework on social–emotional learning to be completed with parents and newsletters on programme topics to facilitate parental involvement. LIFT98–101,125 included several components to facilitate family involvement. Parents were able to call a classroom telephone line when the classroom teacher recorded details of the current intervention activities. They were also invited to attend a series of parents’ evenings to discuss parenting skills and strategies. Although RHC92,95 also included parent workshops, it was a ‘wrap-around’ intervention strategy that included extensive out-of-school enrichment and activities, including summer camps, to target out-of-school domains.

Primary academic subject within which the intervention is delivered

English was the most common academic subject into which intervention curricula were integrated (n = 7) (Figure 6). 3,51–57,93,96,103,104,108–111,117,134,136–138 One intervention was integrated into drama classes,93,119,120 one into an art class,133 one into a chemistry class135 and four into more than one subject. 55,84,94,97,105–107,112,123,126–128,132 Five interventions did not specify which academic subject they should be integrated into. 55,60,83,85–92,95,98–102,113–116,118,121,122,124,125,129–131,139–141 None of the included interventions was integrated into specific health classes although in some interventions (e.g. KAT and the Gatehouse Project) authors indicated that a health class would be suitable for the intervention to be integrated into. 60,88–91,121,122,126,127

FIGURE 6.

Primary academic subject into which health education is integrated.

Extent of integration of health and academic education

A key axis of variation in the interventions that included studies evaluated was the extent of integration; that is, the degree to which lessons were fully integrated (with activities addressing health and academic learning objectives simultaneously and seamlessly) or partially integrated (with interventions comprising separate learning activities for health and academic learning objectives within one overall package).

In 13 out of the 19 interventions (68%),3,51–57,60,84,88–91,94,96,105,107–111,119–122,126–128,132–138 integration was full in the sense that interventions included activities aiming to address health and academic learning objectives seamlessly. Interventions involving full integration did not attempt to separate health-related and academic-related learning activities. The clearest example of this in the included interventions is I-LST,84,128,133 in which the rationale for integration was to step away completely from delivering the LST content in a separate class. Instead, teachers presented and reflected LST content throughout academic subjects. In addition, the Gatehouse Project88–91,122 targeted English classes for the delivery of content on social–emotional learning based on cognitive behavioural principles. Local integration was not the only way in which full integration was achieved. The children’s stories delivered as part of 4Rs3,51–57 and Youth Matters109–111 were intended to support literacy, language and arts teaching in schools, and subsequent lessons on social–emotional learning reflected and extended literacy-based learning. In addition, student projects created as part of the Youth Matters interventions were shared with the school community.

In 4 out of the 19 interventions (32%),55,83,85–87,92,93,95,97–104,106,112–118,123–125,129–131,139–141 integration was more partial, typically these interventions included activities separately addressing health and academic objectives but within one overall package. Interventions involving partial integration set aside time specifically for health and academic learning. For example, Positive Action55,83,85–87,102,113–116,118,129,130 included a set of manualised lessons to be delivered on a weekly basis across the multiple years of the intervention. RHC,92,95 LIFT98–101,125 and PATHS55,97,106,112,123 included discrete content on study skills. PATHS also provided materials for teachers to integrate content into their lessons and in RHC teachers were provided with professional development on linking academic success with behavioural development.

Approach to integration

There were three overall approaches to integration, via the use of literature, local development of integration or linking to other developmental issues and concerns. In interventions using a literature-based pathway to integration, children’s books were used as a starting point for the discussion of social–emotional learning skills. Of note is that all six interventions (seven evaluations) that primarily used a literature pathway to integration sought to address violence outcomes, including, in particular, bullying. For example, 4Rs3,51–57 used weekly lessons divided into seven units in which each unit began with a children’s book. Social–emotional learning was supported with lessons following on from the unit’s book. The Youth Matters109–111 intervention followed a similar format, with a story used to lead into modules relating to issues or skills (i.e. peer norms around bullying or how to respond to bullying). The Bullying Literature Project96,134 used children’s books over five sessions as a starting point to engage in a variety of role-playing, writing and behavioural modelling activities to practice the lessons offered by each book, each session included both literature and follow-up activities. The Steps to Respect93,103,104,117 intervention sequenced literature integration differently; students began with social–emotional skills lessons and then participated in a classroom ‘unit’ based on children’s literature to reinforce lessons on pro-social skills. Even when we judged that literature was not the primary strategy used by an intervention, this approach was present. For example, the Gatehouse Project targeted English classes to integrate cognitive–behavioural principles in the lessons delivered.

In interventions using a local development pathway to integration, teachers developed their own ways of connecting health and academic education. 88–91,122 In all cases, teachers were supported to do this by pre-existing materials and curricula on health and social–emotional well-being subjects. Three interventions (three evaluations) primarily used this approach to integrate academic and health education. I-LST84,128,132 departed from the original LST programme in that lessons were not provided as part of discrete PSHE classes. Instead, teachers of academic subjects in participating schools developed a ‘matrix’ of the lessons and topics from the LST manual against academic subjects and developed a local plan for introducing relevant LST material across different classes. The underlying idea was to reinforce material across different academic subjects and potentially increase the intervention ‘dose’ that students would receive compared with a standard health education class. Although extensive detail regarding KAT126,127 was not provided in the main trial report, classroom teachers delivered lessons relating to alcohol consumption and students participated in pairs and groups to produce work for a parents’ evening. Thus, integration of academic and health education was specific to each classroom. Finally, PATHS55,97,106,112,123 provides materials to teachers (e.g. posters and homework) to support the integration of PATHS activities on social–emotional skills into daily instruction on academic subjects. This strategy was also present in other interventions, although we judged that it was not the primary strategy used. For example, Positive Action55,83,85–87,102,113–116,118,129,130 encouraged teachers to locally integrate Positive Action content into academic classes, even though Positive Action was also delivered via discrete lessons.

Interventions that used a linking approach to the integration of academic and health education emphasised the inter-relationships between academic success and ‘successful’ child and adolescent development, specifically in terms of physical health and social–emotional well-being. This strategy was the primary approach used in four interventions (five evaluations). Positive Action,55,83,85–87,102,113–116,118,129,130 throughout its extensive manualised curricula, links academic success with health and well-being. Thus, lessons taught to students on, for example, self-regulation and social skills, are linked to academic lessons, social–emotional development and positive health behaviours. Skills taught to students are also intended to influence both pro-social behaviours and academic success. The Gatehouse Project88–91,122 took a similar approach, emphasising the basis of the intervention as a global understanding of child and adolescent development; thus, although drug use and bullying victimisation outcomes were measured, these were more distal impacts of the intervention, as opposed to the intervention’s primary concern with social–emotional and behavioural development of adolescents. In LIFT,98–101,125 the linking of academic and developmental concerns was achieved via delivering study skills content alongside social–emotional learning. RHC92,95 focused on both study skills content and on co-delivering academic enrichment with behavioural management and substance use refusal, first by classroom intervention and subsequently by extracurricular opportunities. This strategy was also present in other interventions, although we judged that it was not the primary strategy used. For example, PATHS55,97,106,112,123 linked to developmental concerns by presenting study skills content alongside social–emotional learning content in the last year of the intervention.

Intervention inputs

Our review focused on school-based interventions and these overwhelmingly involved health curricula delivered in classrooms. Some interventions also included after-school or out-of-school components, whereas others had whole-school or family components. As an example of a multicomponent intervention, the Positive Action curriculum:

. . . teaches specific positive actions for the whole self: the physical, intellectual, social, and emotional areas. Together, these make up the comprehensive set of skills for successful learning and living [. . .] The program trains teachers and parents to identify, teach, and reinforce positive thoughts, actions, and feelings about themselves by students and others in the school, leading to continual reinforcement of positive actions and enhanced student bonding with parents and school.

Flay and Allred102

Some interventions required the training of teachers or other staff to deliver the curriculum. We have included it as an input (the dark green boxes in Figure 7) in our overall logic model for these interventions. Some curricula were externally facilitated and, therefore, no internal training of school staff was required.

Curricula addressed a wide variety of topic areas and desired skills. Of note, although our review outcomes were reductions of violence and substance use, only one intervention dealt directly with bullying (the Bullying Literature Project96,134). Overwhelmingly, the curricula concentrated on conflict resolution, problem-solving skills and social–emotional skills, as an example from the 4Rs intervention indicates:

The approach of embedding social-emotional learning and conflict resolution lessons in a balanced literacy delivery strategy, and research tying together the social-emotional and academic domains, support our expectation for longer term effects on behavior and academic achievement.

Jones et al. 3

Mechanisms of change: improving health by eroding boundaries

As reported above, the central theme that emerged from our analysis was that eroding boundaries at multiple and mutually reinforcing levels (e.g. by integrating academic and health education, strengthening relationships between teachers and students, generalising learning from classrooms to the wider school environment and ensuring consistent messages from schools and families) will lead to the development of a community of engaged students oriented towards pro-social behaviour and away from substance use, violence and other risk behaviours.

Hypothesised intervention outcomes

Most of the interventions aimed to reduce aggression, violence or bullying in young people through the mechanisms outlined above, usually in combination. Proximal outcomes, commonly highlighted across the interventions, that led to reduced violence included the development of pro-social skills, emotional intelligence (broadly understood as self-awareness) and self-management (encompassing managing emotions and responding constructively to bullying and conflict). These proximal outcomes were theorised as arising out of the mechanisms described above. Most authors made an overall assumption that antisocial behaviours occur because of deficits in these social and emotional skills.

For example, the Roots of Empathy intervention:

. . . significantly improved the attitudes, knowledge and social emotional competencies of all participating teachers and children. A decrease in the frequency of bullying and an increase in pro-social behaviours were reported by all participants.

Cain and Carnellor94

Hypotheses arising from the theory synthesis

The discussion above suggests the following hypotheses:

1. Interventions that involve multiple components to erode multiple boundaries (e.g. health and academic education, staff–student, classroom–school and school–family) will be more effective than interventions that aim only to erode boundaries between health and academic education.

2. Interventions that aim for full integration of health education into academic subjects will be more acceptable to and less liable to resistance from students and, thus, more effective than interventions that deliver health and academic educational activities but without full integration.

3. Intervention effects on substance use and violence outcomes may be greater among students of low SES, as they are the most likely to benefit from boundary erosion.

It was not apparent from the synthesis of theories of change that there was a discrete subset of interventions that are of particular relevance to the UK. All of the theories of change seemed plausible within a UK context.

On potential benefits

Adult policy stakeholders were positive about the theorised mechanisms of the potential for reductions in violence and substance use, as were some young people. Other young people wondered whether or not interventions might be effective in reducing in-school but not out-of-school risk behaviours.

On multilevel programmes

Overwhelmingly, multilevel programmes with both a whole-school and a family component were seen as likely to be the most effective. Stakeholders suggested that a school ethos supportive of the programme’s goals and approaches would facilitate whole-school programme components. Stakeholders commenting on the Welsh experience highlighted that the Welsh Assembly Government has promoted a health-promoting schools approach that would be conducive to multilevel programming. Young people generally agreed that developing stronger relationships between staff and students would be an important element of multilevel programmes.

It was generally perceived that a whole-school approach may be more feasible in primary schools, given that there is more flexibility. In primary schools, there may be dedicated time to health themes, for example a antibullying week, during which there may be approaches beyond the classroom to extend an integrated, health-promoting curricula.

The inclusion of parents, or having a family component, was universally agreed by stakeholders to be desirable. It was suggested that parents and families play an even stronger role in influencing student behaviour than school curricula or the school environment and, as such, parents and families performing a reinforcing role around health messaging was regarded as highly valuable, if possible. However, it was also agreed that engaging parents could be very challenging, particularly parents of students in secondary school. Parents of higher-risk students who may already be working with school officials were viewed as being easier to engage.

On full integration into academic subjects

It was generally perceived that a fully integrated approach would be more likely to be feasible in primary schools, given that there may be more flexibility in the curriculum. In primary school, teachers engage with students for a full day and may teach one health topic across many academic subjects; whereas in secondary school, teachers typically see students only for specific academic subjects. Participants generally felt that some subjects (e.g. drama or art) may offer particularly appropriate platforms for practical skill development. There was also clear recognition that, if integrated programmes were able to support the achievement of academic learning objectives, receptiveness would generally be high. However, it was also pointed out that pay raises and teacher job security can hinge on their ability to support their students in achieving specific learning objectives, so it may be perceived as a personal risk to adopt a programme that may compromise core academic learning objectives. Where school leaders are judged partly on the basis of objectives concerning health and well-being, integrated programmes would be more likely to be taken up. If a school is judged largely based on academic performance then these programmes may be less likely to be taken up, despite their attempt to bridge health and academic learning objectives. Some stakeholders also suggested that some schools may prefer to implement health programmes that are clearly labelled as such, rather than engage in the more complex task of integrating health and academic education.

Stakeholders agreed that all of the intervention theories of change seemed plausible within a UK context.

Support from school senior managers and the broader political environment

Teachers’ immediate working environment

Teachers’ perceptions of their school as possessing a generally supportive culture for them and other teachers was viewed as enabling higher-quality implementation. This mindset was crucially linked to teachers’ confidence in their teaching environment as one that could support the intervention, which was in turn linked to teachers’ own motivation for intervention delivery. This could have a direct impact on implementation. Three subthemes emerged from 10 studies. 57,84,89,94,108,118,120,123,127,135 Unfortunately, only one study among these123 was designated as a high-quality study with very reliable findings, because of insufficient reporting on methods in the other studies. However, in terms of usefulness of findings, these studies explored important characteristics of implementation and, in some instances, had an explicit focus on the processes of integrating health and academic curricula. 84,108

Teachers’ attitudes towards intervention characteristics

A key theme in six studies86,89,94,118,120,135 concerned the acceptability to teachers of the interventions themselves. This theme linked to the earlier theme about teachers’ views on how supportive the school climate was of the implementation of these interventions. Save for two studies120,135 that were poorly reported, the findings from these studies were designated as having medium to high reliability and usefulness. This subtheme was emphasised across studies as a factor enabling successful implementation to a greater extent than the acceptability of the intervention to students (see Teachers’ belief in and ownership of the curriculum), probably because teachers were typically the primary deliverers of the interventions.

Students’ attitudes towards intervention characteristics

Parental support

External political will

Each of the four countries within the UK have different policies for health and wellness in schools. Unfortunately, we did not speak with stakeholders from Scotland or Northern Ireland, but the differences between England and Wales are stark. In England, PSHE is not a statutory subject. In Wales, Personal and Social Education is part of the basic curriculum for pupils aged 7–16 years at maintained schools. 153 However, as of 2016 in the UK, Ofsted has introduced evaluation criteria linked to student health and welfare and personal development. 40 Integrated health and academic curricula may therefore help to address these wellness-based objectives as well as academic ones. As such, there may be varying levels of receptiveness to integrated health and academic curricula depending on the country context and the emphasis placed on health and wellness alongside academic attainment. The bottom line, according to stakeholders, was simply that there must be political prioritisation of the content within an integrated curriculum for it to have a chance at being taken up.

Alignment with evaluated learning objectives

As we already introduced, in the previous stakeholder consultation feedback on the findings from our synthesis of intervention theory, integrated health and academic curricula were generally viewed positively, but there was consensus that they would be far more likely to be taken up if they helped to meet academic learning objectives, such as the standards assessed by Ofsted. Strongly related to the previous consultation feedback, stakeholders consistently reiterated the need for political will and for this to be communicated to schools, for example via school metrics and inspection standards.

Young people on the ALPHA group could readily see the interconnections between reduced substance use and violence and improved academic attainment. Violence and other antisocial behaviour in schools could hamper learning. Student substance use could impede ability to learn. Adult policy stakeholders acknowledged that it was widely accepted that improved health among students leads to better academic outcomes, but that these relationships are difficult to evaluate.

Internal political will

There were comments from stakeholders on the necessity of buy-in, from both administrators, implementers (usually teachers) and students. The first point raised was that schools must see available programmes as having the capacity to address an issue that the school already recognises as a problem. For example, substance use may not be perceived by all schools as an issue that needs to be addressed and, therefore, programmes emphasising substance use, regardless of whether or not they are integrated, may not be taken up.

It was reiterated on a number of occasions that there must be champions throughout the school who are willing to take up and support an integrated curriculum. A strong evidence-base for the effectiveness of these programmes to achieve outcomes was mentioned by most stakeholders as being a necessary and compelling factor that would influence uptake. Additionally, it was conveyed that students must also see the programme as being relevant, which was a point made by young people on the ALPHA group as well as by the adult policy stakeholders.

Beyond having enthusiasm for a programme, from a very practical point of view, adequate budget allocation by the school to integrated health and academic curricula was highlighted, as well as sufficient time to train teachers (see Effective teacher training). Young people made the point that programmes would not be well delivered when teachers lacked the time and resources to prepare themselves for this form of teaching. Additionally, having trained staff within the school who could mentor implementers and address any problems was seen as highly useful, but often lacking in many health programmes currently offered.

Effective teacher training

Teacher training was a point of interest. Many of the stakeholders had either led or been participants in continuing teacher training. There was a general agreement that it would be very challenging to get full days of teacher time to do training on integrated curricula. However, it was agreed that training in person, with lots of interactive activities and opportunities for role play, would generally be more effective than simply providing teachers with written materials. It was generally agreed that online training could be useful to supplement learning, but that it should not replace face-to-face training. The need to train the trainers was seen as being very important and also relates to having the necessary supports within schools. Furthermore, this type of model would allow a programme to be scaled up for delivery to a wider population.

One important consideration regarding training for delivering integrated health and academic curricula in general is that programmes adopting this platform must be flexible to tailor materials to each school’s individual context. However, given the importance of an evidence base, it was also felt that programmes should have core components delivered with high fidelity to try to reproduce positive outcomes achieved elsewhere.

Differences between primary and secondary schools

In addition to the factors outlined earlier in this chapter, there were a number of reflections on the feasibility of integrated health and academic education in primary schools versus secondary schools. Although it was generally agreed that such interventions could be taken up by secondary schools, it was viewed as considerably less likely. In secondary schools, a narrower focus on academic attainment often consumes the seemingly constantly reducing space for health and wellness curricula. Teachers in secondary schools face enormous pressure for their students to perform well academically and may be less likely to risk academic attainment by giving space to health education, especially within core academic subjects. With fewer classes such as art and drama, which may provide a more natural platform for these integrated curricula, especially when practical skill development is emphasised, delivery in secondary schools may be further challenged. Other comments from both policy stakeholders and young people included the difficulty in discussing substance use and violence in secondary schools where students may already be engaged in these behaviours. Furthermore, it was felt that primary schools, which often use topic-based learning across several different academic lessons, may be much more appropriate platforms for integrated curricula than secondary schools.

Comparison with alternative personal, social and health education delivery models

One stakeholder drew our attention to McWhirter et al. ’s154 Understanding Personal, Social, Health and Economic Education in Secondary Schools, in which different approaches to delivering PSHE are described, with an indication of the advantages and disadvantages of each approach. This source acknowledges teaching PSHE across the curriculum within other subjects as one potential approach. The authors highlight some key advantages, such as avoiding the need for discrete time slots for PSHE and increasing ownership of the curriculum across the school. Among the disadvantages highlighted, there are some that resonate with our own findings. Related to ‘internal political will’, McWhirter et al. 154 emphasise the need for ‘absolute’ commitment from the school leadership team. There are also several other key points raised that were not necessarily reflected in our findings, nor in our consultations with stakeholders. First, when health and academic objectives are brought together, there is a risk that the academic objectives may be seen as having lesser importance and, indeed, emphasis on health topics may become tokenistic. Second, if health education is integrated across a number of different subject lessons, then there is a risk that it will be difficult to ensure consistency and continuity. Third, the risk that evaluating learning objectives related to both health and academic education may prove challenging for teachers when monitoring student progress. 154

Random generation of allocation sequence

Of the 16 included evaluations, 10 evaluations, reported on in 21 publications, did not report enough information to judge the risk of bias arising from allocation sequence generation. 60,87,88,90–93,95–97,103,104,109–111,128–130,132,134,136 The remaining six evaluations (4Rs,3,52,53 KAT,127 LIFT,98,100,125 Second Step,139–141 Learning to Read in a Healing Classroom137,138 and Positive Action Chicago83,113–116) all presented enough information to appraise these evaluations as having a low risk of bias in this domain. Both 4Rs52 and Positive Action Chicago115,116 used random number generators, whereas LIFT drew allocations from a hat. 125 KAT used optimal allocation to determine the randomisation sequence. 127

Allocation concealment

In total, 15 out of 16 included evaluations did not state if or how allocation was concealed, thus preventing the determination of the risk of bias in this domain. Learning to Read in a Healing Classroom137,138 was assessed as having a low risk of bias because of reported methods of concealment.

Blinding

Although blinding is often difficult in trials of school interventions, blinding of outcome assessors may often be possible and is useful for reducing ascertainment bias. Of the 16 included evaluations, only one provided enough information to judge whether or not any blinding occurred. In LIFT,98,125 outcome assessors were blind to allocation. This evaluation was therefore scored as being at a low risk of bias.

Complete outcome data

In total, 15 out of 16 evaluations were judged as having a relatively low or balanced attrition when this was relevant to outcome assessment. In some evaluations, such as the evaluations of Positive Action,87,113–116 whole-school intervention implementation meant that students were followed up at a certain grade level regardless of the duration of their time in the school or whether or not baseline data were collected. Learning to Read in a Healing Classroom137,138 was assessed as having a high risk of bias because of the high levels of missing data.

Complete reporting

Of the 16 evaluations, 12 were judged as being at an unclear risk of selective reporting. This was generally due to a lack of a trial protocol. Two evaluations, 4Rs51 and Second Step,139–141 were judged as being at a high risk of bias from selective reporting because of unclear and incomplete reporting of findings from the 3-year results. Bullying Literature Project–Moral Disengagement and Learning to Read in a Healing Classroom were both judged as being at a low risk of bias because all expected outcomes were reported.

Confounding addressed

Although randomisation should on average produce groups balanced for known and unknown confounders, one of the issues arising in school-randomised or classroom-randomised trials is that the relatively small number of higher-level units (i.e. schools) can produce imbalances between arms. Of the 16 evaluations, 10 were at a low risk of bias from confounding. In these evaluations, confounding was addressed by including a vector of person-level confounders in multivariate regression models. The remaining six evaluations were at an unclear risk of confounding because inadequate information was presented in outcome evaluations.

Accounting for clustering

Evaluations generally were at a low risk of bias owing to inadequate assessment of clustering because clustering was generally well assessed. Of the 16 evaluations, 13 (reported in 35 publications) used appropriate analytical methods to account for clustering;3,51–53,60,83,87,88,90,91,93,97,98,100,101,103,104,109–111,113–116,125,127–130,132,137–141 for example, robust standard errors in the Gatehouse Project90,91 or generalised linear mixed-effects models to nest students within schools (e.g. 4Rs, Positive Action Hawaii and Chicago, PATHS and Youth Matters3,51–53,83,87,109–111,113,114,116,129,130). One evaluation, I-LST,97,115,128,132 presented results that did not account for clustering but observed that results accounting for clustering were not different. One evaluation had an unclear risk of bias (RHC92,95) and two had a high risk of bias in this domain (Bullying Literature Project96,134 and Bullying Literature Project–Moral Disengagement136).

Additional sources of bias

Most evaluations (14/16, reported on in 35 publications)3,51–53,60,83,87,88,90–93,95,97,98,100,101,103,104,113–116,125,127–130,132 used a variety of methods to reduce other sources of bias; for example, additional methods of addressing missing data or sensitivity analyses for different model assumptions. Two evaluations, Youth Matters110^;111 and Bullying Literature Project,96,134 did not present the relevant information to permit a clear determination of other sources of bias.

Suitable control group

Of the 16 evaluations, 10 (reported in 29 publications) reported enough information to judge that the intervention and control groups were similar on key characteristics or accounted for differences in the analysis. 3,51–53,60,83,87,88,90–92,95,109–111,113–116,128–130,132,136–141 However, six evaluations (reported in 12 publications) did not provide enough information to judge whether or not the intervention and control groups were sufficiently similar; the evaluations did provide evidence of baseline differences between the groups but did not demonstrate how these differences were accounted for. 93,96–98,100,101,103,104,125,127,131,134

A visual summary of possible sources of bias by intervention is presented in Figure 8.

FIGURE 8.

Possible sources of bias within eight dimensions by intervention.

Grading of Recommendations, Assessment, Development and Evaluations assessment of primary outcomes

The quality of evidence for outcomes by KS is presented in Table 3. For most combinations of outcome and KS, the quality of evidence was low or very low. However, for substance use outcomes in KS3 and 4 and for violence victimisation outcomes in KS2, quality of evidence was moderate.

Key stage 2 substance use outcomes

Of the three evaluations presenting substance use outcomes for KS2, one evaluation, Positive Action Chicago,116 presented an omnibus outcome comparing counts of substances used. Both the KAT127 and Positive Action Hawaii87 interventions presented outcomes relating to alcohol use. In addition, Positive Action Hawaii87 presented outcomes relating to smoking and illicit drug use.

Key stage 3 substance use outcomes

Five evaluations reported outcomes relating to substance use in KS3: the Gatehouse Project,60,90 LIFT, I-LST,128 Positive Action Chicago114 and Raising Health Children. 92 Of these, the Gatehouse Project60 and Positive Action Chicago114 reported omnibus substance use outcomes. Alcohol, smoking and illicit drug use outcomes were reported by the same five evaluations: the Gatehouse Project,60,90,91 LIFT,101 I-LST,128 Positive Action Chicago114 and RHC. 92

Meta-analyses

As specified in the protocol,1 we undertook separate meta-analyses for alcohol use, smoking and illicit drug use. Because the preponderance of outcomes relating to illicit drug use measured marijuana use, we also meta-analysed marijuana outcomes separately as an exploratory analysis. We did not undertake meta-analysis for omnibus substance use outcomes alone given that only two evaluations would have been included. Finally, as specified in the protocol,1 we undertook an overall meta-analysis of substance use outcomes in KS3.

Interventions integrating academic and health education had a marginally non-significant effect in reducing alcohol use among students in KS3 (d = –0.11, 95% CI –0.23 to 0.004) (Table 6 and Figure 9). However, this analysis, which included 23 effect sizes across five evaluations, had substantial between-studies heterogeneity (I² = 54%). Meta-analyses were not indicative of an effect of these interventions for smoking specifically (d = –0.05, 95% CI –0.12 to 0.02) (Figure 10; see Table 6), based on 13 effect sizes from five evaluations. However, these interventions were effective in reducing illicit drug use (d = –0.07, 95% CI –0.14 to –0.01) and marijuana use specifically (d = –0.10, 95% CI –0.16 to –0.04) (Figure 11; see Table 6). Unlike the meta-analysis of alcohol outcomes, meta-analyses for smoking and illicit drug use all had negligible between-studies heterogeneity (I² = 0%). A pooled analysis across all substance use outcomes, including 55 effect sizes from five evaluations, suggested a small but statistically significant global effect of these interventions (d = –0.09, 95% CI –0.17 to –0.01), albeit with some between-studies heterogeneity (I² = 35%) (Figure 12; see Table 6).

TABLE 6 - Summary of KS3 substance use outcomes

Outcome	k (n)	Standardised MD (95% CI)	I2, study level (%)
Alcohol	5 (23)	–0.11 (–0.23 to 0.004)	54
Smoking	5 (13)	–0.05 (–0.12 to 0.02)	0
Illicit drug use: marijuana	5 (10)	–0.10 (–0.16 to –0.04)	0
Illicit drug use	5 (14)	–0.07 (–0.14 to –0.01)	0
All drug use outcomes	5 (55)	–0.09 (–0.17 to –0.01)	35

FIGURE 9.

Key stage 3 substance use outcomes (alcohol): effect size.

FIGURE 10.

Key stage 3 substance use outcomes (smoking): effect size.

FIGURE 11.

Key stage 3 substance use outcomes (illicit substance use): effect size.

FIGURE 12.

Key stage 3 substance use outcomes (all): effect size.

Key stage 4 substance use outcomes

Three evaluations reported outcomes relating to substance use in KS4: the Gatehouse Project,90,91 I-LST132 and RHC. 92 All outcomes reported related to alcohol use, smoking and illicit drug use; that is, no included evaluations reported omnibus substance use outcomes in KS4. All evaluations reported each of these three outcome types.

Meta-analyses

As specified in the protocol, we undertook separate meta-analyses for alcohol use, smoking and illicit drug use (namely, marijuana use). We also undertook an overall meta-analysis of substance use outcomes in KS4. Meta-analyses for alcohol use integrating 15 effect sizes from three evaluations did not suggest that interventions integrating health and academic education had a statistically significant impact on alcohol use (d = –0.01, 95% CI –0.09 to 0.06) (Table 7 and Figure 13). This analysis had some minor statistical heterogeneity, with a between-studies I² of 2%. In contrast, meta-analyses for smoking suggested a small but statistically significant effect of these interventions in reducing smoking (d = –0.08, 95% CI –0.15 to –0.01) (Figure 14; see Table 7). In this analysis, which included nine effect sizes from three evaluations, between-studies heterogeneity was negligible (I² = 0%). Interventions also appeared to have a positive effect in reducing marijuana use (d = –0.10, 95% CI –0.17 to –0.03) (Figure 15; see Table 7, based on 10 effect sizes from three evaluations. A pooled analysis integrating all 34 effect sizes from the three evaluations suggested a small but significant overall effect of these interventions in reducing substance use (d = –0.06, 95% CI –0.09 to –0.02) (Figure 16; see Table 7). Between-studies heterogeneity was negligible in this analysis as well (I² = 0%).

TABLE 7 - Summary of KS4 substance use outcomes

Outcome	k (n)	Standardised MD (95% CI)	I2, study level (%)
Alcohol	3 (15)	–0.01 (–0.09 to –0.06)	2
Smoking	3 (9)	–0.08 (–0.15 to –0.01)	0
Illicit drug use: marijuana	3 (10)	–0.10 (–0.17 to –0.03)	0
All drug use outcomes	3 (34)	–0.06 (–0.09 to –0.02)	0

FIGURE 13.

Key stage 4 substance use outcomes (alcohol): effect size.

FIGURE 14.

Key stage 4 substance use outcomes (smoking): effect size.

FIGURE 15.

Key stage 4 substance use outcomes (illicit drug use): effect size.

FIGURE 16.

Key stage 4 substance use outcomes (all): effect size.

Key stage 5 substance use outcomes

At the end of the eighth year from the start of the intervention, participants in the original year 6 cohort from LIFT98 were significantly less likely to have initiated alcohol use (OR 0.93; p < 0.05) or smoking (OR 0.90; p < 0.01) and less likely to have initiated illicit substance use at marginal non-significance (OR 0.91; p < 0.10).

Trajectories of substance use

Two studies presented analyses of the trajectories of substance use behaviours: LIFT98 and RHC. 92 The interpretation of trajectory-based analyses (also described as longitudinal or growth curve models) is different from end point-based analyses. In trajectory-based analyses, the effectiveness of the intervention is measured in terms of how different the intervention group and the control group are in change over time in the outcome of interest. Thus, in these analyses, the effectiveness of the intervention as compared with the control is estimated via a regression model, where the interaction term of intervention condition and time is the parameter of interest.

In LIFT,98 students were followed up over 8 years to examine changes in frequency of tobacco use, alcohol use and illicit drug use. The intervention did not affect change over time in substance use behaviours in the whole population, but intervention recipients were less likely to use tobacco (β = –0.10; p < 0.05), alcohol (β = –0.15; p < 0.001) or illicit drugs (β = –0.12; p < 0.05). When authors tested a model including the moderating effect of sex on intervention effectiveness, females receiving the intervention had lower levels of average illicit drug use (β = –0.11; p < 0.05), and slower linear (β = –0.14; p < 0.05) and quadratic (β = –0.11; p < 0.05) growth over time. Females also had slower linear (β = 09.14; p < 0.001) and quadratic (β = –0.13; p < 0.05) growth over time in tobacco use.

In RHC,92 change in substance use over time was examined over 4 years. Authors estimated frequency of use and prevalence of use jointly in their model, and examined a diversity of model fits. The intervention did not have a significant effect on change in the prevalence of alcohol use over time (years 7 to 9: β = –0.005, SE 0.198; years 9 to 11: β = 0.047; p = 0.190). In contrast, although the intervention did not result in a significant reduction in frequency of use over time in school years 7 to 9 (β = –0.29, SE 0.095), the intervention did result in a significant decrease in change in frequency of use over time in school years 9 to 11 (β = –0.199, SE 0.096; p < 0.05). That is, after both groups increased frequency of alcohol use between school years 7 to 9, the intervention group reduced frequency of alcohol use faster than the control group in school years 9 to 11. Similarly, although the intervention did not significantly impact change over time in prevalence of marijuana use (linear: β = 0.055; p = 0.104; quadratic: β = –0.008, SE 0.143), students receiving the intervention had a greater decrease over time in frequency of use of marijuana (β = –0.223, SE 0.052; p < 0.001). That is, intervention students reduced frequency of marijuana use more quickly than control students. The intervention group was not associated with linear or quadratic changes in prevalence (linear: β = –0.153, SE 0.105; quadratic: β = –0.123, SE 0.155) or frequency (linear: β = –0.008, SE 0.042; quadratic: β = –0.033, SE 0.092) of cigarette use over time.

Key stage 2 violence perpetration outcomes

Findings for violence perpetration in KS2 were inconsistent across evaluations. Unlike findings for the year 2 cohort in LIFT,125 effects of the intervention in the year 6 cohort at the end of the first intervention year on observed physical aggression in the playground were constant over values of the pre-intervention score [d = –0.14 at mean, 1 standard deviation (SD) and 2 SDs above the pre-intervention mean]; these findings were also described as ‘statistically significant’.

After the first intervention year (year 4) of 4Rs,52 intervention participants were not different from control participants on teacher-reported aggression (b = 0.02, SE 0.05, based on a 1–4 scale). In contrast, after the second intervention year,2 students who received 4Rs had lower levels of teacher-reported aggression than their control peers (d = –0.21; p < 0.05).

The Bullying Literature Project, implemented in year 4 and year 5 classrooms, did not generate a change in physical aggression by either teacher report on individual students (intervention group: mean 1.12, SD 0.47, n = 95; control group: mean 1.19, SD 0.47, n = 55; p = 0.67) or student report (intervention group: mean 1.20, SE 0.44, n = 90; control group: mean 1.14, SD 0.36, n = 42; p = 0.84) when measured at 1 week post intervention. 96 This finding was the same in the Bullying Literature Project–Moral Disengagement version (F_1,80 = 0.83; p = .431), implemented in year 4 classrooms only, although only combined student-reported physical and emotional bullying estimates were available. 136

In PATHS, which was implemented starting in year 4 classrooms,97 student-reported and teacher-reported aggression were measured using multilevel models incorporating all waves of measurement and effect sizes estimated corresponding to each wave of measurement; thus, standard errors for effect sizes are not provided directly. Small positive effects of the intervention on student-reported aggression at the end of the first intervention year (d = –0.048, 95% CI –0.189 to 0.092) and at the start (d = –0.064, 95% CI –0.205 to 0.076) and end (d = –0.048, 95% CI –0.188 to 0.093) of the second intervention year gave way to a small deleterious effect of the intervention at the end of the third intervention year (d = 0.082, 95% CI –0.060 to 0.224). The intervention had an opposite effect on teacher-reported aggression, with initially small but negative intervention effects at the end of the first intervention year (d = 0.036, 95% CI –0.105 to 0.178) and at the start of the second intervention year (d = 0.035, 95% CI –0.107 to 0.178), suggesting progressively greater effects of the intervention at the end of the second (d = –0.005, 95% CI –0.146 to 0.136) and the third (d = –0.199, 95% CI –0.338 to –0.060) intervention years.

At the end of the third intervention year (year 6) in Positive Action Chicago,116 students reported both lower counts of bullying behaviours (IRR 0.59, 95% CI 0.37 to 0.92) and of serious violence-related behaviours, including ‘cut[ting] or stab[bing] someone on purpose’ (IRR 0.63, 95% CI 0.45 to 0.88). Findings from Positive Action Hawaii87 were similar for both counts of student-reported violent behaviours (IRR 0.42, 90% CI 0.24 to 0.73) and teacher-reported violent behaviours (IRR 0.54, 90% CI 0.30 to 0.77). For students in year 6 (the fourth or fifth intervention year), intervention recipients were less likely to report cutting or stabbing someone (OR 0.29, 90% CI 0.16 to 0.52) or shooting someone (OR 0.24, 90% CI 0.14 to 0.40). Teachers were less likely to report that students hurt others (OR 0.61, 90% CI 0.38 to 0.97) or got into a lot of fights (OR 0.63, 90% CI 0.47 to 0.84).

In the first evaluation of Steps to Respect,104 the intervention, implemented among students ranging from year 4 to year 7, was associated with decreased bullying based on playground observation (F_91.3 = 5.02; p < 0.01) at the end of the first intervention year, but not direct aggression based on student report (F_68.7 = 2.05; p > 0.05). The second evaluation of Steps to Respect,93 implemented among students ranging from years 4 to 6, revealed a similar pattern. Although teacher reporting of physical bullying perpetration was less in intervention schools than in control schools at the end of the first intervention year (OR 0.61, t₂₉ = –3.12; p < 0.01), student reports did not suggest a difference between schools on bullying perpetration (t₂₉ = –1.06).

Finally, at the end of the second intervention year (year 6) of Youth Matters,110 students in intervention schools were not less likely to report violence perpetration (OR 0.85, 95% CI 0.29 to 1.47; p = 0.585). Evaluators of Youth Matters also explored the use of a latent class analysis method to classify intervention recipients as either victims, bullies or bully–victims. Proportions of intervention and control recipients classified as bullies or bully–victims were not significantly different at the end of the first (intervention group: 21%, n = 356; control group: 22%, n = 392) or second (intervention group: 19%, n = 244; control group: 23%, n = 293) intervention years. 109

Key stage 3 violence perpetration outcomes

The three evaluations examining violence perpetration outcomes in KS3 had dissimilar results. At the end of the sixth intervention year (year 9) in Positive Action Chicago,115 students receiving the intervention reported a lower count of violence-related behaviours than their peers receiving the control intervention (IRR 0.38, 95% CI 0.18 to 0.81; equivalent to d = –0.54). Students also reported fewer bullying behaviours (d = –0.39), and parents reported that their children engaged in fewer bullying behaviours (d = –0.31). Significance values for these estimates were not presented, but both were supported by significant condition-by-time interactions in multilevel models, indicating that the intervention group showed an improved trajectory over time compared with the control group. In contrast, after the third intervention year (year 7) in Youth Matters,109 proportions of students were not different in the collective bully and bully–victim groups (both groups, 16%; intervention group, n = 283; control group, n = 289). Finally, findings for Second Step were reported at the end of the first, second and third years of intervention. At the end of the first school year (year 7), students in intervention schools had decreased odds of physical aggression (OR 0.70; p < 0.05) but not sexual harassment and sexual violence perpetration (OR 1.04; p > 0.05). 139 These findings did not hold to the end of the second school year for physical aggression (OR 0.80, 95% CI 0.59 to 1.08); however, sexual harassment and sexual violence perpetration was significantly reduced in intervention schools in Illinois (OR 0.72, 95% CI 0.54 to 0.95) but not Kansas (OR 0.99, 95% CI 0.71 to 1.48). 140,141 At the end of the third school year, there were no direct effects of Second Step on sexual harassment perpetration (β = 0.005, SE 0.012); findings for physical aggression were not available.

Trajectories of violence perpetration

Five evaluations presented analyses of trajectories of violence perpetration behaviours: 4Rs,3 PATHS,97 Positive Action Chicago,115 Steps to Respect103 and Youth Matters. 110,111 Evaluations were inconsistent in their findings, and comparison of findings was made challenging by the different time operators used in analyses.

In 4Rs,3 teacher-reported aggression was examined in four waves of data collection over 2 years. The intervention was associated with a decrease in the rate of change in aggression over time (β = –0.03, SE 0.01; p < 0.05). This difference was standardised as a difference in growth parameters of 0.05 SDs, indicating a small intervention effect. In PATHS,97 data on teacher-reported aggression were collected up to five times over 3 years. Neither the linear interaction between intervention and time (β = 0.400, 90% CI –0.246 to 1.047) nor the quadratic interaction between intervention and time (β = –0.137, 90% CI –0.263 to 0.011) suggested a significant intervention effect in change of aggression over time. However, the general pattern was an increase in aggression in both groups followed by a faster rate of decline in the intervention group starting during year 5. Similarly, findings for student-reported aggression did not reveal a significantly different linear (β = –0.195, 90% CI –0.517 to 0.127) or quadratic (β = 0.049, 90% CI –0.013 to 0.111) difference between intervention and control groups in change over time.

In Positive Action Chicago,115 data on bullying were collected eight times by student reports and six times by parent reports over the duration of the trial. Both student and parent reports suggested that the intervention slowed the rate of increase in bullying behaviours. In models examining student reports, the linear interaction between intervention and time suggested a slower rate in the increase of count of bullying behaviours reported (IRR 0.85, 95% CI 0.76 to 0.95), although this was slightly offset by a quadratic interaction term (IRR 1.02, 95% CI 1.00 to 1.04). In models examining parent report, the linear interaction between intervention and time suggested a similarly slower rate in increase of bullying behaviours (IRR 0.92, 95% CI 0.89 to 0.97).

In the first evaluation of Steps to Respect, bullying behaviours were assessed by observation and by student report four times over 2 years. 103 In models analysing observation data, the interaction between intervention assignment and grade was statistically significant (F_1,202 = 12.86; p < 0.01). This translated into an increase per year in terms of bullying occasions per hour of 0.035 (p < 0.01) in the control group compared with a non-significant increase of 0.04 in the intervention group. However, student reports of direct aggression did not suggest a difference in rate of change over time (F_1,1670 < 1), with both groups increasing in average bullying behaviours reported over time (intervention group: 0.10/year; p < 0.01; control group: 0.13/year; p < 0.01).

In Youth Matters,109–111 bullying perpetration was assessed four times over 2 years. The log odds of bully status did not change to a significantly different degree in students receiving Youth Matters compared with those in the control group (β = 0.079, SE 0.090; p = 0.378). 110 This result was confirmed after 3 years of data collection. 111 Follow-up models, including latent transition analyses, suggested that more intervention students transitioned from the bully latent class to the uninvolved latent class between baseline and the end of the first intervention year (54% of bullies in the intervention group transitioned to uninvolved vs. 37% of bullies in the control group; p = 0.009) and between the end of the second intervention year and the end of the third intervention year (44% of bullies in the intervention group transitioned to uninvolved vs. 34% of bullies in the control group; p = 0.058). 109

Violence victimisation

Nine intervention evaluations reported outcomes relating to violence victimisation. These outcomes were reported across 13 papers. Seven intervention evaluations included violence victimisation outcomes measured in KS2, three intervention evaluations included violence victimisation outcomes measured in KS3 and one intervention evaluation included outcomes measured in KS4 (Tables 10 and 11).

Key stage 2 violence victimisation outcomes

Overall, although evaluations were similar in the follow-up period, they did not have a clear effect on the basis of reports alone on violence victimisation as measured in KS2. Intervention evaluations reporting victimisation outcomes in this period were the Bullying Literature Project,96 Bullying Literature Project–Moral Disengagement136, Learning to Read in a Healing Classroom137, PATHS,97 both the first104 and second evaluations of Steps to Respect,93 and Youth Matters. 109,110

The Bullying Literature Project, implemented in year-4 and year-5 classrooms, did not generate a change in physical victimisation by either teacher report on individual students (intervention group: mean 1.04, SD 0.23, n = 95; control group: mean 1.04, SD 0.21, n = 55; p = 0.39) or student report (intervention group: mean 1.35, SD 0.54, n = 90; control group: mean 1.43, SD 0.66, n = 42; p = 0.57) when measured at 1 week post intervention. 96 However, students receiving the Bullying Literature Project–Moral Disengagement136 version did report a decrease in victimisation (both physical and emotional combined) after the intervention (intervention group: mean 1.76, SD 0.81 to mean 1.60, SD 0.66, n = 42; control group: mean 1.23, SD 0.38 to mean 1.38, SD 0.53, n = 42), with a significant time-by-treatment interaction in an analysis of variance (ANOVA) (F_1,80 = 7.42; p = 0.047).

Learning to Read in a Healing Classroom137 examined relational and physical victimisation after 1 year of intervention implementation and found no significant effect of the intervention (weighted d = –0.01, SE 0.06). This intervention was implemented in years 3 to 5.

Promoting Alternative Thinking Strategies, which was implemented starting in year 4 classrooms, measured student-reported victimisation using standardised MDs and found small but non-significant increases at the end of the first intervention year (d = 0.044, 95% CI –0.098 to 0.185), at the start (d = 0.074, 95% CI –0.067 to 0.216) and end (d = 0.092, 95% CI –0.050 to 0.234) of the second year and at the end of the third year (d = 0.089, 95% CI –0.053 to 0.231) following the start of intervention implementation. 97 As for violence perpetration outcomes, standard errors were obtained via correspondence with authors.

Steps to Respect, which was evaluated and implemented across the range of years in the equivalent of KS2 in two different trials, did not find differences in student-reported bullying victimisation at the end of the first intervention year in either the first trial (intervention group: mean 0.80, SD 1.51; control group: mean 0.86, SD 1.44; F < 1)104 or the second trial (intervention group: mean 2.11, SD 1.03; control group: mean 2.18, SD 1.06; t₂₉ = –1.15). 93 Frey et al. 104 also undertook playground observation at the end of the first intervention year, which was suggestive of a marginally non-significant decrease in bullying victimisation (intervention group: mean 0.9, SD 0.82; control group: mean 1.01, SD 0.83; F_72.4 = 3.74; p < 0.10).

Finally, Youth Matters, which was implemented starting in Year 5, examined bullying victimisation through both continuous and dichotomous measures. Because of non-normality of continuous data, scores were log transformed. At the end of the second intervention year, the difference in log-transformed scores was suggestive of a decrease in bullying victimisation measured continuously (difference –0.171, SE 0.083; p = 0.049). 110 At the end of the second year, intervention recipients were less likely to have reported bullying victimisation as measured dichotomously, but this difference was not significant (OR 0.61; p = 0.098). Based on the latent class analysis, proportions of intervention and control recipients classified as victims or bully–victims were not significantly different at the end of the first (intervention group: 47%, n = 356; control group: 46%, n = 392) or second (intervention group: 45%, n = 244; control group: 47%, n = 293) intervention years. 109

Meta-analyses

As stated in our protocol, we undertook a meta-analysis for violence victimisation in KS2. The multilevel meta-analysis included 12 effect sizes from seven evaluations. Interventions integrating health and academic education did not have a statistically significant impact in reducing violence victimisation, with a standardised MD of –0.02 (95% CI –0.10 to 0.05) (Figure 17). Statistical heterogeneity was not large in this meta-analysis, with a between-studies I²-value of 1.7%.

FIGURE 17.

Violence victimisation outcomes: effect size.

Key stages 3 and 4 violence victimisation outcomes

Intervention evaluations presenting violence victimisation outcomes in the equivalent of KS3 and KS4 did not suggest evidence of effectiveness. Outcomes in KS3 were presented by Second Step,139–141 Youth Matters109,111 and the Gatehouse Project. 90 The Gatehouse Project also presented outcomes for KS4.

In Second Step, neither peer victimisation (OR 1.01; p > 0.5) nor sexual harassment and violence victimisation (OR 1.01; p > 0.05) were different between students in intervention schools and control schools after the first intervention year;139 this remained the case at the end of the second intervention year (peer victimisation: OR 0.94, 95% CI 0.75 to 1.18; sexual victimisation: 0.91 95% CI 0.72 to 1.15). 141

In Youth Matters, differences in the log-transformed scores for bullying victimisation were not suggestive of a decrease in victimisation in intervention recipients compared with control recipients, but this difference was no longer significant (difference –0.123, SE 0.068; p = 0.08). 111 However, at the end of the third intervention year (equivalent to year 7), fewer students in the intervention group were members of the victim or bully–victim classes than in the control group (intervention group: 36%, n = 283; control group: 45%, n = 289). 109 Based on our own chi-squared test, this difference was significant (p = 0.029).

The Gatehouse Project, which was implemented starting in year 9, did not find evidence of a change in bullying victimisation at the end of the first intervention year (OR 1.03, 95% CI 0.86 to 1.26). Evaluators also did not find a significant difference in the prevalence of bullying victimisation at the end of the second (OR 1.03, 95% CI 0.78 to 1.34) or third (OR 0.88, 95% CI 0.68 to 1.13) intervention years, which corresponded to the first 2 years of KS4.

Trajectories of violence victimisation

Three evaluations presented analyses of trajectories of violence victimisation over time: PATHS,97 Steps to Respect 103 and Youth Matters. 110,111 None of the evaluations consistently suggested that interventions were associated with decreases in rates of change in bullying victimisation over time. In PATHS,97 data on student-reported victimisation were collected up to five times over 3 years. Neither the linear interaction between intervention and time (β = 0.245, 90% CI –0.230 to 0.720) nor the quadratic interaction between intervention and time (β = –0.032, 90% CI –0.406 to 0.342) suggested a significant intervention effect in change in experiences of bullying over time. In the first evaluation of Steps to Respect,103 bullying victimisation was assessed four times over 2 years. Both groups reported non-significant decreases over time in the average number of bullying behaviours experienced (intervention group: –0.08/year; control group: –0.07/year), but these slopes were not significantly different over time (F_1,1672 < 1). Finally, in Youth Matters, bullying victimisation was assessed as both a continuous and a binary variable. Binary outcome analyses after 2 years suggested a faster decrease in the log odds of bully victim status over time, but this did not approach significance (β = –0.146, SE 0.078; p = 0.061). 110 Continuous outcome analyses after 3 years of follow-up111 did not suggest a significant difference between groups in the change in the degree of bullying victimisation over time (intervention by time: β = –0.017, SE 0.014; p = 0.236). Follow-up analyses using latent transition analysis suggested that more intervention students than control students transitioned from bullied to uninvolved between each measurement occasion [between baseline and end of first intervention year: 26% in intervention group vs. 19% in control group (p = 0.04); end of first intervention year to end of second intervention year: 36% in intervention group vs. 20% in control group (p = 0.01); end of second intervention year to end of third intervention year: 45% in intervention group vs. 38% in control group (p = 0.007)].

Key stages 1 and 2 academic attainment outcomes

Three evaluations examined academic attainment in KS1 and KS2: RHC,95 4Rs3,52 and the second evaluation of Steps to Respect. 93 Findings were not collectively suggestive of effectiveness, although meta-analysis was judged inappropriate. Although Learning to Read in a Healing Classroom138 presented effects on academic attainment, these were not presented as estimates of the intervention’s direct effects and thus are not presented here.

In RHC, students were recruited in years 2 and 3 at the start of the intervention (thus straddling KS1 and KS2). Academic performance was measured by both teacher and parent report, and reporting was stratified by sex. Because the range of the outcomes was not clear, we calculated standardised MDs (Cohen’s d) to facilitate interpretation. Based on teacher report, males in the intervention group (n = 273) performed better than males in the control group (n = 227) at the end of the first intervention year (intervention group: mean 3.25, SD 1.18; control group: mean 3.06, SD 1.21; d = 0.16, 95% CI –0.02 to 0.34) and significantly at the start of the second intervention year (intervention group: mean 3.20, SD 1.15; control group: mean 2.81, SD 1.16; d = 0.34, 95% CI 0.16 to 0.52) and the end of the second intervention year (intervention group: mean 2.21, SD 1.16; control group: mean 2.98, SD 1.24; d = 0.28, 95% CI 0.10 to 0.45). Females in the intervention group (n = 224) were similarly better performing than females in the control group (n = 214) at the end of the first intervention year (intervention group: mean 3.35, SD 1.16; control group: mean 3.18, SD 1.25; d = 0.14, 95% CI –0.05 to, 0.33), the start of the second intervention year (intervention group: mean 3.28, SD 1.21; control group: mean 3.12, SD 1.17; d = 0.13, 95% CI –0.05 to 0.32) and the end of the second intervention year (intervention group: mean 3.34, SD 1.18; control group: mean 3.26, SD 1.18; d = 0.07, 95% CI –0.12 to 0.26), but differences were not significant at any point. Differences at the end of the second intervention year in teacher-reported outcomes were tested in a multilevel model and intervention recipients had significantly improved academic performance relative to their control peers (b = 0.188; p = 0.019). Parent-reported academic achievement reflected an improvement for males at the end of the first year (d = 0.26, 95% CI 0.09 to 0.44) and at the end of the second year (d = 0.23, 95% CI 0.05 to 0.41), but not for females (end of first intervention year: d = 0.09, 95% CI –0.10 to 0.27; end of second intervention year: d = 0.10, 95% CI –0.08 to 0.29). However, differences at the end of the second intervention year in parent-reported outcomes were tested in a multilevel model and intervention recipients overall were not significantly better in terms of academic performance (b = 0.082; p > 0.05).

In the second evaluation of Steps to Respect,93 including students in years 4 to 6, teacher-reported academic achievement was not significantly better in intervention schools at the end of the first intervention year (t₂₉ = –0.15). In the evaluation of 4Rs, which included students starting in year 4, intervention recipients were not different from control recipients at the end of the first academic year on academic skill (b = –0.02, SE 0.08), maths (b = –7.40, SE 5.69) or reading (b = –5.35, SE 4.96) (36-point scale). 52 Findings at the end of the second intervention year for maths (b = –4.35, SE 10.25) and reading (b = –0.99, SE 10.73) suggested a similar pattern. 3

Key stage 3 academic attainment outcomes

The evaluation of Positive Action Chicago83 included academic attainment outcomes at the end of the sixth intervention year, when students were in year 9. Although maths outcomes from standardised tests were not reported specifically (aside from an observation that differences were not statistically significant), intervention students transitioning from year 8 to year 9 were significantly better on reading (d = 0.83; p = 0.026). However, there were only small differences in year 9 students on self-reported grades (d = 0.02) or teacher-reported academic ability (d = 0.14). CIs were not presented for these estimates, but condition-by-time interactions in multilevel models were not statistically significant for self-reported grades or for the better-fitting model in teacher-reported academic ability.

School-level academic attainment

Both evaluations of Positive Action presented school-level academic attainment outcomes. In Positive Action Chicago,83 at the end of the sixth intervention year, schools receiving the intervention were better than control schools on maths (d = 0.38) and reading (d = 0.22). Although CIs for these findings were not provided, findings from two-tailed significance tests of condition-by-time interactions in multilevel models were not significant for reading or maths.

In Positive Action Hawaii, differences between schools in percentage at or above the average for reading and for maths were examined at the end of the fourth and fifth intervention years. For each outcome and follow-up point, estimates were provided for both nationally standardised tests and Hawaii-specific standardised tests. At the end of the fourth intervention year, intervention schools were better, but not significantly so, on nationally standardised maths (g = 0.50; p = 0.495) and reading (g = 0.58; p = 0.108), and significantly so on Hawaii-specific maths (g = 0.69; p = 0.04) and reading (g = 0.72; p = 0.029) tests. Findings were similar at the end of the fifth intervention year for nationally standardised maths (g = 0.52; p = 0.291) and reading (g = 0.54; p = 0.028) and Hawaii-specific maths (g = 1.1; p = 0.006) and reading (g = 0.65; p = 0.043).

Relevance of findings

Generally, there seemed to be interest among adult policy and youth stakeholders, the former representing schools, non-governmental organisations and public health institutions. However, perhaps unsurprisingly, there were differing levels of interest. Given that our primary positive finding was a reduction in substance use in KS2 and KS3, if stakeholders were concerned with substance use, these findings were of interest. Stakeholders from schools or organisations that did not emphasise substance use were less interested. More than half of stakeholders were also very interested in outcomes related to academic achievement, and suggested that these should be given emphasis before schools might be willing to consider them. This comment resonates well with some of our findings in previous consultations when we were reflecting on process findings, as alignment with learning objectives and school targets was seen as essential.

Young people on the ALPHA group were surprised to see an effect on substance use but not on violence when the latter is much more visible in schools and, they theorised, open to reductions from this sort of school-based intervention.

Many stakeholders agreed that the findings from outcome evaluations, although necessary, should not be presented in isolation, and argued that they should be presented in the context of our findings about proposed mechanisms of action and suggested barriers and facilitators of implementation.

There was a discussion about effect sizes and whether or not our findings were compelling. Most stakeholders are conversant in appreciating public health research, and as such were not expecting enormous effect sizes. Others, however, felt that the effect sizes might need to be larger to compel administrators to take on an intervention in their school, especially if that intervention involved teacher time and resources and could potentially compromise academic learning.

Importantly, there were various comments raised about the importance of geographical context, with an appreciation that these findings are limited insofar as they mostly do not emerge from the UK and this might dampen the potential impact of such findings. Using these findings to design and implement a UK-based intervention was thought to be considerably more useful in terms of creating an evidence base that UK stakeholders would be more likely to subscribe to.

Communicating findings

There was considerable variation between stakeholders on this point. However, overwhelmingly, the suggestion that information must come in a digestible manner through a trusted channel resonated among stakeholders. ‘Digestible’ may take on many forms. Teachers felt that findings should be communicated as if for a lay audience, removing jargon and ‘academese’. Other stakeholders who are more engaged with research directly wanted full scientific explanations and lengthier reports alongside summaries. Channels like conferences, meetings and websites (or webinars to communicate research findings) were suggested as potential means of communicating findings. Mass e-mails or e-mails in general were dismissed as being unhelpful.

There was an underlying sentiment from all stakeholders that schools, teachers and administrators are bombarded with information about prospective interventions; to stand out, it must come with a ‘seal of approval’ from a trusted channel that has worked closely with the school. ‘Trusted channels’ were perceived differently, however. It was suggested that simply originating from academic institutions may be enough to make the work eye-catching, whereas others suggested that working with known non-governmental organisations or government institutions would be essential. The presentation of findings in terms of content and platforms of dissemination needs to be tailored to the audience.

Finally, stakeholders were unanimous in their understanding of the importance of engagement of an administrative team in schools in order for such interventions to be taken up. Subject heads or heads of pastoral care were suggested as key audiences, although it was noted that, with the goal of school-wide effects or possibly cross-curricular work, the head teacher or assistant head teachers would also have to be championing the intervention. As such, communicating to people occupying these types of roles was felt to be essential.

Searching and inclusion criteria

Our focus was interventions that integrate health and academic education. However, most interventions that included studies evaluated were seeking a practical way to introduce health education into the school timetable, and integration was not always included in report titles or abstracts or at the centre of the theory of change. Searching for these interventions was therefore difficult and it is possible that we missed some potentially relevant studies. Screening was also challenging. All study reports of included studies explicitly described some degree of integration between health and academic education but the extent and clarity of this description varied enormously. It is possible that we excluded studies of interventions where in practice health education was sometimes or always integrated with academic lessons but where this was not mentioned as part of the description of the intervention in study reports. Despite this risk of excluding potentially relevant studies, we think that restricting inclusion to studies of interventions where integration was explicitly referred to as part of the description of the intervention was the only way to ensure that the review had a clear focus and avoided a situation in which screening decisions required detailed examination of intervention materials or lengthy discussion with authors.

We excluded a number of quasi-experimental studies that may have provided useful insights. However, we think that our strategy of excluding these, based on the fact that this class of intervention is amenable to evaluation via RCTs that provide more accurate estimates of intervention effects, was correct.

Theory reports

There were some limitations both in our review methods and in the literature synthesised. In terms of review methods, the main challenge was in appraising the quality of reports of theory. There was little guidance available from other studies as to how to do this. As is the case in most syntheses of theory,158 no reports were excluded on the basis of quality and all contributed, albeit to different degrees, to the synthesis on the basis that they provided useful insights despite some areas of weakness. The detail provided on intervention theory varied enormously between reports. Reports providing more detail or theories that were reported on by multiple reports tended to be assessed as of higher quality. There was quite a low rate (65%) of agreement between the two reviewers assessing quality assessment. This reflected several challenges. First, with regards to generalisability, many theories of change were intervention-specific and thus it was not obvious how generalisable they were intended to be. Second, when assessing whether or not theories were based on existing evidence, we found that intervention theories of change often drew from multiple existing theories, each with a different evidence base, rendering overall assessment challenging. Finally, when judging clarity, different authors assumed different levels of prior understanding of constructs among readers, and so provided variable levels of explanations for constructs and their relationships. Reviewers thus found themselves making quite subjective decisions about whether or not authors’ assumptions about readers’ prior knowledge was reasonable. In some cases, authors providing more text describing constructs and pathways led to a higher quality score, whereas in others authors’ assumptions about tacit knowledge were considered reasonable. Because of these challenges, our assessment of quality did not greatly inform how we weighted each report when synthesising. Instead, their contribution to the synthesis was determined by the extent to which they contributed insights into theory that could be reciprocally translated with those from other reports or contribute to line-of-argument syntheses alongside other reports.

In terms of limitations of the literature synthesised, there was a general assumption in the literature that risk behaviours arise from deficits in individual assets such as knowledge, social and emotional skills and attitudes, albeit within a broader social and institutional context. However, it has been argued that behaviours such as violence or substance use emerge as symbolic practices among young people who are, for complex structural and institutional reasons, disengaged from schools in terms of learning and the social community. 75,159 Thus, behaviours such as smoking, drug use or violence might occur not merely through individual deficits but as a result of rationally chosen and socially sanctioned decisions, albeit perhaps in a context of limited choices. The lack of consideration of such mechanisms in intervention theories in this review might lead such interventions to fail to address important influences on young people’s substance use and violence because of insufficient dose or traction to transform the engagement and sense of connectedness, particularly of socially disadvantaged students. Moreover, despite some mention of the importance of relationships outside the classroom, the classroom environment was overwhelmingly the focus of theories of change for these interventions, which may fail to address the broader context in which young people develop behaviours. Furthermore, only a few study reports discussed the theorised interaction between educational and non-educational assets and how this would contribute to reductions in risk behaviours. 92,95,102,111 Theoretical models such as developmental cascades61 and positive youth development160 have the capacity to address this relationship, but they are not emphasised across our included studies.

Finally, most theories did not consider how contextual factors might interact with mechanisms of change to generate outcomes, as would be the case with theories informed by realist ontologies. 161 Haegerich and Metz162 extrapolated a summary of moderating factors for some of the same interventions in our synthesis, which included student (e.g. SES, ethnicity, sex, intervention dose) and contextual (e.g. school, family, cultural, policy) characteristics. However, discussions of the role of these factors were consistently underdeveloped in most theories in our synthesis, despite potential implications for the impact of interventions and the extent to which they might translate between different populations and settings.

Process evaluations

Our synthesis of evidence from process evaluations was somewhat limited by the paucity of in-depth evaluation of the factors that promote or hinder implementation of this category of intervention. Most process evaluations limited themselves to quantitative assessment of implementation fidelity and acceptability, and many failed to report on how implementation was affected by characteristics of interventions, deliverers, participants or school contexts. Evaluations were also very diverse in focus and methods so that thematic synthesis was challenging. Nevertheless, our synthesis did succeed in identifying recurrent and relevant themes from this diverse and rather impoverished body of literature.

Our review overall included 35 outcome evaluations but only 16 process evaluations. This imbalance is common in the literature evaluating social interventions. For example, a review of implementation data by Michie et al. 163 found that only 5–30% of publications of experimental studies had detailed intervention descriptions at all. This lack of reporting of intervention and process presents challenges to those aiming to use existing evidence to translate interventions to new settings, a problem noted by several authors across disciplines. 163–168

Outcome evaluations

One challenge that arose in our meta-analysis of outcome evaluations was the limitation of statistical methods in accounting for trajectory-based evidence. For example, the effectiveness of RHC92,95 was most clearly evidenced in the examination of differences in trajectories of frequency of substance use. Examination of trajectories is appropriate in complex interventions aiming to target longitudinal processes of child and adolescent social development, especially as the goal of some interventions was explicitly to target developmental cascades to interrupt the evolution of negative behaviour patterns. This contrasts with ‘standard’ trials, in which effects are measured at prescribed end points. Although a trajectory-based analysis is an intuitive choice for interventions integrating health and academic education, especially when those interventions are hypothesised to have temporally distal effects, meta-analytic methods are not available to synthesise evidence arising from these analyses meaningfully. In addition, trajectory analyses may be most appropriate when considering the uptake or frequency, rather than the initiation, of harmful behaviours. Differences in RHC92,95 in last-year use of substances were not in evidence, but intervention and control groups clearly diverged in the frequency of both alcohol and marijuana use over time.

Our ‘matrix’-based meta-analysis method had both strengths and limitations. One strength is that it could examine effects in specific developmental phases of relevance to educators and intervention implementers. However, one limitation is that it involved combining estimates across a diversity of follow-up times. In the narrative synthesis, we attended to the follow-up times attendant to each outcome measurement; however, the few studies included in each meta-analysis precluded examination of heterogeneity by follow-up time.

A consistent problem across assessments of violence perpetration was that measures were frequently indirect, that is, they measured broader constructs such as ‘aggression’, despite our attempts to locate the most proximal estimate of violent behaviours possible. Measures also drew on a diversity of informants, including student reports, teacher reports and observation, but findings were rarely consistent across informants.

Underlying theories of change for interventions integrating academic and health curricula

An overarching theory that emerged was that eroding boundaries at multiple and mutually reinforcing levels – by integrating health and academic education, by transforming relationships between teachers and students, by generalising learning from classrooms to the wider school environment and by ensuring consistent messages from schools and families – will lead to the development of a community of engaged students oriented towards pro-social behaviour and away from substance use, violence and other risk behaviours.

The interventions in this review aimed to integrate and thus erode boundaries between health and academic education. They also generally aimed to train teachers to model positive behaviours, reinforce such behaviours among students and enhance their relationships with students. We characterised these as fitting within an overarching theme of boundary erosion, but in doing so clarified that this means transforming relationships from being instrumentally focused on academic learning to being more affective and focused on students’ broader development. This work in the classroom was supported by other components delivered at multiple levels (e.g. the overall school environment and the family), which aimed to ensure that learning and reinforcement of positive behaviours occurred beyond the classroom. We interpreted this mode of delivery as a mechanism that aimed to erode boundaries between what occurred in classrooms and other settings such as the wider school and students’ families. Through these multiple but theoretically related mechanisms, interventions were thus intended to promote experiential learning, the practising and reinforcement of positive behaviours and enhanced relationships between teachers and students and between students and their pro-social peers. In turn, it is theorised that these will provide students with various attributes necessary to reduce engagement in substance use and violence as well as to increase academic attainment. These attributes include the knowledge and skills necessary to avoid substance use and prevent conflict, the attitudes and social norms supportive of such behaviours and students’ sense of security in and positive connectedness to their classrooms and schools.

Ultimately, eroding boundaries between health and academic education, teachers and students, classrooms and the wider school, and schools and families was theorised as being critical to establishing new frameworks of family, classroom or school organisation that are conducive to promoting both academic and social–emotional outcomes.

Characteristics affecting implementation of interventions integrating health and academic curricula

To answer our original research question, ‘what characteristics of interventions, deliverers, participants and school contexts facilitate or limit successful implementation and receipt of such interventions?’, we summarised facilitators of and barriers to the implementation of our interventions of interest. Although contributing factors varied widely depending on the intervention, several – often linked – themes did emerge from our synthesis, namely concerning the necessity of support from senior school managers and a conducive broader policy environment, a positive teaching environment, positive pre-existing teacher and student attitudes towards integrated health and academic interventions, favourable staff opinions about the autonomy and innovation that the interventions enabled and parental support of interventions.

It is worth noting that many of the themes above relate to factors affecting implementation that might apply generally to school-based health promotion or social and emotional learning interventions. Here, we aim to draw out what our synthesis suggests about factors affecting the implementation of our specific category of interventions that integrate health and academic education. First, this category of intervention particularly benefits from consistent cross-school support from administrators and colleagues in integrating health across the curriculum. Strong networks, continuous training and shared understanding about the overall aims of integration take time to build and efforts to sustain; thus, ongoing support from administrators, both practically and in terms of morale, is crucial. Second, interventions need to be flexible and locally adaptable if they are to mesh with the existing teaching environment and curriculum. Third, such interventions are innovative and challenging and so require teachers and staff belief in, and commitment to, integration as a longer-term aim to improve students’ health and social and emotional learning. But such support appears to be promoted both because teachers value the scope that they provide for local adaptation and professional autonomy and because students value the chance to engage in learning methods that are more participative and topics that appear more relevant to students’ lives than might normally be the case in traditional academic subjects.

To our knowledge, there are no existing reviews of interventions that integrate health and academic education. However, reviews of related interventions can help in contextualising our findings. In their review of health promotion interventions in schools, Chilton et al. 169 similarly noted that school and teacher cultural norms concerning substance use affected the extent to which interventions addressing this were successfully implemented. Staff investment overall was critical, including support from administrators. Likewise, Pearson et al. 48 echoed the importance of engaging staff and suggested that ‘implementation hinges on negotiation and programme delivery and the acceptability (or otherwise) of the programme to those who deliver it’. They further commented on the importance of deliverers’ enthusiasm for the intervention and the need to root it in their perceived responsibility for its success. Bonell et al. ’s review64 of process evaluations of interventions aiming to increase the healthiness of school environments reported, based on five studies, on the importance of a health intervention’s alignment with school ethos as a predictor of its success, as well as the importance to good implementation of the broad participation of all staff and support from administrators. Rimm-Kaufman and Hulleman170 noted similar factors in a review of social and emotional learning interventions in primary schools, emphasising teachers’ enthusiasm for interventions as being pivoted on their overall culture of education on these subjects and of these skills. Coupled with school-wide support and ongoing mentoring from higher-level staff, a supporting ethos enhanced teachers’ commitment to interventions and was thus crucial to their success.

Indeed, our review provides evidence that teachers’ perceptions of their school’s teaching culture is a key determinant of successful implementation, something which did not emerge as a key theme in the other reviews cited above. This factor may reflect the importance of genuine integration between health and mainstream academic elements when delivering this particular category of intervention.

Outcomes of interventions integrating health and academic education

In our synthesis of outcome evaluations, quality of evidence was highly variable and often related to whether or not meta-analysis was possible. The strongest evidence for the effectiveness of interventions integrating academic and health education was found in the reduction of substance use in KS2 and KS3, and a meta-analysis of the effectiveness of these interventions in reducing violence victimisation in KS2 did not find an effect. Included estimates of academic attainment as a secondary outcome did not permit a conclusion as to the impact of these interventions on this domain.

Our findings extend and reflect the findings from prior related systematic reviews on substance use. Although a previous landmark review41 on alcohol use prevention in schools did not pool study estimates, authors nonetheless concluded that interventions had the potential to be effective. Our meta-analyses found specific evidence of effectiveness in KS3, but not KS4, for interventions integrating health and academic education. Although effects were small in magnitude, at a population level and with universal implementation the impact could be substantial. Similarly, a systematic review of school-based programmes to reduce smoking found evidence for interventions on preventing initiation of smoking. 42 Although our analysis combined all smoking outcomes, we were able to find evidence for effectiveness in reducing smoking in KS4. Again, the magnitude of effects was small but of potential public health significance at a population level. Finally, a major systematic review on illicit drug use prevention in schools found ‘small but consistent’ effects. 43 As was the case in our meta-analyses, the authors of this existing review noted that most included studies focused on marijuana use outcomes. Our findings reflect theirs in sum and substance, but we were able to locate effects more specifically as occurring in KS3 and KS4, with weaker evidence supporting effectiveness in KS2.

Because we were largely unable to undertake meta-analysis in respect of violence outcomes, our ability to compare with previous systematic reviews is limited. Our findings suggested a mixed pattern of results, and we were hampered by heterogeneous definitions of violence. In contrast, a systematic review of school-based programmes to reduce bullying and victimisation found significant reductions for each. 44 However, this review included non-randomised evidence and a diversity of intervention models.

Across outcome domains, few evaluations showed a consistent pattern of results. Both evaluations of Positive Action83,87,113–116,129,130 presented clear and compelling evidence of consistent effects in both substance use and violence perpetration. In substance use alone, the evaluation of LIFT98–101,125 suggested small but sustained effects in substance use initiation. RHC92,95 did demonstrate differences in frequency, but not prevalence, of substance use. In violence perpetration alone, evaluations presented mixed findings within evaluations. For example, findings from the first evaluation of Steps to Respect93,103,104 suggested decreases in observed, but not student-reported, bullying. The evaluation of Youth Matters109–111 had inconsistent findings by analysis method, that is, whereas the main analysis did not present evidence of an effect in bullying perpetration or victimisation at longest follow-up, an exploratory analysis that focused on classifying students into uninvolved students, bullies, victims or bully–victims presented some evidence of increased transition to uninvolved in the intervention group.

This emergent distinction between clearly effective and less clearly effective interventions suggests that examination of these differences is in order. That is, what sets apart Positive Action83,87,113–116,129,130 and LIFT98–101,125 (and, to a lesser degree, RHC)92,95 from other interventions? One possible answer is that trials of all three interventions had substantial developer involvement, which is associated with improved effectiveness in a variety of intervention domains. 171 An equally important, and equally plausible, answer is in the developmental period in which these interventions were initiated. All three interventions began in the equivalent of English and Welsh schools’ year 6 or earlier, and all three interventions incorporated ‘wraparound’ techniques beyond integration of academic and health education. For example, Positive Action emphasised a whole-school approach,55,83,85–87,102,113–116,118,129,130 LIFT emphasised home–school interactions98–101,125 and RHC92,95 included substantial programming beyond academic and health integration over several years. Moreover, what sets apart these interventions from the Gatehouse Project, which incorporated similar wraparound in the school setting, is the relatively earlier timing of intervention initiation. Positive Action in particular is a highly manualised intervention including lessons over multiple school years and concomitant efforts to change school climate. Echoing findings from our synthesis of process and theory studies, Positive Action was the intervention most likely of the evaluations included in our synthesis of outcome evaluations to involve erosion of boundaries between students and teachers.

First e-mail

(PDF download)

Follow-up email

(PDF download)

Suggestions from Beki Langford

De Vries H, Dijk F, Wetzels J, Mudde A, Kremers S, Ariza C, et al. The European Smoking Prevention Framework Approach (ESFA): effects after 24 and 30 months. Health Educ Res 2006;21:116–32.

De Vries H, Mudde A, Kremers S, Wetzels J, Uiters E, Ariza C, et al. The European Smoking Prevention Framework Approach (ESFA): short-term effects. Health Educ Res 2003;18:649–63.

De Vries H, Mudde A, Leijs I, Charlton A, Vartiainen E, Buijs G, et al. The European Smoking Prevention Framework Approach (EFSA): an example of integral prevention. Health Educ Res 2003;18:611–26.

Holm K, Kremers S, De Vries H. Why do Danish adolescents take up smoking? Eur J Public Health 2003;13:67–74.

Vartiainen E, Pennanen M, Haukkala A, Dijk F, Lehtovuori R, De Vries H. The effects of a three-year smoking prevention programme in secondary schools in Helsinki. Eur J Public Health 2007;17:249–56.

Suggestions from Brian Flay

Caring School Communities (URL: www.collaborativeclassroom.org/research-articles-and-papers).

PATHS (the module for grade 5 of which is believed to include study skills).

Positive Action (URL: www.positiveaction.net/) [see also bflay.net (URL: www.bflay.net)].

Second Step (related publications), which explicitly focuses on school connection and engagement with teachers.

Conduct Problems Prevention Group. Initial impact of the fast track prevention trial for conduct problems: I. The high-risk sample. J Consult Clin Psych 1999;67:631–47.

Conduct Problems Prevention Research Group. The effects of a multiyear universal social-emotional learning program: the role of student and school characteristics. J Consult Clin Psych 2010;78:156–168.

Crean HF, Johnson DB. Promoting Alternative THinking Strategies (PATHS) and elementary school aged children’s aggression: results from a cluster randomized trial. Am J Comm Psych 2013;52:56–72. 97

Curtis C, Norgate R. An evaluation of the Promoting Alternative Thinking Strategies curriculum at key stage 1. Educ Psych Pract 2007;23:33–44.

Greenberg MT, Kusche CA. Preventive intervention for school-aged deaf children: the PATHS Curriculum. J Deaf Stud Deaf Educ 1998;3:49–63.

Greenberg MT, Kusche CA, Cook ET, Quamma JP. Promoting emotional competence in school-aged children: the effects of the PATHS curriculum. Dev Psychopath 1995;7:117–36.

Kam C, Greenberg MT, Kusché CA. Sustained effects of the PATHS curriculum on the social and psychological adjustment of children in special education. J Emot Behav Disord 2004;12:66–78.

Kam C, Greenberg MT, Walls CT. Examining the role of implementation quality in school-based prevention using PATHS curriculum. Prevention Science 2003;4:55–63.

Little M, Berry V, Morpeth L, Blower S, Axford N, Taylor R, Bywater T, Lehtonen M. Tobin K. The impact of three evidence-based programmes delivered in public systems in Birmingham, UK. Int J Confl Violence 2012;6:260–72.

Malti T, Ribeaud D, Eisner MP. The effectiveness of two universal preventive interventions in reducing children’s externalising behavior: a cluster randomized controlled trial. J Clin Child Adolesc Psychol 2011;40:677–92.

Malti T, Ribeaud D, Eisner M. Effectiveness of a universal school-based social competence program: the role of child characteristics and economic factors. Int J Confl Violence 2012;6:249–59.

Riggs NR, Greenberg MT, Kusché CA, Pentz MA. The mediational role of neurocognition in the behavioral outcomes of a social-emotional prevention program in elementary school students: effects of the PATHS curriculum. Prevention Science 2006;7:91–102.

Schonfeld DJ, Adams RE, Fredstrom BK, Weissberg RP, Gilman R, Voyce C, et al. Cluster-randomized trial demonstrating impact on academic achievement of elementary social-emotional learning. School Psychology Quarterly 2014.

Seifert R, Gouley K, Miller AL, Zabriski A. Implementation of the PATHS curriculum in an urban elementary school. Early Educa Dev 2004;15:471–86.

Suggestions from Tom Roderick

Brown JL, Jones SM, LaRusso MD, Aber JL. Improving classroom quality: teacher influences and experimental impacts of the 4Rs program. J Educ Psychol 2010;102:153–67. 54

Suggestions from Mark Greenberg

Mark Greenberg shared the grade 5 curriculum of the PATHS programme and suggested that this year in particular integrates with literacy development. Possible publications including study of this grade include the following.

Crean HF, Johnson DB. Promoting Alternative Thinking Strategies (PATHS) and elementary school aged children’s aggression: results from a cluster randomized trial. Am J Commun Psychol 2013;52:56–72.

Kam CM, Greenberg MT, Walls CT. Examining the role of implementation quality in school-based prevention using the PATHS curriculum. Prev Sci 2003;4:55–63.

Kam C, Greenberg MT, Kusché CA. Sustained Effects of the PATHS^® Curriculum on the Social and Psychological Adjustment of Children in Special Education. J Emot Behav Disord 2004;12:66–78.

Seifer R, Gouley K, Miller AL, Zakriski A. Implementation of the PATHS curriculum in an urban elementary school. Early Educ Dev 2004;15:471–85.

Quality assessment tool

Consider if the study has reported the following. Provide justifications for your response wherever possible.

1. Clarity

   • Are constructs clearly defined?

   • Is there a clear description of the intended pathways from intervention inputs to behaviour change?

2. Plausibility/feasibility

   • Is the theory logical (e.g. do the pathways from intervention inputs to behaviour change seem plausible)?

   • Is there a description of existing empirical evidence supporting the theory?

3. Testability

   • Do the authors refer in any way to empirical testing of the theory?

4. Ownership

   • Was the theory developed with or by practitioner stakeholders (e.g. teachers)?

   • Was the theory developed with or by community stakeholders (e.g. students)?

5. Generalisability

   • Is the theory presented as one that is applicable to different contexts?

   • Does the theory describe how the mechanisms might play out differently in different contexts?

   • Do authors present empirical evidence of the generalisability of the theory?

Intervention

1. Intervention location

   • Details [country; area(s) of country]

   • Not stated

2. Description of intervention [the nature of the intervention(s)/components and how it was delivered to intervention group(s)]

   • Details (overall aim of the intervention: what knowledge and skills it aims to build, key components of the intervention, who delivered the intervention; deviations from original protocol, etc.)

   • Not stated

3. Description of the integration of academic and health curricula

   • Details (description of integration; rationale for integration)

   • Not stated

4. Frequency and duration of the intervention

   • Details

   • Not stated

5. Target population

   • Details (ages/grades; any mention of a universal vs. targeted intervention component; specific characteristics, i.e. from a low SES neighbourhood; how they were recruited)

   • Not stated

6. Provider (teachers, counsellors, nurses, external facilitators, etc.)

   • Details (qualifications, level of experience, how they were recruited)

   • Not stated

7. Provider organisation description

   • Details (area of focus, years of experience doing similar work, aims of organisation, etc.)

   • Not stated

8. How intervention was developed

   • Details [background; founder(s); are there guiding theories and, if so, how are these explained in terms of theory of change data extraction, etc.]

   • Not stated.

Evaluation

1. Research questions or hypotheses

   • Details

   • Not stated

2. Evaluation examined (tick all which apply): feasibility/fidelity/quality/intensity, coverage/reach/accessibility, acceptability/satisfaction, mechanism, and/or context

3. Overall design within which evaluation is part: RCT, matched comparison, uncontrolled before/after, cross-sectional, other (please state)

4. Timing of evaluation

   • Details

   • Not stated

5. Sampling methods

   • Details

   • Not stated

6. Sample size, response rates at baseline

   • Details

   • Not stated

7. Sample size, response rates at follow-up

   • Not stated

   • Details

   • Not appropriate

8. Sociodemographic characteristics of participants at baseline/follow-up

   • Details

   • Not stated

9. Methods of data collection

   • Details (period of data collection, types of data, comments on quality of data, etc.)

   • Not stated

10. Methods of data analysis

    • Details

    • Not stated.

Intervention details

1. Study location (country, area of country)

   • Details

   • Not stated

2. Intervention start date

   • Details

   • Not stated

3. Intervention duration

   • Details

   • Not stated

4. Target population (indicate whether this is a student, classroom, school, teacher or other population)

   • Details

   • Not stated

5. Description of intervention activities and how they were delivered to intervention group(s)

   • Details

   • Not stated

6. Description of components delivered to control group(s)

   • Details

   • Not stated

7. Description of integration

   • Details

   • Not stated

8. How intervention was developed

   • Details

   • Not stated

9. Provider organisation description

   • Details

   • Not stated.

Evaluation details

1. Research question(s) or hypotheses

   • Details

   • Not stated

2. Timing of outcome evaluation

   • Details

   • Not stated

3. Unit of allocation

   • School

   • Classroom

   • Student

   • Other (please indicate)

   • Not stated

4. Generation of allocation sequence (e.g. What was the random component of the sequence generation process – this may include minimisation?)

   • Details

   • Not stated

5. Concealment of allocation

   • Details

   • Not stated

6. Blinding

   • Details

   • Not stated

7. Sampling strategy

   • Details

   • Not stated

8. Sample size, overall response rates at baseline

   • Details (intervention group)

   • Details (control group)

   • Not stated

9. Sample size, overall response rates at follow-up

   • Details (intervention group)

   • Details (control group)

   • Not stated

   • Not appropriate

10. Sociodemographic characteristics at baseline/follow-up

    • Details

    • Not stated

11. Were baseline equivalence/differences between arms reported?

    • Details

    • Not stated

12. How were differences between intervention and comparison groups controlled?

    • Details

    • Not stated

13. For each outcome measure, report the following

    1. Description of outcome measure provided

       • Details

       • Not stated

    2. Outcomes stated (in protocol, from outset)

       • Primary outcomes

       • Secondary outcomes

       • Not stated

    3. Post hoc outcomes reported

       • Description

       • Not stated

    4. Evidence of reliability

       • Description

       • Not stated

    5. Data collection methods

       • Description

       • Not stated

    6. Baseline response rate

       • Details (intervention group)

       • Details (control group)

       • Not stated

    7. Follow-up response rate

       • Details (intervention group)

       • Details (control group)

       • Not stated

    8. Effect sizes

       • Overall

       • By sex

       • By age

       • By SES

       • By ethnic subgroup

       • No effect size

    9. Study analysis was intention to treat?

       • Yes

       • No

       • Not stated

    10. Study analysis appropriately accounted for clustering?

        • Yes

        • No

        • Not stated

    11. Study analysis adjusted for confounders?

        • Yes

        • No

        • Not stated.