Notes
Article history
The research reported in this issue of the journal was funded by the PHR programme as project number 10/3006/13. The contractual start date was in August 2012. The final report began editorial review in June 2021 and was accepted for publication in April 2022. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ manuscript and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.
Permissions
Copyright statement
Copyright © 2024 Blair et al. This work was produced by Blair et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.
2024 Blair et al.
Chapter 1 Introduction
Background
In 2007, the UK was ranked bottom of 21 nations in a United Nations Children’s Fund (UNICEF) report on child and adolescent health and well-being in high-income countries,1 stimulating the development of national initiatives to improve children’s and young people’s health. A subsequent UNICEF report in 20132 revealed an improvement since this first study, with the UK ranked 16th out of 26 nations. Although there have been some improvements in child health and well-being, the UK has the lowest rate of young people going into further education across all 26 countries and some of the highest child alcohol abuse and teenage pregnancy rates3 (although recent evidence suggests teenage pregnancy rates in the UK have halved since 20074). The current intervention was developed in this context, but remains relevant today.
Social and emotional well-being (SEW) comprises (1) emotional well-being: being happy, confident and not anxious or depressed; (2) psychological well-being: managing emotions, autonomous problem-solving, experiencing empathy and resilience; and (3) social well-being: forming good relationships with others and avoiding behavioural problems, including violence or bullying. 5 It enables people to function well and meet the challenges of life.
The importance of primary-age children’s SEW has been recognised by the UK and Scottish governments and is reflected in a strong body of current and recent government policy and initiatives. For England and Wales, this includes ‘Every Child Matters’, ‘Social and Emotional Aspects of Learning’ and ‘Your Child, Your Schools, Our Future: Building a 21st Century Schools System’. 6–12 Scottish initiatives include ‘Early Years and Early Intervention’, ‘Equally Well’, ‘Getting It Right For Every Child’ and the ‘Curriculum for Excellence’s Health and Well-being Outcomes’. 13–16 The Curriculum for Excellence (CfE) is the Scottish Government’s framework guidance for education, which aims to achieve a transformation in education in Scotland by providing a more coherent, more flexible and enriched curriculum from ages 3 to 18 years. 17
Many of these policies are informed by the World Health Organization’s health-promoting schools framework,18 which requires schools to address, simultaneously, the domains of school ethos, curriculum and family/community involvement. In Scotland, SEW aspects of learning are embedded within the CfE, with health and well-being as important as literacy and numeracy. There are also a number of localised curriculum programmes that have been used successfully within primary schools in Scotland, including Creating Confident Kids (a programme based on the English and Welsh Social and Emotional Aspects of Learning programme), The Motivated School and Being Cool in School. These programmes lack robust evaluation but are currently supported by the Scottish Government’s Rights, Support and Wellbeing Team (previously the Positive Behaviour Team), which adopts both universal and targeted approaches to positive behaviour through improving relationships and environments in schools.
Scientific background
Traditionally, school-based public health interventions aimed at improving the health and well-being of young people have focused on the prevention or reduction of specific health conditions or behaviours, such as obesity; inactivity; alcohol, tobacco and illicit drug use; and sexual risk taking. However, systematic reviews have shown that these interventions have had very mixed and often limited effects on outcomes, with few interventions proving to have a strong long-term impact. 19–25 In addition, there is growing evidence that universal interventions addressing the underlying determinants of non-communicable health conditions and risk behaviours might have greater impact and be more cost-effective at improving health and well-being than approaches targeted at the most vulnerable or those already experiencing problems. 26 In this chapter, we provide a scientific rationale for this study, which evaluated an intervention for primary school children that aims to improve their Social and Emotional Education and Development (SEED). We also describe the aim and research questions for the study. The key components of the SEED intervention and the underpinning theory of change are explained in more detail in Chapter 2. For an overview of the literature search strategy, see Appendix 1.
The primary school setting
There is evidence that programmes addressing the underlying determinants of non-communicable health conditions and risk behaviours need to be introduced in the early years of primary school and sustained over time to include key transition periods. 7,10,27 Improved SEW during primary school has been shown to have an impact on physical health and to be a protective factor against a range of risk behaviours in later years, including tobacco use, illicit drug use and alcohol misuse; violence and crime; and teenage pregnancy. 7,10,28–32 A 2011 review and meta-analysis including > 200 controlled studies of school-based interventions designed to enhance the social and emotional skills of children aged 5–18 years27 found positive benefits on a range of outcomes, including significant improvements in social and emotional skills, attitudes and positive social behaviours, and academic performance. Maximum benefits were observed when the programmes were evidence based and well implemented by school staff. In addition, there is strong evidence that interventions with a longer duration (i.e. multiyear interventions) are likely to have a greater effect than shorter-term programmes. 7,10,33
Other systematic reviews of the effectiveness of universal interventions to improve the mental well-being (encompassing emotional, psychological and social well-being) of children in primary education found that curriculum-only interventions appear to be effective only in the short term. In contrast, there is good evidence to support the use of programmes that combine a social and emotional development curriculum with components that focus on behaviour management and improvement of child–teacher relationships. 7,10,28–32,34 Furthermore, programmes that include a significant teacher training component show considerable promise, as do those that include a parenting support component. 7,10,28–32,34 The multicomponent intervention approach appears to be particularly effective in improving mental health, as well as reducing bullying and violence. 7,10,28–32 One of the key studies in this area is the Seattle Social Development Project (now known as Raising Healthy Children), which was implemented in year 1 of elementary school for 6 years, with follow-up into adolescence and young adulthood. The programme sought to promote connectedness to school and family and to strengthen children’s social competencies. It consisted of three components: teacher training, child social and emotional skill development, and parent training. Follow-up at age 21 years revealed significant reductions in the intervention group, compared with the control group, in health risk behaviours (including alcohol, tobacco and illicit drug use and sexual risk behaviour), violence and crime, and emotional and mental health issues, and increases in positive functioning in university or work. 7,10,35–38 Some of these effects remained significant when the study population was followed up at age 30 years. 39
When devising the SEED intervention, there was little robust evaluation of interventions aimed at improving SEW in UK primary schools. One exception is a recent cluster randomised controlled trial (RCT) of the Roots of Empathy (ROE) intervention, carried out in 74 primary schools in Northern Ireland. 40 ROE is a Canadian programme designed to promote empathy in children through demonstration of the attachment relationship. 41 It consists of annual visits from a mother and infant to the school over the course of a year and is designed to facilitate the labelling of feelings and the exploration of the relationship between feelings and behaviour. The investigators found that, among the intervention group, children (year 5, aged 8–9 years) were rated by their teacher as being more prosocial and exhibiting less difficult behaviour post intervention. However, only the latter was sustained over the 36-month follow-up period. No difference in effect by gender or socioeconomic status was observed. An economic analysis indicated that the ROE intervention was cost-effective. These findings are in accordance with those of the only other cluster RCT of ROE,42 which reported improvements in prosocial behaviour (although this was not maintained over time) and reduction in aggressive behaviour in the short and longer terms.
Whole-school intervention approaches
Whole-school interventions aim to adjust the school environment in an effort to improve health and have been shown to improve social competence and reduce aggression and health risk behaviours. 43–50 A comprehensive review of the impact of school environment interventions in both the primary and secondary school settings on health and well-being51 identified 10 experimental or quasi-experimental studies with a variety of outcomes. Interventions fell into one of three categories: promotion of sense of community and better interpersonal relationships to reduce aggression and other risk behaviours; encouragement of advocacy by staff and pupils for healthier eating and physical activities; and improvement of school playgrounds. Results were mixed, with many studies having important methodological limitations. The authors51 concluded that school environment interventions have the potential to reduce violence and aggression, but highlighted the need for more robust intervention studies. A parallel Cochrane review synthesised the evidence from cluster RCTs of multicomponent interventions aiming to improve health in schools that included input to the curriculum, changes to the school ethos, and family or community engagement. 52 Although the authors found some evidence for effectiveness, such as reduced bullying and improvement in some health behaviours, including smoking, physical activity and dietary outcomes, there was little evidence of effect on other outcomes, such as alcohol use and mental health. Few studies reported on these last two outcomes, and the methodological quality of included studies was generally considered to be low to moderate.
Some of the best evidence for the effectiveness of the whole-school approach within the secondary school setting comes from the Australian Gatehouse Project, which was designed to improve SEW through the promotion of a sense of social inclusion and connection in secondary schools. 53–56 Importantly, the strategies used to achieve this varied between schools, according to pupils’ perceptions of need, with the conceptual framework focusing on three ways of strengthening connectedness to school: (1) building a sense of security and trust, (2) enhancing communication and social connectedness and (3) building a sense of positive regard through valued participation in aspects of school life. 57,58 The effectiveness of the Gatehouse Project was evaluated using a RCT design. 57 Although the Gatehouse Project did not affect depressive symptoms or social and school relationships,53–56 it reduced substance use for a cohort of pupils in the intervention schools followed up longitudinally 2 and 3 years after the intervention began. 53–56 Subsequent year-8 pupils (aged 13–14 years) in intervention schools, surveyed 5 years after the trial began, also reported lower rates of substance use than those surveyed in the control schools. 57,58 Both the Seattle Social Development Project8,11,36–39 and the Gatehouse Project included changes to the school curriculum that focused on developing or improving SEW.
There is also some evidence that targeted programmes can have an impact on SEW. A review of targeted programmes in primary schools reported modest improvement, particularly in social problem-solving and development of positive peer relations, with lengthy, multicomponent programmes. 59 The evidence therefore supports an integrated approach that provides targeted support for those experiencing particular difficulties within a supportive whole-school approach to promote mental well-being. 26,43 Furthermore, a review of family-based programmes to promote mental well-being10,60 supports the need for generic programmes to promote mental well-being at a population level and more intensive programmes for more serious problems among individuals. National Institute for Health and Care Excellence (NICE) guidance on promoting children’s SEW in primary education strongly supports the adoption of universal approaches, which have the capacity to address emotional well-being in a connected way that reduces potential for stigmatisation, but which also include provision of targeted approaches and early identification of children at risk. 61
Scientific rationale
Although the evidence base suggests several promising intervention approaches, there are limitations and gaps that need to be addressed. Most of the evidence for improving SEW among primary school children is non-UK and largely US-based, and many of the programmes evaluated (particularly family or parenting programmes) target vulnerable children only, as opposed to taking a universal approach. The effectiveness of universal approaches that incorporate elements such as parent support has rarely been investigated in the UK and needs to be determined through robust evaluation. In addition, the cost-effectiveness of addressing health and well-being in general, and SEW in the primary school years in particular, has rarely been determined. However, economic evaluation of the Seattle Social Development Project revealed the substantial economic benefits of such an approach in the US setting, which were far greater than those of basic curriculum-based programmes implemented during early adolescence. 62 To date, evaluation studies have rarely examined the differential effects of specific programmes according to gender and socioeconomic status. It is essential to evaluate the effectiveness of programmes on these subgroups to ensure that there is no widening of health inequalities between these groups, and to identify whether or not programmes may be effective in narrowing health inequalities. Finally, many studies have been criticised for methodological limitations, including having short-term follow-up only, participant attrition and not including key transition periods within the duration of follow-up. 47–52 The transition period between primary and secondary has been shown to be an important indicator for well-being and attainment in later life;7,10,63–65 therefore, it is important to include this key period within the follow-up period of any primary school-based intervention.
The SEED intervention was developed for Scottish primary school children based on the evidence already outlined. For the development, components and theoretical underpinning of the SEED intervention, see Chapter 2.
Study aim and research questions
The overarching aim of this study was to rigorously evaluate the impact of the SEED intervention on improving pupils’ SEW via a stratified cluster RCT. The main research questions were addressed by the complementary outcome, process and economic evaluations.
The pupil-related research questions were as follows:
-
Does the SEED intervention improve pupils’ SEW?
-
If so, is the impact different for specific subgroups of pupils (gender, deprivation)?
-
-
Is the SEED intervention more effective if started with younger children [SEED younger cohort (YC) vs. older cohort (OC)]?
-
What is the duration of the SEED intervention effect?
-
Does the SEED intervention improve the social and emotional experience of transition from primary to secondary school?
-
What is the impact on health behaviours of the SEED intervention during early secondary school years?
-
What are pupils’ experiences of the SEED intervention?
The teacher-related research questions were as follows:
-
Are there changes in teachers’ knowledge, attitudes and behaviour relating to developing pupils’ SEW?
-
Were teachers involved, and, if so, how were they involved, in selecting initiatives to respond to the pupils’ needs assessment?
-
What contextual factors facilitate or inhibit the delivery of the SEED intervention?
-
What contextual factors support or hinder the ability of the SEED intervention to improve pupils’ SEW?
-
Which teachers engage best with the SEED intervention?
-
What are teachers’ experiences of the SEED intervention?
The parent-related research questions were as follows:
-
Do parents report a difference in their child(ren)’s emotional and social development?
-
If applicable, what are parents’ experiences of the SEED intervention?
The economic research question was as follows: is SEED cost-effective?
Chapter 2 The Social and Emotional Education and Development intervention
The development of the Social and Emotional Education and Development intervention
The SEED intervention is designed to promote SEW among primary school pupils. The primary aim of the SEED intervention is to change the school environment (policy-making, teacher and pupils’ relationships and actions) to bring about improvements in SEW. It does so by supporting staff to examine data from their school community, identify the SEW needs of pupils and develop plans to address those needs. 66
The development of the SEED intervention was funded by the Medical Research Council (MRC)/Chief Scientist Office (CSO) Scottish Collaboration for Public Health Research and Policy and was based on a systematic search of the published literature, grey literature and websites to identify existing interventions that aimed to improve the SEW of schoolchildren. We also consulted educational psychologists (EPs), education researchers and other education professionals in Scotland, the USA and Australia to identify promising interventions.
The general model of the SEED intervention is encapsulated by the communities of care, in which the use of intelligence (research or other information) to identify local needs, co-production of priorities and actions to address those needs, and the evaluation of the impact are of fundamental importance. 67 One community of care in particular, known as the Gatehouse Project, was instrumental in helping us design the SEED intervention. Developed in Australia and co-led by the SEED RCT co-investigator and co-author Lyndal Bond, it aimed to build capacity in secondary schools to improve the emotional and mental health needs of young people and was successful in reducing cigarette and drug use, antisocial behaviour and sexual risk behaviour. 53–58
The Gatehouse Project aimed at improving emotional well-being by promoting pupils’ connectedness and belonging to their school. It did so through identification of the relevant risks and protective factors, co-production of an action plan (AP) with teachers and identifying evidence-based interventions to improve relationships between teachers and pupils and pupils’ life skills, all within an evaluative action research cycle. The essential elements were (1) a conceptual and operational framework that made sense to teachers and health researchers, (2) the active combination of health promotion with education reform, (3) an educator working closely with schools as a facilitator or critical friend, (4) establishment of an implementation team, (5) the use of local data to inform direction and strategies and prioritise actions and (6) integration of the curriculum component with the classroom and whole-school context. In so doing, the Gatehouse Project introduced standardised functional components while providing flexibility to accommodate and adapt to local circumstances. 53–58
An early model of the SEED intervention was evaluated in four primary schools in Glasgow. 68 The staff in the pilot schools welcomed the SEED intervention, mainly because it met the existing requirements to enhance health and well-being as part of the Scottish Government’s CfE. 17 EPs also welcomed the intervention. Head teachers (HTs) wanted more flexibility in managing the feedback loops and adapting the programme to the requirements of their schools.
Preliminary discussions and the formative evaluation resolved several aspects of delivery of the SEED intervention. It was decided to assess schools’ organisational needs by means of a survey of all school staff, including classroom assistants and support staff. Pupils’ needs assessments were based on the key outcome measure for the trial [i.e. the Strengths and Difficulties Questionnaire (SDQ)]32,69–71 in primary school year 1 (P1) (ages 4–6 years) and primary school year 5 (P5) (ages 8–10 years) classes, with the option to include year-2 and year-6 pupils for very small schools. The SDQ data would be reported with minimal interpretation, without prescribing specific new programmes. Instead the emphasis would be on self-reflection, solution-focused discussion and appreciative inquiry. 72 EPs linked to the school would play a central role in facilitating the SEED intervention, given their role in supporting pupils and staff. Quality improvement officers (QIOs) would be encouraged to support the implementation of agreed initiatives.
Key components of the Social and Emotional Education and Development intervention
The SEED intervention comprises five components (Figure 1).
Component 1: data collection – pupils’ social and emotional well-being and school culture
Pupils’ SEW was assessed using the SDQ. Data were gathered as part of the trial at baseline [i.e. time 0 (T0)] and at follow-ups 1 and 2 [i.e. time point 1 (T1) and time point 2 (T2), 2 years and 3 years post T0, respectively], which allowed three intervention cycles over the course of the study. Additional questions captured data on pupils’ friendships with peers, relationships with staff and expectations of transition to secondary school. Staff perspectives of school culture were assessed through a survey that included the HTs, teaching staff, classroom assistants and support staff. The survey questionnaire included questions on staff stress and distress, whole-school relationships, perceptions of pupil behaviour and well-being, and school ethos. Parents’/carers’ perceptions of the school were captured in a parent survey that included questions on school ethos and environment. For further details of the questionnaires and survey methods, see Chapter 3.
Component 2: feedback reports – analysis and feedback to schools
The data from component 1 were analysed and compiled into individualised reports for each school. To facilitate comparison and reflection, the data were aggregated by cohort (i.e. P1, P5, staff and parents). For T1 and T2, the reports depicted the changes in scores from T0 to T1 and T2. In addition to the individual school reports, an ‘all schools’ report was produced depicting the same data collated across all intervention schools involved in the SEED RCT. Each school was positioned relative to the others in either graphic or tabular form to facilitate comparison (two examples relating to T0 and T2 are available in Report Supplementary Material 1 and 2, respectively).
A SEED resource guide (see Report Supplementary Material 3) was also provided to staff that contained an up-to-date and comprehensive review of programmes to help pupils improve their social and emotional education and well-being. All known programmes delivered to pupils, staff or parents in Scotland that were evidence based or had a strong theoretical underpinning were included. The content of the resource guide was identified through our systematic search and a review of relevant websites, and further information was sought from programme providers if possible. The resource guide was greatly assisted by two existing reviews: the American website Collaborative for Academic, Social and Emotional Learning (CASEL)33,73 and the Australian website KidsMatter. 74 All resources included in the guide were rated in terms of the strength of evidence of effectiveness and were ideally endorsed by the Scottish Government as aligning with the central ethos of the CfE. 75 Summary information was provided for each programme, including the target population, the aim, key components, effectiveness and links to further information. The guide was updated after 2 years.
Component 3: presentation and reflective discussion
Staff from each school chosen to implement the SEED intervention were invited to a presentation and reflective discussion (RD) session in school. The data outlined in component 1 were presented, followed by a discussion, aimed at providing a safe space for open discourse and identifying and prioritising key actions. EPs were included because of their pivotal role in providing additional support to pupils and staff. The teaching staff were encouraged to co-present the data (with the research team); when possible, the EPs co-led the RDs with teaching staff.
The RD sessions aimed at developing a commitment to positive change, co-producing tailored school-wide initiatives and establishing broad support for those initiatives. An assets-based approach was adopted whereby feedback was provided using positive language with reference to best hopes and preferred futures. When considering how to respond to feedback, staff were encouraged to be aware of their goals, considering their preferred options and existing teaching commitments. Reflection was encouraged as part of a continual process of change and development and not an annual event, nor an end in itself.
It was recognised that reflective sessions could be considered sensitive; therefore, the senior management team of each school was given the option of having a separate presentation and RDs. It was impractical to invite parents to the feedback sessions, but schools were encouraged to share results of the needs assessment with parents in other ways, for example through newsletters or electronic communications.
Component 4: selection of actions
There were several co-produced ground rules when selecting actions. Agreed actions should be practical and not add excessively to existing workloads, and preferred options should be aligned with the ethos and strategic goals of the school. The RDs should lead to the formulation of an AP (see Appendix 5), either by all staff or through an ‘action team’ subgroup, which was sometimes the senior management team. Staff were encouraged to use the SEED resource guide to help in their selection of appropriate actions and activities with short-, medium- and long-term time frames. It was recommended that the key priorities, and how these would be implemented, be set out in the APs and focus on:
-
the classroom, for example use of existing interventions such as Creating Confident Kids and Cool in School
-
whole-school initiatives, including training for staff and parents, supporting the implementation of restorative practices aimed at promoting proactive classroom management and interactional instruction, the understanding of the importance of the SEW of children and the opportunity of being positive role models. 18
When possible, the APs were incorporated into the school improvement plan, which schools are obliged to submit to their local authority (LA), typically in a 1- to 3-year cycle.
Component 5: implementation
After the development of APs, schools implemented their activities, supported by their EP and, when appropriate, QIO. The SEED RCT researchers maintained contact with schools during this period, but the intention was that schools should use their usual support systems to implement and maintain initiatives to replicate how the intervention would work in a real-life setting.
Meetings were held with EPs from LAs to discuss the delivery of the intervention and the role of the EP in that delivery. The dominant role of the EP was that of facilitation based on appreciative enquiry, principles that would be familiar to EPs and therefore require minimal training. EPs working in Scottish schools are commonly accredited as chartered psychologists by the British Psychological Society and have membership of the Scottish Division of Educational Psychology. QIOs were also invited to these meetings, although they were less involved. Qualifications for QIOs are unclear, but this role would be likely to require a formal teaching qualification and extensive experience in LA educational settings, including leadership roles and meeting national education priorities.
Underpinning theory of change
The SEED intervention draws on several theories, outlined in Figure 2.
Socioecological framework
The socioecological framework76–78 distinguishes different levels of social influence that operate simultaneously and interact with each other. The levels are commonly described as intrapersonal (e.g. genetics, personality), individual (biographical factors, attitudes and beliefs), interpersonal, community, organisational and macro. The SEED intervention operates at four of these levels:
-
The individual, for example changing pupils’ and staff cognition and attitudes. These include raising staff members’ awareness of SEW as a result of participating in the presentations and RDs, and improving pupils’ core competencies such as self-awareness and social awareness through tailoring the classroom curricula.
-
The interpersonal, such as improving relationships between staff and pupils through greater focus on SEW and between staff by engaging in RD.
-
Organisational, for example changing the culture of a school through a collective commitment to address SEW and develop trust between staff and pupils.
-
The SEED intervention also interacts with macro-level influences, in particular the policy environment as set by the Scottish Government’s CfE17 and education authorities’ translation of that policy at local level.
Co-production
The term ‘co-production’ describes an equitable approach to intervention innovation, development and evaluation. The fundamental principle is that users design and deliver services in equal partnership with professionals. 67 Greenhalgh et al. 79 set out three key principles required for effective co-production in community-based research:
-
A systems perspective comprising an iterative and flexible design that adapts an intervention to local need. This was of central importance to the SEED intervention, including the choice of activities through RD.
-
The framing of research as a creative enterprise with human experience at its core. Research based on pupils’ SEW was the starting point of each of the intervention cycles in the SEED intervention.
-
An emphasis on process, including the framing of the programme, the nature of relationships, and governance and facilitation arrangements, especially the style of leadership. HTs and EPs were identified as local leaders and facilitated the SEED intervention through appreciative enquiry.
Social learning theory
Social learning theory posits that new behaviours can be learnt, and existing behaviours changed, through modelling behaviour, either by demonstration or being taught. 80,81 The concept of ‘modelling’ involves the development of self-efficacy, intentions and planning, and modification through social approval. 80,81 People observe credible role models, with whom they can identify, engaging in particular behaviours. They see the benefits of these behaviours and are motivated to adopt similar behaviours. This learnt behaviour is then positively reinforced by significant others. For example, it was important that staff took ownership of the SEED intervention, and, as part of their collective commitment, they modelled SEW by demonstrating their emotional competency to each other and their pupils.
School connectedness
Connectedness refers to diverse aspects of people’s social experiences, ranging across systemic dyadic relationships, perceptions of relationships, satisfaction with institutions and feelings of belonging. It is associated with a host of positive health outcomes for children and young people. 82 The concept of connectedness is very similar to Markham and Aveyard’s,83 Bonell et al. ’s,47,48,51 Jamal et al. ’s49 and Naghieh et al. ’s50 notion of pupils being ‘committed’ to their school, in that they are able to meet the challenges of learning and accept the school’s norms of behaviour and ethos.
It was decided that dedicated time for staff to better understand the school’s SEW needs, to reflect on these and to discuss how to respond would have a favourable impact. First, the SEED intervention was designed to enhance staff awareness and understanding of both pupils’ and staff members’ SEW problems. Second, the discussion of SEW was expected to lead staff to express and share problems, which would, in turn, improve collegiate relationships. These two processes would enhance motivation and collective commitment for positive change to tackle and prevent SEW problems.
Delivering classroom and environmental SEW interventions should lead both staff and pupils to become more aware of, and practise, their SEW skills and lead staff to reward pupils who exercise these skills. If successful, this should develop pupils’ core competencies such as self-awareness, self-management, social awareness, relationship skills and responsible decision-making, thus contributing to the ‘instructional order’. 83 Another pathway to these outcomes is through the ‘regulatory order’,83 whereby rewarding SEW skills contributes to pupils’ connectedness and commitment to their school. 47–51,82 Theoretically, improvement in school connectedness should lead to reductions in disruptive behaviour, poor well-being and absenteeism among pupils. 84
The next chapter will present the methods for the outcome, process and economic evaluations.
Chapter 3 Methods
Trial design
The SEED intervention was a stratified cluster RCT of a multicomponent primary school intervention in Scotland. Process and economic evaluations were conducted as part of the trial. Randomisation was carried out at the level of secondary school learning communities (clusters comprising the primary schools that feed into each secondary school) to minimise contamination; all participating primary schools within a learning community were allocated to the same arm of the trial. The allocation ratio was 1: 1. Once randomised, the intervention schools received the SEED intervention (see Chapter 2) and the control schools continued with their SEW activities as normal. 66
Trial setting
The trial recruited 38 primary schools from 18 learning communities across three Scottish LAs, anonymised as LA1, LA2 and LA3 in this report. There were four waves of data collection in the main study [baseline (T0) and three follow-ups: T1, T2 and time point 3 (T3)] and a further fifth wave [time point 4 (T4)] following the prespecified primary outcome at T3. The baseline (T0) and the first follow-up (T1) data collection and intervention delivery took place in primary schools. Data collection for the OC of pupils at the second (T2), third (T3) and fourth (T4) follow-ups took place in secondary schools.
Changes to trial design
The SEED intervention was registered with an International Standard Randomised Controlled Trial Number (ISRCTN) in April 2013 (see https://fundingawards.nihr.ac.uk/award/10/3006/13 and https://doi.org/10.1186/ISRCTN51707384). No changes were made to the trial design, aims or primary and secondary outcomes after registration. However, we did secure additional funding, also from the National Institute for Health and Care Research (NIHR), to conduct longer-term follow-up (T4) with the OC just before they reached the end of compulsory education (aged 16 years).
Participants
Eligibility criteria for local authorities
Any LA within a reasonable geographical location accessible to the main study site was eligible for participation. Any LA with involvement in the SEED pilot was excluded, as were LAs that required opt-in consent for parents. LAs known to be taking part in studies similar to the SEED RCT were also excluded. Three eligible Scottish LAs were selected based on the spread of urban and rural populations, socioeconomic diversity and reasonable proximity to the research unit base.
Eligibility criteria for schools
Within each of the three selected LAs, all state primary schools and associated secondary schools were eligible. Special schools (offering specialist education independently of mainstream school base), independent schools and home-educated children were excluded (accounting for ≈ 4.3% of primary school children in Scotland85–88).
Eligibility criteria for participants
Eligible staff included all teaching and non-teaching staff within study primary schools. At T0 all primary school pupils in P1 (aged 4–6 years; YC) or primary school year 5 (P5) (aged 8–10 years; OC) in the 2012–13 academic year were eligible to participate. Baseline (T0) measures were collected in academic year 2012/13, and follow-up 1 (T1) measures were collected in academic year 2014/15. Because teachers or classroom support assistants completed the SDQ on behalf of their pupils, individual pupils were not excluded on the basis of additional support needs or language difficulties unless it was decided, in liaison with the class teacher and HT, that the SDQ would not be a valid measure for a particular child. Pupils completing the self-completed questionnaire were given additional support when needed, unless it was decided, in liaison with the class teacher and HT, that the pupil’s level of understanding of the questions would not allow them to complete the questionnaire accurately (e.g. not understanding spoken English or severe learning difficulties) or when participation would be detrimental to the young person’s well-being.
Sample size
Based on an average of 38 pupils per year at each school and the initial estimate of 36 schools participating, we estimated that there would be a potential 2736 pupils for recruitment at baseline. We expected few pupils to opt out of participation and assumed that 75% of the target population could be followed up for 4 years, meaning a total of 2052 pupils, 1026 from each age group, with an average 28.5 pupils per year per school.
A cautious intraclass correlation coefficient (ICC) of 0.05 was assumed (i.e. 5% of the variation in the primary outcome will be attributable to school- and class-level variability), giving a design effect of 2.37. Therefore, with 513 pupils followed up in both the intervention and control arms of the trial (allowing for attrition), the effective sample size was 216 per group (513 ÷ 2.37) within both the YC and the OC.
This sample size would provide 95% power at a 5% significance level to detect a between-group difference of 0.35 standard deviation (SD) units, within each age band. Assuming the SD of the primary outcome [i.e. the Strengths and Difficulties Questionnaire-Total Difficulties Score (SDQ-TDS)] to be 6 points, this equated to an average between-group difference of 2.1 points. There was also 80% power to detect differences of 0.27 SD units (i.e. 1.6 points on the SDQ-TDS).
Assuming an equal split between boys and girls, then, within each gender, the study had 95% power to detect intervention effects of 0.49 SD units (3.0 SDQ-TDS points), or 80% power to detect an effect of 0.38 SD units (2.3 SDQ-TDS points).
We aimed to recruit 38 primary schools to minimise the impact from any school dropping out during the trial. We expected 1026 P1 (aged 4–6 years) and 1026 P5 (aged 8–10 years) pupils to participate (allowing for 25% attrition), and 1094 sets of parents (allowing for 60% attrition). We estimated that there would be 200 teachers of the OC and YC, but all staff would be invited to complete evaluation questionnaires.
Recruitment, consent and retention
Local authorities level
The directors of education in the selected LAs were approached by e-mail and telephone. This was followed by meetings with nominated EPs and QIOs. Three LA directors of education granted permission, and we were instructed to contact primary schools directly, inviting them to take part. One LA refused because it was in the middle of a restructure.
School level
Purposive sampling was used to recruit 38 schools across the three LAs between November 2012 and February 2013. Demographic information for the participating LAs was collated for all potential primary schools, and clustered learning communities were identified. We prioritised inviting schools that enabled us to represent LA levels of rurality, denomination, school size and proportion of pupils eligible for free school meals (as a proxy for deprivation).
We aimed to recruit up to three primary schools belonging to the same learning community to limit the number of secondary schools needed for data collection following the OC’s transition from primary to secondary school, and owing to limits on the number of learning community clusters available in each LA. Invitation letters and participant information sheets (see Report Supplementary Material 4) were sent to primary school HTs by post, with follow-up telephone contact and visits in person by the research team if requested by the school. Two further rounds of selection and invitation were necessary owing to lack of response or schools declining to participate. In total, 91 schools were approached, and recruitment stopped (in February 2013) when 38 primary schools from 18 learning communities had agreed to participate: 15 schools in LA1, 13 in LA2 and 10 in LA3. Once a school agreed to participate, it was sent an information pack with a checklist of next steps and asked to nominate a key contact to liaise with the trial team.
At the end of T1 (early summer 2015), link secondary schools in participating primary schools’ learning communities were contacted to give notice of the trial and to invite them to support the ongoing data collection for the participating pupils joining their schools in August 2015.
Individual level: school staff, pupils and parents
As the SEED intervention operated at the whole-school level, once a school had agreed to participate, it was not possible for school staff or parents/carers of individual pupils in intervention schools to opt themselves or their child out of the intervention itself, only the data collection for evaluation. For the data collection, individual-level recruitment was required for two cohorts of pupils in each school, their parents/carers and all primary school staff; informed consent was sought for all.
Pupils
In early 2013, parents/carers of pupils in both cohorts at participating primary schools were sent invitation letters and participant information sheets (see Report Supplementary Material 4) seeking opt-out consent for their child to be part of the data collection for the SEED intervention and evaluation. Opt-out consent was critical because previous research over many years has shown that the lower participation with opt-in consent is strongly biased away from the most vulnerable children. 89 Letters were produced with both the school and the MRC/CSO Social and Public Health Sciences Unit (SPHSU) logos and were signed by both the school HT and trial chief investigator. Letters were sent to pupils’ home addresses via the school office. A date was given on the letter by which opt-out slips should be returned, giving parents/carers at least 2 weeks to consider the information and ask questions before data collection would commence. Opt-out slips were collated by the schools and reported to the research team or returned directly to the research team by parents/carers. Ahead of each wave of data collection, participant lists of consented pupils were sent by the trial team to the school’s key contact confirming trial participants. Participant information sheets explained that pupils could be withdrawn from the evaluation at any time by their parents/carers.
Before the first follow-up data collection (T1, 2015), this process was repeated for pupils who were new to the participating primary schools since baseline. No new participants were recruited to the study at T2 or T3. Parents/carers of existing participants were sent letters at each follow-up wave of data collection reminding them of the trial, explaining the process for the current wave of data collection and reminding them of their right to withdraw their child from this.
In addition to parental consent, pupils at classroom sessions were reminded that completion of the self-completed questionnaire was voluntary, even if their parent had not opted them out. Pupils at these sessions (OC at all waves; YC at T3 only) were read an age-appropriate information sheet (see Report Supplementary Material 4) by a member of the research team and given an opportunity to ask questions before beginning the questionnaire. If pupils were absent, an ‘absentee pack’ containing their questionnaire, information and instructions for their teacher was left with the school and returned at a later date via post or collected from the school (see ‘instructions for absentees’ in Report Supplementary Material 4).
Parents/carers
Parents/carers of the two cohorts of pupils were invited to complete parent surveys related to their school and their child at T0, T1 and T3. In the initial letter to parents asking for consent to include their child in the trial, parents were informed that, if their child took part in the trial, parents/carers would also be sent voluntary questionnaires. Parents/carers were not assigned separate identifiers, but were given questionnaires with the identifier of their child; therefore, no record was made of the name of the completing parent/carer. Parent/carer questionnaire packs were sent via school bags (‘pupil post’) with a covering letter and prepaid envelopes for return to the research unit. Consent to participate was assumed by return of completed questionnaires.
Primary school staff
At each wave, the school’s key contact was asked to provide a list of current staff whom they wished to include in the staff survey. A recommendation was made that all teaching and non-teaching staff be included, but this was ultimately at the discretion of each school. Thus, the number of additional staff, such as auxiliary, janitorial or visiting staff (e.g. part-time music teachers), varied between schools. All staff identified by the school were sent a participant information sheet, a letter and a copy of the staff questionnaire (see staff questionnaire in Report Supplementary Material 5 and subsequent changes listed in Report Supplementary Material 6). Consent to participate was assumed by return of completed questionnaires at each wave. This recruitment process meant that there were new staff participants in all four waves.
Retention
To optimise retention among trial schools, each participating school was given £500 on completion of baseline data collection and a further £500 following the final follow-up (T3). In addition, a payment at each wave was made to compensate for the time taken by teachers to complete SDQs for their pupils at a rate of £4.38 per participating pupil, calculated on the basis of the cost of 10 minutes of primary teacher cover.
Throughout the life of the trial, the research team maintained communication with schools, staff and parents, providing periodic updates on the status of the trial and responding to any requests for information in a timely manner. Christmas cards were sent to all participating schools each year. During the trial, several primary schools changed HTs or key trial contacts. The new HTs were contacted by e-mail in the first instance, and this was followed up by telephone calls and/or meetings in person to explain the trial and the school’s involvement in it. In all these instances, these schools were retained in the trial.
For staff and parents, reminder letters were sent with the questionnaires in each wave. Schools were encouraged to let parents of the participating cohorts know of the trial through school newsletters to raise awareness and encourage participation. When possible, schools were asked to set aside collegiate time to complete the SEED survey to optimise participation. Regardless of how it was administered, staff were told that they could return their questionnaires in a prepaid envelope directly to the research unit to reassure about confidentiality of responses.
Over the course of the four waves of data collection, a small number of pupils who had previously been opted out were opted back in by their parents/carers, and a small number of pupils were opted out. Some pupils moved between trial schools, and some pupils left trial schools and were not located in other trial schools.
Withdrawals
The right to withdraw from data collection was emphasised to all individual pupils, parents/carers and staff at each wave. Pupil and parental opt-outs, withdrawals and pupil refusals (pupils who had been consented by parents/carers, but who opted themselves out of the self-completed questionnaire) were recorded with reasons for withdrawal, when known, and a check to ensure that this was actioned on the trial participant database.
Data collection
Questionnaire data were collected via two main methods: classroom-based fieldwork sessions supporting self-completion of pupil questionnaires, and self-completion of questionnaires by adults and return via post or school collection (teacher-completed SDQs, staff questionnaires and parent/carer questionnaires). For example copies of the questionnaires used at T0, see Report Supplementary Materials 5, 7 and 8. All changes to questionnaires from T0 can be found in Report Supplementary Material 6.
In the final follow-up at T4, questionnaires were primarily administered electronically on tablets or on pupils’ own phones.
Confidentiality and anonymity
All data were anonymised. Pupil and staff participants were allocated unique identification numbers (IDNOs) and barcodes were used on all questionnaires and detachable pages to account for and track returned data. Identifying information linking participants to IDNOs was stored securely in a database accessible only by main research and Population Health Research Facility (PHRF) staff at the MRC/CSO SPHSU, University of Glasgow, and held separately from participant questionnaire responses. Questionnaires for parents/carers of participating pupils were allocated the IDNOs for the pupil they were being asked to complete questionnaires about. All questionnaires were designed with detachable pages so that names could be removed from blank questionnaires before completion and identifying data (e.g. postcode or job title) would be returned, processed and stored separately from other non-identifying questionnaire responses.
Results forming part of the intervention were fed back to schools randomised to the intervention in aggregate form, broken down by year group. Neither staff nor parents were informed of individual pupils’ SDQ scores. If parents contacted the trial team requesting to know their child’s SDQ scores, we reminded them of our commitment to confidentiality regarding individual scores. Responses from free-text fields in the staff and parent questionnaires were fed back to schools in a way that removed any identifying information.
Care was taken to conduct interviews ethically and sensitively. Complete confidentiality of interview content and recordings was ensured, and best efforts were made to ensure that quotations and comments could not be traced back to the interviewee. Participating schools will not be identified in any publication and pseudonyms will be used when necessary.
Classroom-based fieldwork
Pupil self-completed written questionnaires (including, but not limited to, the SDQ) were administered in a classroom session lasting approximately 1 hour, supported by a team of trained fieldworkers from the PHRF.
Classroom sessions were scheduled in liaison with the school for a suitable time during school hours and a detailed overview of the session was given to schools in advance. For primary school sessions, schools were asked to block off 1 hour. For secondary school sessions, two periods (40–50 minutes each) were usually allocated, with pupils returning to class either at the end of the first period or whenever they completed the questionnaire during the second period.
In both primary and secondary settings, at the beginning of the session, fieldworkers introduced their team to the pupil group; explained what the session would entail; emphasised that consent had been given by their parents/carers, but that pupils could opt not to complete the questionnaire if they wished; and answered questions. Throughout the session, pupils completed their individual questionnaires supported by fieldworkers who answered questions as they arose or offered one-to-one support if pupils required it (e.g. reading the questions aloud). During questionnaire completion, care was taken to ensure that pupils had as much privacy as possible. Pupils were requested and encouraged to complete questionnaires quietly on their own and not to confer with neighbours. At the end of the session, pupils and staff were thanked for their participation and support and pupils were given a debrief leaflet (see Report Supplementary Material 9) that included a telephone number for ChildLine, and a verbal explanation of the leaflet.
Returned questionnaires
Teacher-completed SDQs (for YC pupils in all waves and OC pupils at T0 only) were delivered to the school along with instructions on how these should be completed, deidentified and returned. The majority of these were then collected in bulk from the school by fieldworkers, but small numbers were returned in Freepost envelopes to the research base.
Staff and parent/carer questionnaires were packed in individual questionnaire packs containing a participant information sheet (see Report Supplementary Material 4) and Freepost return envelopes. Staff and parents/carers were given the option of returning these individually by post in the envelopes provided or by leaving them with the school office, where a SEED fieldworker would collect them. Instructions were given on how to detach pages that contained identifying data, and participants were asked to return these in Freepost envelopes separate to the main questionnaire.
Data processing, entry and management
A Microsoft Access® (Microsoft Corporation, Redmond, WA, USA) database was used to generate IDNOs, to securely store participants’ names (and associated school details) and to log and track questionnaires via barcodes attached to each questionnaire/detachable page. Anonymised questionnaires were passed to the Robertson Centre for Biostatistics (RCB) Clinical Trials Unit at the University of Glasgow for data entry, then returned to the SPHSU for further data entry and later archived. The detached identifying data and all free-text data were entered in a blind two-pass system by two different PHRF staff members onto a Microsoft Access database. Any disagreement between these passes resulted in the entries being flagged for a third pass. If there was any further disagreement about what had been written by participants or uncertainty about what should be entered, the research team was consulted.
Pupil self-completed questionnaires that were completed electronically at T4 were transferred directly to a secure PHRF database, removing the need for manual entry. Anonymised data were then passed via secure file transfer to the RCB.
On return from the RCB, anonymised paper questionnaire data were stored in a locked, secure data room in the PHRF. Electronic questionnaire data entered by the PHRF were stored confidentially on password-protected servers. All data collected were stored securely in compliance with the General Data Protection Regulations for the European Union and UK (URL: https://gdpr-info.eu). Data were kept in either locked filing cabinets or password-protected databases accessible only by main researchers and PHRF staff. The final data set was accessed only by approved members of staff from the research team and the RCB. All data will be kept for 10 years, in line with University of Glasgow research governance framework regulations for clinical research.
Randomisation
Schools were randomised to control or intervention arms after collection of baseline data in April 2013.
Randomisation was performed when school-level data were available for 38 consenting primary schools. These schools were clustered within 18 learning communities. This was achieved using a computer program written by co-investigator and author Alex McConnachie, within the RCB, using the following procedure.
There were 48,620 possible ways of allocating nine learning communities to the intervention and control groups, from the 18 learning communities involved in the study. Of these, 13,050 possible allocations would achieve an equal allocation of primary schools into each arm of the trial. Of these, 792 achieved the minimal imbalance of primary schools within each LA. Of these, 384 allocations minimised the imbalance with respect to urban/rural classification. Minimising imbalance in terms of school denomination (Roman Catholic or non-denominational) reduced the number of potential allocations to 240. A total of 68 possibilities minimised imbalances within each stratum of minority ethnic populations, and 36 minimised imbalance with respect to the percentage of the school population with free school meal entitlement. Finally, 14 potential selections were identified that ensured that the total expected number of children in each arm of the trial differed by no more than 10%, within both the P1 and P5 cohorts. One of these sets of allocations was then selected at random as the final allocation of schools to the intervention and control arms of the trial.
Once generated, the allocation list was stored in a secure area of the RCB network, with access restricted to those responsible for generating school feedback reports. The statisticians responsible for producing the final analysis report did not have access to the random allocations until all statistical programming had been completed and checked, and the trial database had been locked.
Outcome measures
Primary outcome measures
The primary outcome measure for pupils was the SDQ-TDS. 32,69–71 This is an internationally validated, publicly accessible 25-item behavioural screening questionnaire designed to assess levels of emotional and behavioural problems comprising four ‘difficulties’ subscales and one ‘strengths’ subscale. The SDQ-TDS is calculated by summing the scores on the four difficulties subscales.
The pupil self-completed and parent/carer SDQs were administered as part of the larger pupil/parent questionnaires (see the T0 pupil questionnaire in Report Supplementary Material 7, the T0 parent/carer questionnaire in Report Supplementary Material 8 and all changes to questionnaires from T0 in Report Supplementary Material 6). The teacher-completed SDQ was administered as a stand-alone assessment measure given to teachers to complete in class time (see Report Supplementary Material 10). For the OC, the SDQ was self-completed at all five time points (including T4, when only the OC were surveyed), was completed by teachers at T0 and was completed by parents at T0, T1 and T3. The YC SDQs were teacher-completed at four time points (T0 to T3), self-completed at T3 only and parent-completed at T0, T1 and T3. See Appendix 2 for details of the SDQ scoring.
Secondary outcome measures
For pupils, the secondary outcome measures assessed in the pupil questionnaire were as follows:
-
The five validated SDQ subscales: four difficulties subscales (emotional symptoms, conduct problems, hyperactivity/inattention and peer relationship problems) and one strengths subscale (prosocial behaviour). Higher scores represent greater issues in all the difficulties subscales, whereas lower scores represent greater challenges in the prosocial ‘strengths’ scale.
-
Pupils’ social and emotional skills: self-awareness, social awareness/empathy, self-management, responsible decision-making, emotional regulation and self-esteem.
-
Pupils’ well-being and relationships at school – school relationships; whole and class level, liking school, happy friendships, school climate, experience of antisocial behaviour and participation in antisocial behaviour.
-
Pupils’ well-being and relationships at home – family relationships, materialism and daily quality of life.
These measures were collected through pupil self-completed questionnaires. The questionnaires were designed (some for previous studies such as Healthy Respect Phase 2,90 which involved some of the SEED RCT co-investigators) and refined through piloting. 68,91
The secondary outcome collected from the OC at T2 and T3 was the impact of the intervention on health-related behaviours (smoking, drinking and drug use).
For staff, the secondary outcomes were staff attitudes towards pupil behaviour, pupil confidence, pupil engagement and pupil relationships.
Additional quantitative staff measures were collected on staff–staff relationships, school ethos, self-efficacy, staff mental health [Warwick–Edinburgh Mental Well-being Scale (WEMWBS),9,10,92 used, with permission, from T1 onwards], perceptions of management, staff–pupil relationships, staff support, training opportunities, staff who played a key role in developing SEW in the school and school support for SEW. For a full list of measures, including sources (when applicable), see the T0 staff questionnaire in Report Supplementary Material 5 and all changes to questionnaires from T0 in Report Supplementary Material 6.
Changes to outcomes
The pupil domain scores were refined between baseline and the first follow-up to validate the groupings of questionnaire items fed back to schools as part of the intervention. Confirmatory factor analysis was carried out, which did not alter the items included in questionnaires, but changed the membership of items within some of the secondary outcomes (see Report Supplementary Material 5 for the T0 staff questionnaire and Report Supplementary Material 6 for all changes to questionnaires from T0).
Blinding
The trial researchers were not blind to intervention arm. PHRF fieldworkers were kept blind to which schools were intervention or control, but it was occasionally necessary for unblinded trial researchers to accompany and support fieldwork. EPs supporting the intervention delivery were not blinded.
Statistical methods
Analyses were conducted to compare the mean SDQ-TDS at T3 between the intervention schools and control schools. For the YC, teacher-completed SDQs were used for the analysis at all visits; for the OC, teacher-completed SDQs were used at the baseline visit, with self-completed questionnaires being used at the follow-up visits. The treatment effect estimate, 95% confidence interval (CI) and p-value were estimated from a multilevel linear regression model with mean SDQ-TDS as the outcome. Predictors within the model were baseline SDQ-TDS, trial arm, cohort, gender and stratification variables that were used when carrying out the randomisation (see above).
In the model, hierarchical random effects were included to account for the clustering of pupils within primary schools and secondary schools. Components of variance and ICCs were reported. In the prespecified analysis plan, there was no plan to adjust the p-values for multiple testing; therefore, all p-values have been reported without adjustment.
This model was extended to estimate the intervention effects within subgroups of the population defined by cohort (younger/older), gender (male/female) and deprivation level (deprived/not deprived). Interaction terms were added to the model to provide separate intervention effect estimates for each subgroup. These analyses were repeated for T1 and T2 data.
In addition, repeated measures analyses were carried out for the primary outcome at all time points (baseline and three follow-ups) simultaneously. Time point was included as a categorical variable, and intervention effects were estimated by including intervention-by-time point interaction terms. Models were adjusted for cohort, gender and the randomisation stratification variables. Gender and cohort were further examined using subgroups of the population (younger male/older male/younger female/older female) and extending the model to estimate the intervention effects within these subgroups.
Analyses of all secondary outcomes (apart from health-related behaviours for the OC) were carried out using a repeated measures analysis over all time points, as for the primary outcome. For all analyses, ICCs were estimated. OC pupils’ health-related behaviours were summarised at each time point (T2, T3 and T4 only) and trial arms were compared using Fisher’s exact tests.
Additional analyses
To test the robustness of the analysis, sensitivity analyses were performed by looking at alternative assumptions regarding missing outcome measures. Assumptions included last observation carried forward and return to baseline. This analysis was carried out on the primary analysis results only.
The secondary outcome repeated measures analysis models were extended to estimate the intervention effects within the gender, cohort and deprivation level subgroups by including interaction terms for each group in the model to provide separate intervention effects.
For the staff domain scores that were not included as secondary outcomes (staff–staff relationships, school ethos, self-efficacy, perceptions of management, staff–pupil relationships, staff support, training opportunities, valued member of staff and school support for staff emotional well-being), repeated measures analyses were carried out at all time points, as for the primary and secondary outcomes.
The SDQ-TDS was split into two groups using cut-off points (normal: 0–15; not normal: ≥ 16); using these subgroups at baseline, interaction terms were added to the SDQ subscale repeated measures models to determine the intervention effect within these subgroups. In addition, the responses to staff questionnaires were examined for the subgroups of teaching and non-teaching staff.
The OC had the additional T4 visit when the pupils were in fourth year of secondary school. Data collected at T4 were used in the analyses only if the pupil had data recorded for at least one of the first three follow-up visits. The main analyses conducted for the primary outcome were repeated to include these additional T4 data. This model was extended to estimate the intervention effects within subgroups of the population defined by gender (male/female) and deprivation level (deprived/not deprived). Repeated measures analyses were carried out at all time points (T0 to T4) to determine the effect of the intervention over the duration of the trial.
The analyses conducted for the secondary outcomes were repeated to include the data collected at T4, and the health-related behaviours were summarised for this follow-up visit, with Fisher’s exact tests carried out to compare the trial arms.
Process evaluation
A process evaluation was embedded in the trial to interpret the trial outcomes and to answer secondary research questions related to process. The delivery of the SEED intervention was monitored in all intervention schools primarily through observation of data feedback and RD sessions in all schools, and in-depth interviews with key contacts (usually the HT or depute HT) at the start and the end of the trial, when possible. Additional data collected included qualitative and quantitative data from staff, pupil and parent questionnaires and researchers’ perceptions of schools. For details on data sources, see Appendix 5, Table 55.
In addition, in-depth data were collected from four case study schools, which were purposively selected from the intervention and control arms of the trial in year 2 of the trial. The case studies represented schools that were active in engaging with the SEED evaluation and schools with reluctant engagement, and schools with high versus schools with low SDQ scores at baseline (Table 1). Engagement was measured by the response rate to the baseline staff survey. One intervention case trial school was replaced in year 3 and we had limited data for another school owing to difficulties arranging interviews with the key contacts. In response, we decided to supplement the four original intervention case study schools in year 5 of the trial with two additional schools based on the original selection criteria. In the case study schools, additional data were collected, including interviews with EPs and focus groups with pupils from both cohorts (see pupil focus group and EPs interview topic guides in Report Supplementary Material 11). Four control schools were selected as case studies. However, for the purposes of this report, data from all control schools were analysed because of the relatively small number of data available for control schools. No process evaluation data were collected or reported at T4.
School ID | When selected | Selection characteristics | LA | School demographics (size, rurality, denomination, level of deprivation) |
---|---|---|---|---|
IC3P | Year 2 (after baseline) | Active engagement, high SDQ scoresa | C | Very large school, large town, non-denominational, low deprivation |
IB1P | Year 2 (after baseline) | Active engagement, low SDQ scores | B | Large school, city, Roman Catholic, high deprivation |
IB2P | Year 2 (after baseline) | Reluctant engagement, high SDQ scoresa | B | Very large school, city, non-denominational, high deprivation |
IA6P | Year 2 (after baseline) | Mixed engagement (started active, ended up reluctant), low SDQ scores | A | Large school, large town, non-denominational, medium deprivation |
IA3P | Year 5 (after final follow-up) | Active engagement, medium SDQ scores | A | Small school, rural, non-denominational, low deprivation |
IA8P | Year 5 (after final follow-up) | Reluctant engagement, high SDQ scoresa | A | Large school, large town, non-denominational, high deprivation |
In the original trial protocol,66 we planned to undertake observations of teachers’ classroom management skills. Once the first round of data feedback and schools’ APs was complete, it became clear that classroom management was only one type of classroom initiative that schools chose to implement (others selected other initiatives from the SEED Resource Guide, available in Report Supplementary Material 3); few did so, and thus there was limited value in observing lessons. Instead, this resource was transferred to observe and record all the RD sessions, as these are a critical and consistent component in the implementation of the intervention. Questions on the type of classroom or whole-school SEW initiatives being implemented were included in the staff survey, and additional questions were asked on fidelity and breadth of implementation across the school.
For topic guides for HT interviews, see Report Supplementary Material 11. Note that ‘baseline’ interviews were collected at T1 and covered perspectives on taking part in the SEED RCT in the baseline wave (and throughout the trial for the T3 interviews). The interviews also explored the school’s context and practices around SEW prior to the SEED intervention, reasons for taking part in the SEED trial and (for intervention schools) the process of implementing the intervention and perceived impact. All schools were invited to participate in interviews; when possible, these were carried out in person for the case study schools and over the telephone for the other schools. Baseline interviews were carried out in five case study schools (two intervention and three control schools) and 17 non-case study schools (seven intervention and 10 control schools). T3 interviews were carried out in six case study schools (four intervention and two control schools) and 19 non-case study schools (11 intervention and eight control schools). All pupil focus groups were conducted by research staff and/or fieldworkers in person following a data collection visit (see Report Supplementary Material 11 for focus group topic guides). Interviews and focus groups were conducted by members of the research team, digitally recorded and transcribed verbatim. Observational notes were taken and written up by members of the research team.
Ethics and consent
For the HT interviews, staff members were given information sheets and sufficient time to ask questions before deciding if they wished to take part. Written consent was obtained prior to conducting the interviews if held face to face. Verbal consent was obtained for telephone interviews and written consent forms returned by post following completion of interviews. Prior to interviews, informed consent was gained from the school HT and interviewee (if different). Parental consent was gained for participation in pupil focus groups and informed written consent was also sought from pupils in person (see the consent forms in Report Supplementary Material 4 and process evaluation consent forms in Report Supplementary Material 11). Information on storage of data can be found in Data processing, entry and management.
Analysis
A coding framework was developed through initial exploration of the data, pilot coding and by discussion between Sarah Blair, Daniel Wight, Marion Henderson, Susie Smillie, Carrie Parcell and Craig Macdonald, and was informed by MRC guidance on process evaluation in RCTs of complex interventions. 93 The coding framework was structured into six sections: (1) the context for SEW in schools (pre-existing situation prior to involvement in the SEED trial); (2) methodological issues of participation in the SEED trial; (3) implementation processes for the SEED intervention; (4) results of the SEED intervention (immediate effects of the SEED intervention or other SEW programmes that are observable/visible, are attributable to the SEED intervention/SEW programmes and can be directly influenced by school); (5) implementation processes of SEW programmes in control schools; and (6) results of SEW programmes in control schools. For more detail on the full coding framework, see Appendix 5, Proposed simplified Social and Emotional Education and Development coding framework.
A first stage of analysis was carried out prior to knowing the outcomes of the trial, to avoid bias in the analysis. 93 Case study data were coded to the agreed framework by Sarah Blair and Carrie Parcell using NVivo 10 (QSR International, Warrington, UK). The coding framework was mapped to the principal research questions from the protocol and to a set of enhanced research questions. These additional research questions explored the differences between the arms of the trial in SEW activities implemented, the extent to which the SEED intervention worked as intended, the degree to which the SEED intervention was embedded in intervention schools and the contextual factors that may have facilitated or obstructed the implementation of the SEED intervention (see Appendix 5, Table 55). The relevant codes were then extracted, and analysis of the case study schools was conducted independently by Sarah Blair, Daniel Wight, Sally Haw and Lawrie Elliott. An interim report was produced following discussion and consensus.
Following the main trial outcomes, a further stage of analysis was conducted to help interpret the findings. This stage focused on the SEED theory of change (see Figure 2) to assess the strength of existing data and the evidence available to explore further. Each proposed pathway within the theory of change was mapped to the existing data; if there were sufficient data from the case studies to provide evidence for the proposed pathways in the theory of change, no further analysis was undertaken. If evidence was moderate or weak, additional coding and analysis were performed on the data available from all schools using the existing coding framework. At this stage, selected data were also explored thematically to gain a more nuanced understanding of the mechanisms of change underpinning the intervention and to allow the generation of themes that could explain pathways beyond those proposed in the theory of change.
Further details on the methods employed in the analysis of the mixed-methods and quantitative data can be found in Chapter 6.
Economic evaluation
Overview
As the effects of the SEED intervention are expected to affect costs and outcomes outside health services, in line with NICE’s recommendation for public health interventions, a public sector perspective [including NHS, Personal Social Services (PSS) and local government perspectives] was adopted for the analysis. 94 Data on the costs include those costs associated with the provision of the intervention, including staff time, travel and consumables at each stage of the intervention, as well as the resource use cost collected at T1 and T3 in the public sector, such as general practitioner (GP) visits and police visits. The Child Health Utility-9 Dimensions (CHU-9D)95–98 was used in the trial to collect the utility data, among the OC only, for the cost–utility analysis via the self-completed questionnaires. As the SDQ-TDS was the primary outcome for both cohorts, this enabled a cost-effectiveness analysis, per se, of all the pupils. The costs and outcomes were both discounted at 1.5% per annum in the base-case analysis, as recommended in the NICE methods of technology appraisal and public health guidance. 94 All analyses were undertaken according to the principle of intention to treat and in Stata®/SE 14.0 (StataCorp LP, College Station, TX, USA).
Intervention costs
Data on the cost of implementing the SEED intervention were collected by the research team. The intervention cost was concerned with three components of the intervention (as stated in Chapter 2, Key components of the Social and Emotional Education and Development intervention): (1) a needs assessment of pupils using the SDQ screening tool collected during baseline data collection, (2) analysis of SDQ data by the research team and (3) feedback provided to schools based on the needs assessment through reflective feedback sessions and the development and implementation of APs.
The economic evaluation focused on the ‘incremental’ (or ‘additional’) cost of the intervention, compared with the control group. The control group received the same needs assessment data at the end of the trial; however, it did not receive analysis and reflective feedback sessions from the research team and EPs. Given this, the cost of SDQ data collection for each participant was offset as it was conducted in both arms. Any additional costs for the resources purchased by the school to carry out the activities were not included in the intervention cost, as the resources required varied because of the specific activities recommended and the existing resources in the school. Despite this, we recorded the extra resources required in each school (see Appendix 4, Table 49). Most of the activities were carried out with no additional cost (i.e. based on existing resources), and the per-pupil additional cost was minimal. Compared with the control group, the intervention is viewed as a reflective and co-produced process during which the bespoke activities were selected based on the data collated by the research team. Any ‘research-related’ cost was removed from the intervention cost. The intervention cost is collected specific to each school; thus, the school-specific per-pupil intervention cost was added to each pupil’s resource use cost in the intervention arm.
Healthcare, social care service and other public service resource use
Resource use data were collected from the pupils’ parents/carers via questionnaires (see Report Supplementary Material 12) that were distributed via pupil school bags at T1 and T3. Public sector service use data collected included visits to the NHS services, social care services and other public sector services. The NHS services included GP visits; practice nurse visits; accident and emergency attendance; visits to speech therapists, occupational therapists, school nurses, physiotherapists, psychiatrists and child and adolescent mental health services; outpatient appointments; inpatient stays; NHS dentist visits; and NHS optician visits. The use of social care and public sector services was collected in terms of social worker visits, EP visits and police attendance. Unit costs in the financial year 2016/17 published by the Personal Social Services Research Unit (PSSRU)99 or NHS reference costs100 were attached to each item of resource use. Other published sources were used if the unit cost was unavailable from the above two sources (see Appendix 4, Table 50).
Utility and quality-adjusted life-years
The CHU-9D95–98 was collected from the OC at T0, T1, T2 and T3. Utility values were generated from the CHU-9D index values for the OC and expressed as quality-adjusted life-years (QALYs). The area under the curve method was used to estimate QALYs over a 12-month period, following the trapezium rule, assuming a linear change in utility between each assessment time point. 101 The CHU-9D is a generic preference-based health-related quality-of-life instrument suitable for use with children aged 7–17 years. It has demonstrated itself to be a practical and valid measure for use in economic evaluations of child and adolescent healthcare programmes. 95–98 Valuation of the CHU-9D was directly elicited from an adult and adolescent population. Preference weights were derived from 300 members of a UK adult population using a standard gamble technique for use in children aged 7–11 years. 95,96,98 Subsequently, preference weights were derived from best–worst scaling discrete choice experiment interviews of 590 Australian adolescents aged 11–17 years. 95,96,98 The Australian adolescent weights were used for valuation of the CHU-9D responses in this trial. The utility values range from 0.3261 [if the respondent answers level 5 (worst state) for all dimensions] to 1 [if the respondent answers level 1 (perfect state) for all dimensions].
Discounting
The final evaluation costs were valued in the year 2016/17, which was the year of the latest version of the published unit cost in PSSRU and NHS reference costs. As the SEED intervention accrues costs each year over the 4 years, discounting was required to bring future costs and benefits to present value. Therefore, in line with NICE’s recommendations for economic evaluations of public health interventions, a discount rate of 1.5% was applied to both costs and health effects in the base-case analysis, and a discount rate of 3.5% was used in sensitivity analyses. 100
Missing data
The resource use data were collected for the 12-month periods prior to T1 and T3 only, because parents were not surveyed at T2 (partly to reduce burden on schools), which means that the resource use data between T1 and T2 are missing. Given that costs and outcomes should cover the entire trial period, it was decided that available resource use data between T2 and T3 be used as a proxy for the missing resource use data between T1 and T2. For the missing individual responses in the questionnaires, or the missingness of the whole questionnaire, removing incomplete cases is usually not recommended as this may bias estimates or result in a large number of data being discarded, leading to severely reduced power. 102,103 To maximise the use of the returned resource use questionnaires, the incomplete response of any single item was assumed to indicate that no resource was used for that item during the assessment period. This was followed by a complete-case analysis because of the very high number of missing data (see Chapter 1, Whole-school intervention approaches). According to Jakobsen et al. ,104 if the missingness is considerable (the authors give an example of > 40%), a complete-case analysis is recommended, but the results may be considered only as hypothesis-generating. 104 Therefore, a complete-case analysis was adopted. In addition, multiple imputation with chained equations was also initially conducted for log-transformed utility values and log-transformed total costs at each data collection wave at aggregate level (compared with individual dimension and resource use item). However, the two-tier clustered estimation model could not proceed, which may be because of the large number of missing data; therefore, the complete-case analysis was determined to be the base-case analysis.
Cost–utility analysis
Like the primary outcome analysis, multilevel mixed-effects models were used to estimate the difference in mean costs and QALYs, which adjusted the standard errors and 95% estimates for the two-tier nested clusters, that is primary schools linked to each secondary school (learning communities). This cluster effect assumed a fixed effect within each cluster, but a random effect between the schools at each tier of the cluster. This non-independence is measured by the ICC, which represents the proportion of variance due to between-cluster variation. 105
The distributions of the costs and the QALYs were visualised using histograms. The mixed-effects generalised linear models were originally performed with gamma family and log-link, which took into account the skewed cost distribution and QALY distribution, followed by a recycled prediction to estimate mean cost and QALYs for each arm. However, owing to the large number of missing data, the number of pupils within each cluster significantly reduced. This led to the failure to converge of the generalised linear model, and the subsequent recycle prediction could not proceed. To address this, the right-skewed cost data (i.e. majority of the participants have small cost) were log transformed and a linear mixed-effects model was applied. In contrast to the distribution of the cost data, the QALY data were left skewed, with a large proportion of observations lying at the higher end of the QALY scale. Therefore, the QALY data were transformed by taking the log transform of the decrements of QALYs, which were calculated as the difference between the maximum number of discounted QALYs that could possibly be accrued within the 4-year time horizon (i.e. 3.92) and the actual QALYs gained. 106 This decrement method ensures the distribution of log-transformed QALYs to be closely fitted to a normal distribution.
Covariates for the mixed-effects model were baseline CHU-9D values as recommended37 and all the demographics covariates used in the primary outcome analysis, including baseline SDQ-TDS (continuous, 0–40), gender (categorical: male, female), proportion of pupils receiving free school meals (categorical: < 20%, ≥ 20%), number of pupils in school (categorical: < 250, ≥ 250), religious denomination (categorical: non-denominational, Roman Catholic), location (categorical: large urban, other urban, non-urban), LA (categorical: LA1, LA2, LA3), number of P1 pupils in school (categorical: < 30, ≥ 30) and proportion of pupils from minority ethnic groups (categorical: 0% to < 10%, ≥ 10%, not available or blank).
The incremental cost-effectiveness ratio (ICER) was estimated from the adjusted difference in cost and QALYs from the mixed-effects regression. A 1000-iteration bootstrapping procedure was performed to investigate the uncertainty surrounding the ICER estimate and the probability that the SEED intervention was cost-effective under a wide range of hypothetical thresholds (£0–100,000). Standard errors and 95% CIs for differences in the adjusted cost and QALYs were estimated via the bootstrap, adjusting for clusters. These results were visualised in cost-effectiveness planes and cost-effectiveness acceptability curves (CEACs).
Sensitivity analyses
Five sensitivity analyses varying the cost, and one varying the discount rate, were undertaken (see Appendix 4, Table 50) to assess the robustness of the base-case results of the cost–utility analysis to the alternative cost assumptions. All the scenarios were followed by bootstrap procedures to estimate the uncertainty surrounding the ICER in each. The cost of the intervention is assumed to vary by ± 30%, which takes into account the variations when carrying out this intervention in different settings, for example the analysis and preparation time for the feedback sessions, the staff cost involved, the number of pupils in a school and the location of a school. Owing to the large number of missing cost data, the sample size of the base-case cost analysis was only around 10% of the total population; therefore, the sensitivity analysis varied the assumptions regarding the methods for dealing with the missing data. It was assumed that no resources were used when the data from the whole resource use questionnaire were missing, but CHU-9D was not missing; consequently, the sample size for the economic evaluation was restored to a level similar to that of the primary outcome analysis. With this step, both complete-case analysis and multiple imputation (for the cases that had both CHU-9D and cost missing) were both conducted. The next sensitivity analysis increased the discount rate from the base case of 1.5% to 3.5% for both costs and benefits, as recommended by NICE’s methods for the development of public health guidance. 100 The base-case analysis employed a public sector perspective assuming that the intervention was paid for using public funds. However, given that it is unclear if the intervention will be paid for using public funds or by the individual school when it is rolled out, the last sensitivity analysis employed an educational sector perspective by limiting the cost to the intervention cost only, to reflect the cost-effectiveness when the intervention is paid for by the schools. The difference between the public sector perspective and the educational sector perspective is that the latter excluded the NHS, social care and other public sector resource use costs from the total cost by keeping only the confirmed intervention costs. Therefore, this scenario also addresses the sensitivity of the results to the measurement accuracy of the resource use questionnaire and any potential random NHS, social care and other public sector routine resource use that is unrelated to the intervention.
Cost-effectiveness analysis
In addition to the primary cost–utility analysis, a cost-effectiveness analysis of the whole population (i.e. both the YC and the OC) was conducted on the primary outcome, the SDQ-TDS and the SDQ ‘strengths’ (prosocial) subscale. As with the analysis of QALYs in the cost–utility analysis, the difference in the SDQ scores was estimated using a multilevel mixed model regression adjusting for the school characteristics and then combined with the difference in cost to generate cost per improved SDQ score (both SDQ-TDS and ‘strengths’). A 1000-iteration bootstrap was conducted to explore the uncertainty of these estimates.
Assessment of harms
The intervention and its evaluation was considered to be of low risk to participants. A protocol was in place in the case of any unexpected adverse events. Fieldworkers were encouraged to report negative outcomes or experiences to the study team.
Chapter 4 Outcome results
Introduction
This chapter presents the outcomes of the trial. We start by describing the achieved recruitment of trial schools and participants and their flow through the trial, and overview their characteristics and any differences by arm of the trial at baseline. Then we explore the primary outcome, the SDQ-TDS, at the prespecified main outcome point, follow-up 3 (T3). We also report the primary outcome at follow-ups 1 and 2, and at the longer-term follow-up 4 (T1, T2 and T4, respectively), including subgroup analyses and prespecified moderator analyses. This is followed by results relating to the OC’s pupil-completed secondary (non-SDQ) outcomes; teacher-completed secondary outcomes related to staff members’ experiences at their schools, as well as their perceptions of pupils’ behaviours and attitudes; and, finally, a summary of the findings and key conclusions.
Recruitment of schools and participants
At the start of the trial, we approached 91 schools, from three LAs, to consider both their eligibility for and participation in the trial. Of those, 53 schools did not reply or declined to participate because of lack of time or because of a recent change of HT and/or other critical staff changes.
In total, 38 schools agreed to take part in the trial, and all their P1 (YC, aged 4–6 years; n = 1467) and P5 (OC, aged 8–10 years; n = 1330) pupils were eligible to participate at baseline (T0). Owing to some pupils leaving and others joining the schools, by T1 numbers of eligible pupils had changed to 1621 in the YC and 1441 in the OC. Some OC pupils were lost at T2, as expected, if they did not progress to the secondary school linked to their primary school learning community.
Flow of schools and participants
One primary school dropped out of the intervention arm shortly after randomisation. No other schools completely withdrew from the trial. However, three secondary schools participated only at T3 (two intervention and one control), and one control primary school did not participate at T3 (see Figure 3).
The flow of individual children up to our prespecified outcome (T3) is described in detail in Figures 3 and 4. Figure 3 presents the flow for pupils who had baseline (T0) data; Figure 4 includes those who joined the trial after baseline.
Characteristics of recruited schools and pupils at baseline, by arm of the trial
Schools
Table 2 presents summary information about the 38 schools that participated in the trial, namely their LA, their deprivation level [Scottish Index of Multiple Deprivation (SIMD) decile], the percentage of pupils who were eligible to receive free school meals, their urban/rural classification, the percentage of pupils from a minority ethnic group, and whether or not the schools were denominational (religious). There were no significant differences between randomised groups, as expected, given the process of balancing the randomisation scheme (see Chapter 3).
Characteristic | Trial group | Significance | |
---|---|---|---|
Intervention | Control | ||
LA, n (%) | |||
LA3 | 5 (26.3) | 5 (26.3) | |
LA2 | 6 (31.6) | 7 (36.8) | 1.00a |
LA1 | 8 (42.1) | 7 (36.8) | |
School roll, mean (SD) | 246.9 (107.9) | 241.3 (114.4) | 0.805a |
Number of pupils, mean (SD) | |||
P1 | 39.0 (18.7) | 28.3 (16.7) | 0.906a |
P5 | 34.4 (15.7) | 35.5 (16.3) | 0.832a |
School SIMD decile (1: most deprived; 10: least deprived), n (%) | |||
1 | 1 (5.3) | 4 (21.1) | |
2 | 3 (15.8) | 2 (10.5) | |
3 | 4 (21.1) | 2 (10.5) | |
4 | 2 (10.5) | 3 (15.8) | |
5 | 4 (21.1) | 5 (26.3) | 0.206a |
6 | 0 (0.0) | 1 (5.3) | |
7 | 4 (21.1) | 0 (0.0) | |
8 | 0 (0.0) | 2 (10.5) | |
9 | 1 (5.3) | 0 (0.0) | |
10 | 0 (0.0) | 0 (0.0) | |
Percentage of pupils receiving free school meals, mean (SD) | 22 (10) | 24 (16) | 0.919b |
Area (urban/rural), n (%) | |||
Accessible rural areas | 4 (21.1) | 2 (10.5) | |
Accessible small towns | 1 (5.3) | 1 (5.3) | 0.903a |
Large urban areas | 5 (26.3) | 6 (31.6) | |
Other urban areas | 9 (47.4) | 10 (52.6) | |
Percentage of pupils from a minority ethnic group, n (%) | |||
0–5 | 10 (52.6) | 11 (57.9) | |
5–10 | 5 (26.3) | 4 (21.1) | |
10–20 | 1 (5.3) | 1 (5.3) | 0.8511a |
> 20 | 3 (15.8) | 2 (10.5) | |
Not available | 0 (0.0) | 1 (5.3) | |
School denomination, n (%) | |||
Non-denominational | 16 (84.2) | 14 (73.7) | 0.693a |
Denominational | 3 (15.8) | 5 (26.3) |
Pupils
Table 3 presents summaries of the gender, age (cohort) and deprivation level (SIMD decile) of pupils at baseline. No significant differences were observed between intervention and control pupils’ gender and age distributions. However, despite being balanced at school level, there were significant differences between pupils attending intervention schools and those attending control schools in the levels of deprivation of their area of residence. The statistical analysis plan specified that analyses would adjust for variables used to stratify the randomisation; however, given the differences in pupil profiles between the control and intervention schools, it was decided to adjust for individual-level SIMD decile, based on home postcode, rather than school-level deprivation (although the school-level SIMD decile was used when individual SIMD decile was unknown).
Characteristic | Trial group, n (%) | Significance | |
---|---|---|---|
Intervention | Control | ||
Total sample | 1347 (51.0) | 1292 (49.0) | |
Gender | |||
Male | 689 (51.2) | 652 (50.5) | 0.697a |
Female | 656 (48.8) | 640 (49.5) | |
Missing | 2 (0.2) | 0 (0.0) | |
Cohort | |||
YC | 721 (53.5) | 666 (51.5) | 0.824a |
OC | 626 (46.5) | 626 (48.5) | |
Pupil SIMD decile (1: most deprived; 10: least deprived) | |||
1 | 116 (8.6) | 225 (17.4) | < 0.001a |
2 | 228 (16.9) | 144 (11.1) | |
3 | 249 (18.5) | 127 (9.8) | |
4 | 154 (11.4) | 164 (12.7) | |
5 | 202 (15.0) | 215 (16.6) | |
6 | 50 (3.7) | 86 (6.7) | |
7 | 185 (13.7) | 51 (3.9) | |
8 | 25 (1.9) | 139 (10.8) | |
9 | 112 (8.3) | 67 (5.2) | |
10 | 26 (1.9) | 74 (5.7) |
Primary outcome over time and additional subgroup analysis
The primary outcome was the SDQ-TDS at follow-up 3 (T3), which was 4 years post baseline. For the YC, their SDQs were completed by teachers at all four waves, as the SDQ is not validated for pupil self-completion until age 10 years, and at T3 the YC were still not all 10 years of age. For the OC, their SDQs were completed by teachers at baseline (T0), but were self-completed at the three follow-ups (T1 to T3).
Baseline
Table 4 presents summaries of the baseline (T0) teacher- and parent-completed SDQ-TDS and the five SDQ subscales. The primary outcome is the teacher-completed SDQ-TDS; the subscale scores and the parent-completed scores are relevant to additional analysis presented later in this chapter. Teacher-completed SDQ-TDS and subscale scores showed significantly greater difficulties in the intervention group at baseline. The statistical analysis plan prespecified adjustment for baseline scores in all SDQ-related outcome analyses.
SDQ-TDS and SDQ subscales | Intervention group | Control group | Significancea | ||
---|---|---|---|---|---|
T0, mean (SD) | Pupils (n) | T0, mean (SD) | Pupils (n) | ||
Teacher completed | |||||
SDQ-TDS | 6.99 (6.87) | 1345 | 5.68 (6.00) | 1292 | < 0.001 |
SDQ total difficulties subscales | |||||
Emotional symptoms | 1.62 (2.28) | 1345 | 1.11 (1.78) | 1292 | < 0.001 |
Conduct problems | 0.93 (1.75) | 1345 | 0.82 (1.60) | 1292 | 0.042 |
Hyperactivity | 3.14 (3.13) | 1345 | 2.80 (3.07) | 1292 | 0.002 |
Peer problems | 1.30 (1.89) | 1345 | 0.96 (1.64) | 1292 | < 0.001 |
SDQ prosocial subscale | 7.83 (2.45) | 1345 | 8.08 (2.26) | 1292 | 0.023 |
Parent completed | |||||
SDQ-TDS | 7.45 (5.63) | 483 | 7.70 (6.33) | 405 | 0.980 |
SDQ total difficulties subscales | |||||
Emotional symptoms | 1.69 (1.84) | 484 | 1.78 (1.96) | 405 | 0.668 |
Conduct problems | 1.32 (1.56) | 484 | 1.34 (1.79) | 405 | 0.382 |
Hyperactivity | 3.25 (2.48) | 483 | 3.27 (2.67) | 405 | 0.712 |
Peer problems | 1.19 (1.62) | 484 | 1.31 (1.75) | 405 | 0.330 |
SDQ prosocial subscale | 8.48 (1.64) | 484 | 8.53 (1.70) | 405 | 0.349 |
The parent-completed SDQs (see Table 4) did not show any statistically significant differences between the two trial groups. Although all parents were invited to complete questionnaires, only about one-third complied. Therefore, the parents’ questionnaires may not be representative of the whole sample of pupils, but similar proportions of parents responded in each arm of the trial.
Primary outcome at follow-up 3 (prespecified main outcome point)
Table 5 presents summaries of the primary outcome (i.e. SDQ-TDS) at baseline (T0) and follow-up 3 (T3) for those with data at both time points, and for all available data. For those with data at both time points, summaries of the change over time are also provided. Estimates of the between-group differences are provided based on the primary analysis (baseline-adjusted mixed-effects regression), and from a repeated measures analysis of all available data from all time points. For each effect estimate, a standardised effect size is reported, derived by dividing the estimated intervention effect and CI by the residual SD from the model. This statistical analysis and presentation of results is repeated for all the other follow-up time points later in the report.
Data and trial group | Data summaries, mean (SD) [n] | Intervention effect estimates | ||||
---|---|---|---|---|---|---|
T0 | T3 | T3 – T0 | Estimate (95% Cl); p-value | ICC | Effect sizea (95% Cl) | |
Complete-case data | ||||||
Intervention | 6.33 (6.50) [940] | 8.47 (6.72) [940] | 2.14 (7.45) [940] | −0.84b (−1.78 to 0.10); 0.078 | 0.03 | −0.15 (−0.31 to 0.02) |
Control | 5.23 (5.66) [995] | 8.33 (6.95) [995] | 3.10 (7.35) [995] | |||
All available data | ||||||
Intervention | 6.99 (6.87) [1345] | 8.60 (6.73) [1008] | N/A | −1.30c (−1.87 to −0.73); < 0.001 | 0.02 | −0.27 (−0.39 to −0.15) |
Control | 5.68 (6.00) [1292] | 8.50 (7.10) [1096] |
The primary analysis did not show a statistically significant difference between intervention and control schools (p = 0.078), although the trend was towards lower SDQ-TDSs in intervention schools [estimated difference –0.84 (95% CI −1.77 to 0.09), standardised effect size −0.15 (95% CI −0.31 to 0.02)]. The repeated measures analysis, using all available data at all four time points, did show a statistically significant difference (p < 0.001), with the SDQ-TDSs being lower in intervention schools (estimated difference –1.30, 95% CI −1.87 to −0.73). The standardised effect size was small, at −0.27 (95% CI −0.39 to −0.15).
Table 6 provides summaries of changes in SDQ-TDS between T0 and T3 within subgroups defined by age (OC or YC), gender (male or female), deprivation (deprived or not deprived, based on SIMDs), and the combination of age and gender (four subgroups). The baseline-adjusted regression model of SDQ-TDSs was extended to allow for an interaction between each subgrouping variable and the intervention effect, to estimate and test for differential intervention effects between subgroups of children.
Subgroup | Trial group | Data summaries | Intervention effect estimate | Interaction p-value | ||||
---|---|---|---|---|---|---|---|---|
n | T0, mean (SD) | T3, mean (SD) | T3 – T0, mean (SD) | Estimatea (95% Cl); p-value | Effect sizeb (95% Cl) | |||
YC | Intervention | 504 | 6.59 (6.40) | 6.69 (6.50) | 0.10 (6.67) | 0.09 (−0.92 to 1.10); 0.864 | 0.02 (−0.16 to 0.19) | < 0.001 |
Control | 554 | 5.15 (5.15) | 5.47 (6.29) | 0.32 (5.93) | ||||
OC | Intervention | 436 | 6.03 (6.62) | 10.53 (6.39) | 4.50 (7.62) | −1.91 (−2.96 to −0.86); < 0.001 | −0.33 (−0.52 to −0.15) | |
Control | 441 | 5.34 (6.25) | 11.93 (6.02) | 6.59 (7.49) | ||||
Male | Intervention | 477 | 7.67 (6.95) | 9.48 (6.73) | 1.81 (7.73) | −1.16 (−2.22 to −0.09); 0.034 | −0.20 (−0.39 to −0.02) | 0.237 |
Control | 491 | 6.51 (6.13) | 9.65 (7.30) | 3.14 (7.60) | ||||
Female | Intervention | 463 | 4.95 (5.69) | 7.43 (6.56) | 2.48 (7.15) | −0.52 (−1.59 to 0.55); 0.338 | −0.09 (−0.28 to 0.10) | |
Control | 504 | 3.99 (4.86) | 7.05 (6.35) | 3.06 (7.12) | ||||
Deprived | Intervention | 471 | 7.24 (6.94) | 9.28 (6.90) | 2.04 (7.71) | −0.61 (−1.76 to 0.53); 0.296 | −0.11 (−0.30 to 0.09) | 0.487 |
Control | 496 | 5.41 (5.90) | 8.71 (7.29) | 3.30 (7.38) | ||||
Not deprived | Intervention | 461 | 5.42 (5.90) | 7.66 (6.45) | 2.24 (7.19) | −1.05 (−2.17 to 0.08); 0.067 | −0.18 (−0.38 to 0.01) | |
Control | 499 | 5.05 (5.42) | 7.96 (6.59) | 2.91 (7.33) | ||||
Younger male | Intervention | 245 | 7.55 (6.76) | 8.62 (7.07) | 1.07 (7.42) | 0.01 (−1.22 to 1.24); 0.992 | 0.00 (−0.22 to 0.22) | < 0.001 |
Control | 264 | 5.99 (5.35) | 7.34 (7.26) | 1.35 (6.67) | ||||
Younger female | Intervention | 259 | 5.69 (5.91) | 4.86 (5.31) | −0.83 (5.72) | 0.08 (−1.11 to 1.27); 0.895 | 0.01 (−0.20 to 0.22) | |
Control | 290 | 4.38 (4.85) | 3.78 (4.66) | −0.61 (4.99) | ||||
Older male | Intervention | 232 | 7.80 (7.16) | 10.38 (6.25) | 2.58 (7.98) | −2.36 (−3.62 to −1.11); < 0.001 | −0.42 (−0.64 to −0.20) | |
Control | 227 | 7.11 (6.89) | 12.34 (6.38) | 5.23 (8.07) | ||||
Older female | Intervention | 204 | 4.00 (5.28) | 10.69 (6.55) | 6.69 (6.56) | −1.35 (−2.66 to −0.04); 0.043 | −0.24 (−0.47 to −0.01) | |
Control | 214 | 3.46 (4.85) | 11.50 (5.59) | 8.03 (6.53) |
There was a significant interaction between the intervention and the combination of age and gender (p < 0.001), with the strongest intervention effect seen in the older male cohort [−2.36 (95% CI −3.62 to −1.11; p < 0.001), standardised effect size −0.42 (95% CI −0.64 to −0.20)] and, to a lesser extent, in the older female cohort [−1.35 (95% CI −2.66 to −0.04; p = 0.043), standardised effect size –0.24 (95% CI −0.47 to −0.01)], but with no significant intervention effect observed in either YC. This was mirrored in subgroup analyses by age and gender separately, with a significant intervention effect seen among the older, but not younger, pupils (interaction p < 0.001), and among males, but not females, although the interaction with gender was not statistically significant. There was no evidence of different intervention effects according to deprivation.
Primary outcome at follow-up 1
Table 7 shows the analysis of the SDQ-TDS at T1. This is when both cohorts were still in primary school and the intervention process was ongoing. At T1 there was a significant effect of the intervention from both the primary analysis (p = 0.004) and from the repeated measures analysis using all available data (p < 0.001), with lower SDQ-TDS at T1 in the intervention group.
Data and trial group | Data summaries, mean (SD) [n] | Intervention effect estimates | ||||
---|---|---|---|---|---|---|
T0 | T1 | T1 – T0 | Estimate (95% Cl); p-value | ICC | Effect sizea (95% Cl) | |
Complete-case data | ||||||
Intervention | 6.69 (6.68) [1124] | 8.41 (6.69) [1124] | 1.72 (7.35) [1124] | −1.65b (−2.76 to −0.53); 0.004 | 0.054 | −0.29 (−0.49 to −0.10) |
Control | 5.53 (5.86) [1164] | 8.87 (6.78) [1164] | 3.34 (6.90) [1164] | |||
All available data | ||||||
Intervention | 6.99 (6.87) [1345] | 8.52 (6.78) [1240] | N/A | −1.76c (−2.25 to −1.28); < 0.001 | 0.02 | −0.36 (−0.46 to −0.26) |
Control | 5.68 (6.00) [1292] | 9.03 (6.80) [1305] |
Intervention effect estimates were in favour of the intervention group within all subgroups (Table 8), and were statistically significant for all but the older male and older female groups, although there was no evidence that the intervention effect differed between any subgroups.
Subgroup | Trial group | Data summaries | Intervention effect estimate | |||||
---|---|---|---|---|---|---|---|---|
n | T0, mean (SD) | T1, mean (SD) | T1 – T0, mean (SD) | Estimatea (95% Cl); p-value | Effect sizeb (95% Cl) | Interaction p-value | ||
YC | Intervention | 602 | 6.84 (6.41) | 6.39 (6.25) | −0.45 (6.79) | −2.04 (−3.24 to −0.84); 0.001 | −0.36 (−0.58 to −0.15) | 0.085 |
Control | 598 | 5.35 (5.30) | 6.92 (6.58) | 1.58 (5.80) | ||||
OC | Intervention | 522 | 6.52 (6.98) | 10.74 (6.42) | 4.22 (7.18) | −1.22 (−2.43 to 0.00); 0.049 | −0.22 (−0.43 to 0.00) | |
Control | 566 | 5.73 (6.39) | 10.93 (6.37) | 5.20 (7.47) | ||||
Male | Intervention | 571 | 8.10 (7.14) | 9.47 (6.61) | 1.37 (7.56) | −1.60 (−2.80 to −0.39); 0.010 | −0.28 (−0.50 to −0.07) | 0.828 |
Control | 578 | 6.81 (6.27) | 9.81 (6.93) | 3.00 (7.26) | ||||
Female | Intervention | 553 | 5.23 (5.82) | 7.32 (6.61) | 2.08 (7.11) | −1.70 (−2.91 to −0.49); 0.006 | −0.30 (−0.52 to −0.09) | |
Control | 586 | 4.27 (5.12) | 7.94 (6.50) | 3.67 (6.52) | ||||
Deprived | Intervention | 595 | 7.68 (7.03) | 8.79 (6.80) | 1.11 (7.08) | −1.91 (−3.16 to −0.67); 0.003 | −0.34 (−0.56 to −0.12) | 0.361 |
Control | 593 | 5.92 (6.15) | 9.20 (6.84) | 3.29 (6.96) | ||||
Not deprived | Intervention | 529 | 5.57 (6.08) | 7.98 (6.55) | 2.41 (7.58) | −1.38 (−2.63 to −0.14); 0.029 | −0.24 (−0.46 to −0.02) | |
Control | 571 | 5.13 (5.52) | 8.52 (6.71) | 3.39 (6.85) | ||||
Younger male | Intervention | 293 | 7.86 (6.76) | 7.62 (6.53) | −0.23 (7.36) | −2.03 (−3.40 to −0.66); 0.004 | −0.36 (−0.60 to −0.12) | 0.132 |
Control | 285 | 6.15 (5.49) | 7.98 (6.82) | 1.83 (6.09) | ||||
Younger female | Intervention | 309 | 5.87 (5.91) | 5.21 (5.74) | −0.65 (6.20) | −2.08 (−3.43 to −0.74); 0.002 | −0.37 (−0.61 to −0.13) | |
Control | 313 | 4.61 (5.02) | 5.96 (6.21) | 1.35 (5.51) | ||||
Older male | Intervention | 278 | 8.35 (7.52) | 11.41 (6.12) | 3.06 (7.41) | −1.23 (−2.59 to 0.13); 0.076 | −0.22 (−0.46 to 0.02) | |
Control | 293 | 7.44 (6.90) | 11.58 (6.57) | 4.14 (8.08) | ||||
Older female | Intervention | 244 | 4.43 (5.62) | 9.98 (6.69) | 5.55 (6.68) | −1.19 (−2.60 to 0.22); 0.098 | −0.21 (−0.46 to 0.04) | |
Control | 273 | 3.89 (5.21) | 10.22 (6.08) | 6.33 (6.59) |
The Strengths and Difficulties Questionnaire-Total Difficulties Score at follow-up 2
Table 9 shows the analysis of the SDQ-TDS at T2. It should be noted that T2 is when three schools temporarily did not participate in the trial, and so, along with T4, is the least representative of our outcome time points. When including only pupils with data at baseline and T2 (baseline-adjusted mixed-effects regression), there was no evidence of an intervention effect. However, within a repeated measures analysis, using all available data from all four time points, the intervention effect estimate at T2 was statistically significant (p = 0.012), with lower SDQ-TDSs in the intervention group.
Data and trial group | Data summaries, mean (SD) [n] | Intervention effect estimates | ||||
---|---|---|---|---|---|---|
T0 | T2 | T2 – T0 | Estimate (95% Cl); p-value | ICC | Effect sizea (95% Cl) | |
Complete-case data | ||||||
Intervention | 6.39 (6.42) [869] | 8.36 (7.03) [869] | 1.96 (7.69) [869] | −0.14b (−1.37 to 1.10); 0.827 | 0.06 | −0.02 (−0.24 to 0.19) |
Control | 5.47 (5.77) [938] | 7.83 (6.48) [938] | 2.36 (6.56) [938] | |||
All available data | ||||||
Intervention | 6.99 (6.87) [1345] | 8.52 (7.03) [946] | N/A | −0.73c (−1.30 to −0.16); 0.012 | 0.02 | −0.15 (−0.27 to −0.03) |
Control | 5.68 (6.00) [1292] | 7.97 (6.50) [1034] |
Extending the primary analysis model to look at subgroups (Table 10) found no evidence of intervention effects for any particular groups of pupils.
Subgroup | Trial group | Data summaries | Intervention effect estimate | Interaction p-value | ||||
---|---|---|---|---|---|---|---|---|
n | T0, mean (SD) | T2, mean (SD) | T2 – T0, mean (SD) | Estimatea (95% Cl); p-value | Effect sizeb (95% Cl) | |||
YC | Intervention | 522 | 6.66 (6.38) | 6.83 (6.89) | 0.16 (6.96) | 0.26 (−1.04 to 1.58); 0.692 | 0.05 (−0.18 to 0.28) | 0.068 |
Control | 548 | 5.19 (5.19) | 5.63 (5.89) | 0.44 (5.25) | ||||
OC | Intervention | 347 | 5.99 (6.46) | 10.65 (6.61) | 4.67 (7.94) | −0.78 (−2.20 to 0.64); 0.279 | −0.14 (−0.39 to 0.11) | |
Control | 390 | 5.87 (6.48) | 10.93 (5.98) | 5.06 (7.23) | ||||
Male | Intervention | 449 | 7.68 (6.79) | 9.51 (7.03) | 1.83 (7.90) | −0.01 (−1.36 to 1.34); 0.987 | 0.00 (−0.24 to 0.24) | 0.643 |
Control | 461 | 6.61 (6.18) | 8.81 (6.68) | 2.21 (6.79) | ||||
Female | Intervention | 420 | 5.02 (5.69) | 7.12 (6.82) | 2.10 (7.46) | −0.26 (−1.61 to 1.08); 0.700 | −0.05 (−0.28 to 0.19) | |
Control | 477 | 4.38 (5.11) | 6.89 (6.13) | 2.51 (6.33) | ||||
Deprived | Intervention | 483 | 7.22 (6.64) | 9.38 (7.40) | 2.16 (7.80) | 0.49 (−0.86 to 1.83); 0.479 | 0.08 (−0.15 to 0.32) | 0.066 |
Control | 460 | 5.80 (6.09) | 7.88 (6.71) | 2.08 (6.43) | ||||
Not deprived | Intervention | 386 | 5.36 (5.97) | 7.08 (6.32) | 1.72 (7.54) | −0.73 (−2.08 to 0.62); 0.290 | −0.13 (−0.37 to 0.11) | |
Control | 478 | 5.15 (5.43) | 7.79 (6.25) | 2.64 (6.67) | ||||
Younger male | Intervention | 260 | 7.67 (6.75) | 8.62 (7.27) | 0.95 (7.52) | 0.65 (−0.83 to 2.14); 0.386 | 0.12 (−0.15 to 0.38) | 0.049 |
Control | 260 | 5.91 (5.37) | 6.85 (6.41) | 0.94 (5.53) | ||||
Younger female | Intervention | 262 | 5.66 (5.84) | 5.05 (5.99) | −0.61 (6.28) | −0.16 (−1.60 to 1.29); 0.834 | −0.03 (−0.28 to 0.23) | |
Control | 288 | 4.54 (4.94) | 4.53 (5.15) | −0.01 (4.96) | ||||
Older male | Intervention | 189 | 7.68 (6.87) | 10.73 (6.50) | 3.05 (8.27) | −0.98 (−2.58 to 0.61); 0.228 | −0.17 (−0.46 to 0.16) | |
Control | 201 | 7.50 (7.01) | 11.35 (6.16) | 3.85 (7.84) | ||||
Older female | Intervention | 158 | 3.96 (5.28) | 10.56 (6.76) | 6.60 (7.08) | −0.31 (−1.97 to 1.34); 0.711 | −0.06 (−0.35 to 0.24) | |
Control | 189 | 4.13 (5.36) | 10.49 (5.76) | 6.35 (6.28) |
Primary outcome at longer-term follow-up 4 (6 years post baseline)
Table 11 shows the analysis of the SDQ-TDS at T4. It should be noted that, during T4, three schools did not participate in the study; therefore, along with T2, T4 is the least representative of our outcome time points. T4, our longer-term follow-up (6 years post baseline), involved the OC only. When including only pupils with data at baseline and T4 (baseline-adjusted mixed-effects regression), there was no evidence of an intervention effect. However, within a repeated measures analysis, using all available data from all four time points, the intervention effect estimate at T4 was statistically significant (p = 0.044), with lower SDQ-TDSs in the intervention group.
Data summaries, mean (SD) [n] | Intervention effect estimates | |||||
---|---|---|---|---|---|---|
Data and trial group | T0 | T4 | T4 - T0 | Estimate (95% Cl); p-value | ICC | Effect sizea (95% Cl) |
Complete-case data | ||||||
Intervention | 5.01 (5.99) [258] | 13.59 (5.76) [258] | 8.58 (8.04) [258] | −0.651b (−2.31 to 1.01); < 0.44 | 0.06 | −012 (−0.43 to 0.19) |
Control | 4.38 (5.59) [284] | 14.08 (5.62) [248] | 9.61 (7.22) [284] | |||
All available data | ||||||
Intervention | 5.15 (6.02) [261] | 13.67 (5.76) [274] | N/A | −1.23c (−2.44 to − 0.04); < 0.04 | 0.03 | −0.23 (−0.45 to −0.01) |
Control | 4.45 (5.55) [300] | 14.15 (5.55) [304] |
Extending the primary analysis model to look at subgroups (Table 12) found no evidence of intervention effects for any particular groups of pupils.
Subgroup | Trial group | Data summaries | Intervention effect estimate | Interaction p-value | ||||
---|---|---|---|---|---|---|---|---|
n | T0, mean (SD) | T4, mean (SD) | T4 – T0, mean (SD) | Estimatea (95% CI); p-value | Effect sizeb (95% CI) | |||
Male | Intervention | 139 | 6.71 (6.62) | 12.96 (5.63) | 6.25 (8.47) | –0.79 (–2.67 to 1.10); 0.41 | –0.15 (–0.50 to –0.20) | 0.77 |
Control | 136 | 5.46 (5.84) | 13.57 (5.79) | 8.10 (7.34) | ||||
Female | Intervention | 119 | 4.95 (5.69) | 7.43 (6.56) | 2.48 (7.15) | –0.51 (–2.43 to 1.42); 0.61 | –0.09 (–0.45 to 0.26) | |
Control | 148 | 3.99 (4.86) | 7.05 (6.35) | 3.06 (7.12) | ||||
Deprived | Intervention | 125 | 5.57 (6.17) | 13.55 (5.66) | 7.98 (8.36) | –0.69 (–2.66 to 1.28); 0.49 | –0.13 (–0.49 to 0.24) | 0.85 |
Control | 125 | 5.66 (6.60) | 14.86 (5.75) | 9.20 (8.05) | ||||
Not deprived | Intervention | 162 | 4.42 (5.75) | 13.64 (5.90) | 9.22 (7.68) | –0.49 (–2039 to 1.41); 0.61 | –0.09 (–0.44 to 0.26) | |
Control | 499 | 3.42 (4.48) | 13.49 (5.47) | 10.07 (6.52) |
Primary outcome over time
Figure 5 provides an illustration of the primary outcome for each arm of the trial, showing the cohort by gender subgroups, at each applicable time point. The data shown, zero-centred for T0, are predicted values from repeated measures models fitted within each subgroup, with 95% CIs. In all subgroups, the SDQ-TDS increased between T0 and T1, but more so in the control arm, so that intervention arm scores were lower at T2 for both cohorts. In the YC, scores changed much less over time, with intervention and control arm scores being more similar. The change between T0 and T1 in the OC may be due to the change from teacher-completed to pupil-completed SDQs. The largest difference between the intervention and control arm for the OC was at T3. Boys started with higher SDQ-TDSs than girls from both cohorts, and this difference persisted over time; even at T4, 6 years post T0, the older boys showed a significant difference between the intervention and control arms.
Covariate effects on the Strengths and Difficulties Questionnaire-Total Difficulties Score
Table 13 shows the covariate effects from the baseline-adjusted mixed-effects regression models at each time point. Baseline SDQ-TDSs were strongly predictive of scores at each follow-up point. Older pupils had significantly higher scores and female pupils had significantly lower scores throughout the trial. Otherwise, none of the stratification variables used when assigning schools to trial arms was found to be significantly associated with the SDQ-TDS.
Covariate | T1 modela (95% CI) | T2 modela (95% CI) | T3 modela (specified outcome point) (95% CI) | T4 (longer term) (95% CI) |
---|---|---|---|---|
Baseline teacher-completed SDQ-TDS | ||||
Per 1-point increase | –0.39 (0.34 to 0.44)** | 0.41 (0.37 to 0.46)** | 0.43 (0.39 to 0.47)** | 0.15 (0.06 to 0.24)** |
Cohort (reference: YC) | ||||
OC | 5.11 (4.58 to 5.63)** | 4.44 (3.88 to 5.00)** | 4.07 (3.60 to 4.54)** | N/A |
Gender (reference: male) | ||||
Female | –0.99 (–1.53 to –0.45)** | –0.97 (–1.51 to –0.42)** | –0.63 (–1.12 to –0.15)* | 1.58 (0.62 to 2.54)** |
Percentage of pupils eligible for free school meals (reference: < 20) | ||||
≥ 20% | 0.43 (–083 to 1.69) | 0.74 (–0.81 to 2.29) | 0.81 (–0.65 to 2.27) | 0.94 (–1.10 to 2.96) |
School size (reference: < 205 pupils) | ||||
≥ 250 pupils | –0.77 (–2.93 to 1.38) | –1.37 (–4.51 to 1.76) | –0.06 (–2.60 to 2.48) | 1.16 (–2.7 to 5.02) |
Denomination (reference: non-denominational) | ||||
Roman Catholic | –1.08 (–2.59 to 0.43) | –0.03 (–1.88 to 1.81) | –0.70 (–2.50 to 1.05) | –1.23 (–3.69 to 1.23) |
Urban/rural (reference: large urban) | ||||
Other urban | 1.16 (–1.90 to 4.21) | 0.20 (–3.64 to 4.04) | 2.42 (–1.23 to 6.07) | 1.15 (–3.36 to 5.65) |
Non-urban | –0.42 (–1.93 to 1.10) | 1.02 (–3.38 to 5.43) | 3.6 (–0.54 to 7.78) | –1.21 (–6.52 to 4.09) |
LA (reference: LA1) | ||||
LA 3 | 1.50 (–1.95 to 4.95) | 0.39 (–1.89 to 2.59) | 2.83 (–1.58 to 2.04) | –0.04 (–5.24 to 5.16) |
LA 2 | –0.42 (–1.93 to 1.10) | 0.39 (–1.54 to 2.32) | 0.23 (–1.58 to 2.04) | 0.94 (–1.48 to 3.36) |
Number of P1 pupils (reference: < 30) | ||||
≥ 30 | 0.06 (–1.67 to 1.79) | 0.35 (–1.89 to 2.59) | 0.99 (–1.05 to 3.04) | –3.90 (–6.70 to –1.11)* |
Number of P5 pupils (reference: < 30) | ||||
≥ 30 | 0.52 (–1.44 to 2.49) | 1.52 (–1.30 to 4.33) | 0.73 (–1.59 to 3.05) | 0.08 (–3.39 to 3.54) |
Percentage of pupils from ethnic minority groups (reference: 0–9.9) | ||||
≥ 10 | 0.80 (–0.93 to 2.53) | 0.32 (–1.84 to 2.48) | 0.07 (–1.99 to 2.12) | 0.67 (–2.83 to 2.96) |
Does having a Strengths and Difficulties Questionnaire-Total Difficulties Score in the normal range at baseline moderate the intervention?
Table 14 summarises the SDQ-TDSs over time in each arm of the trial (see Appendix 3, Table 26), stratified by whether the baseline SDQ-TDS was below the threshold for a ‘normal’ score (≤ 15) or above this threshold (i.e. ‘borderline’ or ‘abnormal’). Also shown are the results of fitting a repeated measure regression model, allowing for different intervention effects according to the baseline SDQ-TDS category. There was a significant interaction (p = 0.008), indicating that the patterns of intervention effects over time are different for those with normal baseline SDQ-TDSs than for those with non-normal scores.
Time point | Trial group, mean (SD) [n] | Intervention effect | ||
---|---|---|---|---|
Intervention | Control | Estimate (95% CI); p-value | Effect size (95% CI) | |
Normal baseline SDQ-TDS (≤ 15) | ||||
T0 | 4.93 (4.46) [1166] | 4.38 (4.20) [1184] | ||
T1 | 7.79 (6.36) [988] | 8.26 (6.39) [1075] | –1.15 (–1.68 to –0.63); < 0.001 | –0.27 (–0.39 to –0.14) |
T2 | 7.89 (6.78) [775] | 7.31 (6.14) [874] | –0.09 (–0.66 to 0.48); 0.754 | –0.02 (–0.15 to 0.11) |
T3 | 7.88 (6.39) [836] | 7.95 (6.86) [930] | –0.84 (–1.40 to –0.28); 0.003 | –0.19 (–0.32 to –0.07) |
T4 | 13.61 (5.80) [237] | 13.98 (5.61) [270] | –0.79 (–1.70 to 0.11); 0.087 | –0.18 (–0.39 to 0.03) |
Borderline/abnormal baseline SDQ-TDS (> 15) | ||||
T0 | 20.44 (4.02) [179] | 19.99 (3.80) [108] | ||
T1 | 12.92 (7.31) [136] | 16.20 (7.09) [89] | –3.84 (–5.48 to –2.20); < 0.001 | –0.88 (–1.26 to –0.50) |
T2 | 12.22 (7.87) [94] | 15.03 (6.68) [64] | –2.46 (–4.32 to –0.59); 0.010 | –0.56 (–0.99 to –0.14) |
T3 | 13.23 (7.42) [104] | 13.77 (6.02) [65] | –0.91 (–2.74 to 0.93); 0.333 | –0.21 (–0.63 to 0.21) |
T4 | 13.38 (5.44) [21] | 15.93 (5.88) [14] | –2.30 (–5.75 to 1.15); 0.191 | –0.53 (–1.32 to 0.26) |
For those with normal scores at baseline, average SDQ-TDSs increased in both groups at T1 and remained roughly stable thereafter. Scores in the intervention arm were lower at T1 and T3, but not at T2, similar to the primary analysis. For those with borderline or abnormal SDQ-TDSs at baseline, scores dropped sharply between T0 and T1, but more so in the intervention arm; scores in the intervention arm remained stable thereafter, but in the control arm continued to decline, so that scores were not significantly different in the two arms of the trial by T3. However, the estimated difference between groups at this time point was larger than the corresponding comparison for pupils with a normal SDQ-TDS at baseline, so the lack of statistical significance may reflect a lack of power owing to the smaller numbers with non-normal scores at baseline.
Parent-reported Strengths and Difficulties Questionnaire total difficulties
Parental questionnaires were completed by approximately one-third of parents, at T0, T1 and T2. These are summarised in Table 15. There were no significant intervention effects at either follow-up for the SDQ-TDS.
Time point | Trial group, mean (SD) | Intervention effect | ||
---|---|---|---|---|
Intervention | Control | Estimate (95% CI); p-value | Effect size (95% CI) | |
Pupils with data at T0 and T1 | n = 220 | n = 207 | ||
T0 | 6.61 (5.08) | 6.72 (5.50) | ||
T1 | 7.18 (5.82) | 6.79 (5.66) | 0.21 (–0.54 to 0.95); 0.587 | 0.06 (–0.15 to 0.27) |
Change (T1 – T0) | 0.57 (3.53) | 0.07 (3.81) | ||
Pupils with data at T0 and T3 | n = 184 | n = 158 | ||
T0 | 5.99 (4.69) | 6.25 (4.88) | ||
T3 | 7.02 (6.39) | 6.30 (5.50) | 0.47 (–0.73 to 1.66); 0.445 | 0.11 (–0.17 to 0.38) |
Change (T3 – T0) | 1.03 (4.47) | 0.05 (4.39) |
Secondary outcomes
Strengths and Difficulties Questionnaire subscales
Table 16 reports the intervention effect estimates and 95% CIs for each of the five SDQ subscales, for all pupils, and in subgroups defined by age (cohort), gender, deprivation and baseline SDQ-TDS (see Appendix 3, Tables 26–31 for more detail).
Time point | All (95% Cl) | Cohort (95% Cl) | Gender (95% Cl) | Deprivation (95% Cl) | SDQ score at baseline (95% Cl) | ||||
---|---|---|---|---|---|---|---|---|---|
Younger | Older | Male | Female | Deprived | Not deprived | Normal | Not normal | ||
Emotional symptoms (negative effect estimate favours intervention group) | |||||||||
T1 | −0.64 (−0.84 to −0.45) | −0.84 (−1.12 to −0.56) | −0.36 (−0.65 to −0.07) | −0.65 (−0.94 to −0.37) | −0.65 (−0.95 to −0.36) | −0.70 (−0.98 to −0.42) | −0.60 (−0.91 to −0.30) | −0.46 (−0.67 to −0.26) | −0.71 (−1.58 to 0.17) |
T2 | −0.28 (−0.51 to −0.06) | −0.34 (−0.63 to −0.06) | −0.08 (−0.41 to 0.25) | −0.22 (−0.53 to 0.09) | −0.31 (−0.63 to 0.01) | 0.09 (−0.22 to 0.39) | −0.67 (−1.0 to −0.34) | −0.09 (−0.31 to 0.13) | −0.49 (−1.44 to 0.47) |
T3 | −0.48 (−0.71 to −0.26) | −0.45 (−0.73 to −0.16) | −0.46 (−0.77 to −0.15) | −0.55 (−0.85 to −0.24) | −0.39 (−0.70 to −0.08) | −0.32 (−0.32 to −0.02) | −0.63 (−0.95 to −0.32) | −0.26 (−0.47 to −0.04) | −0.87 (−1.82 to 0.07) |
T4 | −0.32 (−0.81 to 0.17) | N/A | N/A | ||||||
Interaction | p = 0.056 | p = 0.781 | p = 0.001 | p = 0.635 | |||||
Conduct problems (negative effect estimate favours intervention group) | |||||||||
T1 | −0.22 (−0.36 to −0.08) | −0.29 (−0.49 to −0.10) | −0.13 (−0.34 to 0.07) | −0.34 (−0.54 to −0.14) | −0.15 (−0.35 to 0.06) | −0.35 (−0.55 to −0.15) | −0.13 (−0.34 to 0.09) | −0.15 (−0.29 to −0.01) | −0.84 (−1.32 to −0.35) |
T2 | −0.06 (−0.22 to 0.01) | 0.01 (−0.19 to 0.21) | −0.12 (−0.25 to 0.12) | −0.11 (−0.32 to 0.11) | −0.03 (−0.25 to 0.19) | −0.02 (−0.24 to 0.19) | −0.13 (−0.36 to 0.10) | −0.02 (−0.18 to 0.13) | −0.33 (−0.88 to 0.22) |
T3 | −0.31 (−0.47 to −0.15) | −0.07 (−0.27 to 0.14) | −0.58 (−0.80 to −0.36) | −0.51 (−0.72 to −0.29) | −0.13 (−0.34 to 0.09) | −0.49 (−0.70 to −0.28) | −0.13 (−0.35 to 0.09) | −0.27 (−0.42 to −0.12) | −0.24 (0.78 to 0.30) |
T4 | −0.01 (−0.35 to 0.33) | N/A | N/A | ||||||
Interaction | p < 0.001 | p = 0.085 | p = 0.020 | p = 0.031 | |||||
Hyperactivity (negative effect estimate favours intervention group) | |||||||||
T1 | −0.45 (−0.66 to −0.24) | −0.59 (−0.90 to −0.28) | −0.30 (−0.62 to 0.02) | −0.36 (−0.67 to −0.04) | −0.59 (−0.91 to −0.27) | −0.47 (−0.78 to −0.16) | −0.48 (−0.81 to −0.15) | −0.46 (−0.70 to −0.23) | −0.13 (−0.61 to 0.35) |
T2 | −0.20 (−0.45 to 0.51) | −0.01 (−0.32 to 0.31) | −0.44 (−0.80 to 0.07) | −0.22 (−0.56 to 0.12) | −0.22 (−0.56 to 0.13) | −0.11 (−0.44 to 0.22) | −0.34 (−0.70 to 0.02) | −0.07 (−0.32 to 0.18) | −0.48 (−1.01 to 0.05) |
T3 | −0.37 (−0.62 to −0.11) | 0.00 (−0.32 to 0.32) | −0.84 (−1.18 to −0.50) | −0.40 (−0.73 to −0.06) | −0.39 (−0.72 to −0.05) | −0.22 (−0.55 to 0.11) | −0.57 (−0.92 to −0.23) | −0.26 (−0.51 to −0.01) | −0.44 (−0.96 to 0.09) |
T4 | −0.58 (−1.10 to −0.06) | N/A | N/A | ||||||
Interaction | p < 0.001 | p = 0.683 | p = 0.412 | p = 0.091 | |||||
Peer problems (negative effect estimate favours intervention group) | |||||||||
T1 | −0.48 (−0.63 to −0.33) | −0.58 (−0.80 to −0.36) | −0.32 (−0.55 to −0.09) | −0.60 (−0.82 to −0.37) | −0.33 (−0.56 to −0.11) | −0.67 (−0.88 to −0.45) | −0.24 (−0.47 to −0.01) | −0.33 (−0.49 to −0.17) | 0.03 (−0.44 to 0.49) |
T2 | −0.23 (−0.41 to −0.05) | −0.10 (−0.32 to 0.13) | −0.25 (−0.51 to 0.01) | −0.27 (−0.51 to −0.04) | −0.06 (−0.31 to 0.18) | −0.18 (−0.41 to 0.05) | −0.15 (−0.40 to 0.10) | 0.03 (−0.14 to 0.20) | −0.40 (−0.92 to 0.12) |
T3 | −0.19 (−0.37 to −0.02) | −0.05 (−0.28 to 0.17) | −0.31 (−0.55 to −0.06) | −0.34 (−0.57 to −0.10) | −0.01 (−0.45 to 0.23) | −0.41 (−0.65 to −0.18) | 0.09 (−0.15 to 0.33) | −0.05 (−0.22 to 0.11) | 0.39 (−0.12 to 0.90) |
T4 | −0.31 (−0.66 to 0.05) | N/A | N/A | ||||||
Interaction | p = 0.018 | p = 0.208 | p = 0.003 | p = 0.011 | |||||
Prosocial behaviour (positive effect estimate favours intervention group) | |||||||||
T1 | 0.23 (0.04 to 0.43) | 0.28 (0.01 to 0.56) | 0.21 (−0.08 to 0.50) | 0.24 (−0.05 to 0.52) | 0.22 (−0.07 to 0.51) | 0.35 (0.07 to 0.63) | 0.09 (−0.20 to 0.39) | 0.05 (−0.15 to 0.26) | 0.26 (−0.20 to 0.71) |
T2 | 0.08 (−0.14 to 0.31) | −0.18 (−0.47 to 0.11) | 0.31 (−0.02 to 0.64) | 0.01 (−0.30 to 0.31) | 0.06 (−0.26 to 0.37) | 0.08 (−0.22 to 0.38) | −0.02 (−0.34 to 0.30) | −0.21 (−0.43 to 0.02) | 0.53 (0.0 to 1.03) |
T3 | 0.22 (−0.00 to 0.45) | −0.14 (−0.43 to 0.15) | 0.75 (0.44 to 1.06) | 0.45 (0.15 to 0.74) | 0.08 (−0.23 to 0.38) | 0.38 (0.08 to 0.68) | 0.13 (−0.18 to 0.44) | −0.00 (−0.22 to 0.22) | 0.51 (0.03 to 1.00) |
T4 | 0.56 (0.13 to 0.10) | N/A | N/A | ||||||
Interaction | p < 0.001 | p = 0.247 | p = 0.571 | p = 0.045 |
Overall, and in most subgroups, the mean emotional symptoms scores were lower in the intervention group at all follow-up time points. The between-group difference failed to reach statistical significance at T2 for older pupils, female pupils, those living in more deprived areas and those with a SDQ-TDS in the normal range. For those with borderline or abnormal SDQ-TDSs, intervention effect estimates were large, and favoured the intervention group, but were not statistically significant. However, the only subgroup analysis showing differential intervention effects was deprivation, with generally larger effects seen among those living in less deprived areas.
Conduct problem scores were significantly lower for the intervention arm at T1 and T3. Peer problem scores were significantly lower for most of the intervention subgroups. Hyperactivity subscale results showed a significant reduction only for the OC at T2.
The only strength subscale of the SDQ is prosocial behaviour. At T1 the intervention group showed significantly better prosocial behaviour scale scores; at other time points a range of subgroups benefited (see Appendix 3, Tables 26–31).
The analyses included 135 between-group comparisons (see Table 16). We would have expected 20% to be significant by chance alone; we found 63 (47%, more than double what was expected by chance) of these comparisons to be significant at a 5% significance level, all in favour of the intervention arm of the trial. This adds some weight to these results.
Pupils’ social and emotional well-being domain scores
In the OC, pupils self-completed questionnaires at each wave of data collection, from which several measures of SEW were derived. Intervention effect estimates for these SEW domain scores are presented in Table 17 (see also Appendix 3, Tables 33–37 and Table 45). Between-group differences were observed in favour of the intervention group for the self-management and responsible decision-making domains. For self-management, these differences were observed consistently for boys and girls, and for pupils living in more deprived areas and pupils living in less deprived areas; for responsible decision-making, however, benefits in the intervention group appeared to be stronger for boys than for girls, and appeared to be specific to those living in less deprived areas.
Time point | All (95% CI) | Gender | Deprivation | ||
---|---|---|---|---|---|
Male (95% CI) | Female (95% CI) | Deprived (95% CI) | Not deprived (95% CI) | ||
Self-esteem (positive effect estimate favours intervention group) | |||||
T1 | 0.04 (–0.07 to 0.15) | 0.08 (–0.07 to 0.23) | 0.00 (–0.15 to 0.15) | –0.05 (–0.20 to 0.10) | 0.14 (–0.02 to 0.29) |
T2 | 0.11 (–0.01 to 0.24) | 0.17 (0.01 to 0.34) | 0.04 (–0.13 to 0.22) | 0.03 (–0.14 to 0.20) | 0.20 (0.03 to 0.38) |
T3 | 0.07 (–0.04 to 0.19) | 0.01 (–0.13 to 0.15) | 0.10 (–0.04 to 0.25) | –0.07 (–0.21 to 0.08) | 0.18 (0.04 to 0.33) |
T4 | 0.10 (–0.07 to 0.27) | ||||
Interaction | p = 0.190 | p = 0.115 | |||
Emotional regulation (positive effect estimate favours intervention group) | |||||
T1 | 0.05 (–0.07 to 0.17) | 0.03 (–0.13 to 0.20) | 0.07 (–0.11 to 0.24) | 0.04 (–0.13 to 0.21) | 0.06 (–0.12 to 0.23) |
T2 | 0.07 (–0.07 to 0.20) | 0.06 (–0.13 to 0.25) | 0.04 (–0.15 to 0.24) | –0.01 (–0.19 to 0.18) | 0.12 (–0.08 to 0.32) |
T3 | 0.16 (0.03 to 0.28) | 0.14 (–0.02 to 0.30) | 0.02 (–0.15 to 0.19) | 0.05 (–0.11 to 0.21) | 0.11 (–0.06 to 0.28) |
T4 | 0.15 (–0.03 to 0.33) | ||||
Interaction | p = 0.545 | p = 0.789 | |||
Self-awareness (positive effect estimate favours intervention group) | |||||
T1 | 0.02 (–0.06 to 0.10) | 0.03 (–0.08 to 0.14) | 0.00 (–0.11 to 0.12) | 0.04 (–0.13 to 0.21) | 0.06 (–0.12 to 0.23) |
T2 | –0.01 (–0.10 to 0.08) | 0.04 (–0.09 to 0.16) | –0.07 (–0.20 to 0.07) | –0.01 (–0.19 to 0.18) | 0.12 (–0.08 to 0.32) |
T3 | 0.05 (–0.04 to 0.13) | 0.07 (–0.04 to 0.18) | –0.04 (–0.15 to 0.07) | 0.05 (–0.11 to 0.21) | 0.11 (–0.06 to 0.28) |
T4 | 0.06 (–0.07 to 0.19) | ||||
Interaction | 0.453 | 0.789 | |||
Social awareness (positive effect estimate favours intervention group) | |||||
T1 | –0.02 (–0.09 to 0.06) | –0.03 (–0.14 to 0.07) | –0.00 (–0.11 to 0.11) | –0.05 (–0.16 to 0.05) | 0.02 (–0.09 to 0.12) |
T2 | 0.03 (–0.06 to 0.11) | 0.03 (–0.08 to 0.15) | 0.02 (–0.10 to 0.14) | –0.04 (–0.16 to 0.08) | 0.10 (–0.02 to 0.23) |
T3 | 0.09 (0.01 to 0.17) | 0.07 (–0.03 to 0.16) | 0.03 (–0.07 to 0.13) | 0.02 (–0.08 to 0.12) | 0.07 (–0.03 to 0.17) |
T4 | 0.14 (0.03 to 0.25) | ||||
Interaction | p = 0.812 | p = 0.416 | |||
Self-management (positive effect estimate favours intervention group) | |||||
T1 | 0.07 (0.01 to 0.14) | 0.06 (–0.04 to 0.15) | 0.08 (–0.02 to 0.18) | 0.10 (0.01 to 0.19) | 0.04 (–0.06 to 0.13) |
T2 | 0.12 (0.05 to 0.20) | 0.16 (0.05 to 0.26) | 0.06 (–0.05 to 0.17) | 0.15 (0.05 to 0.26) | 0.07 (–0.04 to 0.19) |
T3 | 0.21 (0.13 to 0.28) | 0.18 (0.09 to 0.27) | 0.11 (0.01 to 0.20) | 0.13 (0.04 to 0.22) | 0.16 (0.07 to 0.25) |
T4 | 0.18 (0.72 to 0.28) | ||||
Interaction | p = 0.320 | p = 0.340 | |||
Responsible decision-making (positive effect estimate favours intervention group) | |||||
T1 | 0.05 (–0.02 to 0.12) | 0.02 (–0.08 to 0.11) | 0.09 (–0.01 to 0.18) | –0.02 (–0.12 to 0.07) | 0.12 (0.03 to 0.22) |
T2 | 0.12 (0.04 to 0.19) | 0.15 (0.04 to 0.25) | 0.08 (–0.04 to 0.19) | 0.07 (–0.04 to 0.17) | 0.17 (0.05 to 0.28) |
T3 | 0.15 (0.08 to 0.22) | 0.17 (0.08 to 0.26) | 0.01 (–0.08 to 0.10) | 0.01 (–0.08 to 0.10) | 0.18 (0.08 to 0.27) |
T4 | 0.12 (0.01 to 0.22) | ||||
Interaction | p = 0.004 | p = 0.065 | |||
Materialism (negative effect estimate favours intervention group) | |||||
T1 | 0.02 (–0.08 to 0.12) | 0.05 (–0.08 to 0.19) | –0.02 (–0.16 to 0.13) | 0.07 (–0.07 to 0.21) | –0.02 (–0.16 to 0.12) |
T2 | 0.02 (–0.09 to 0.13) | 0.04 (–0.12 to 0.19) | –0.00 (–0.17 to 0.16) | 0.11 (–0.04 to 0.27) | –0.06 (0.23 to 0.10) |
T3 | –0.11 (–0.22 to 0.00) | –0.10 (–0.23 to 0.03) | –0.02 (–0.16 to 0.11) | 0.04 (–0.09 to 0.17) | –0.15 (–0.29 to –0.02) |
T4 | –0.12 (–0.27 to 0.03) | ||||
Interaction | p = 0.437 | p = 0.209 |
Overall, statistically significant improvements were found for the intervention group at T3 in terms of the emotional regulation, social awareness and materialism domain scores. No between-group differences were observed in the overall analysis of the self-esteem and self-awareness domain scores.
Of the 105 statistical tests represented in Table 17, we found 24 (23%) to be statistically significant at a 5% significance level in favour of the intervention group, with none in favour of the control group.
Domains relating to pupils’ social contexts
Pupil-completed questionnaires also included questions relating to their social context, from which a range of domain scores were derived. Intervention effect estimates for these scores are reported in Table 18 (see also Appendix 3, Tables 38–44). In overall analyses, significant differences in favour of the intervention group were observed at T3 for family relationships, school relationships (relating both to the whole school and to within classes), school climate and participation in antisocial behaviour. A benefit at T2 was also seen for the school relationships (class) domain. No other between-group differences were observed in overall analyses. Within subgroups defined by gender and deprivation, these results were generally consistent, although there was some evidence of greater intervention effects for pupils living in less deprived areas, in particular for school relationships (class) and liking school. There were statistically significant differences in favour of the control group for experience of antisocial behaviour for female pupils at T1 and T2, and for participation in antisocial behaviour at T2 for those living in more deprived areas; however, there were no differences between randomised groups in overall analyses, and no evidence of subgroup-by-intervention interactions for these measures.
Time point | All (95% CI) | Gender | Deprivation | ||
---|---|---|---|---|---|
Male (95% CI) | Female (95% CI) | Deprived (95% CI) | Not deprived (95% CI) | ||
Family relationships (positive effect estimate favours intervention group | |||||
T1 | 0.04 (–0.04 to 0.12 | 0.02 (–0.08 to 0.12) | 0.05 (–0.05 to 0.16) | 0.00 (–0.10 to 0.10) | 0.06 (–0.04 to 0.17) |
T2 | 0.08 (0.00 to 0.17) | 0.09 (–0.03 to 0.20 | 0.09 (–0.03 to 0.20) | 0.09 (–0.03 to 0.20 | 0.08 (–0.04 to 0.19 |
T3 | 0.12 (0.04 to 0.19) | 0.11 (0.02 to 0.21) | 0.07 (–0.03 to 0.17) | 0.06 (–0.04 to 0.16) | 0.12 (0.02 to 0.21) |
T4 | 0.01 (–0.10 to 0.13) | ||||
Interaction | p = 0.770 | p = 0.706 | |||
Happy friendships (positive effect estimate favours intervention group) | |||||
T1 | –0.01 (–0.09 to 0.07) | –0.01 (–0.12 to 0.10) | –0.01 (–0.13 to 0.10) | –0.08 (–0.19 to 0.03) | 0.07 (–0.05 to 0.18) |
T2 | 0.02 (–0.07 to 0.11 | 0.04 (–0.08 to 0.17) | –0.01 (–0.14 to 0.12) | –0.06 (–0.18 to 0.07) | 0.10 (–0.03 to 0.23) |
T3 | 0.07 (–0.02 to 0.15) | 0.07 (–0.03 to 0.17) | –0.01 (–0.12 to 0.09) | 0.03 (–0.08 to 0.13) | 0.03 (–0.08 to 0.14) |
T4 | 0.03 (–0.09 to 0.14) | ||||
Interaction | p = 0.632 | p = 0.075 | |||
School relationships: whole (positive effect estimate favours intervention group) | |||||
T1 | 0.04 (–0.06 to 0.13) | 0.00 (–0.13 to 0.13) | 0.07 (–0.07 to 0.21) | –0.05 (–0.19 to 0.08) | 0.13 (–0.01 to 0.26) |
T2 | 0.08 (–0.03 to 0.19) | –0.01 (–0.16 to 0.14) | 0.17 (0.01 to 0.32) | –0.02 (–0.17 to 0.13) | 0.17 (0.01 to 0.33) |
T3 | 0.23 (0.13 to 0.33) | 0.19 (0.07 to 0.32) | 0.11 (–0.02 to 0.24) | 0.15 (0.02 to 0.27) | 0.15 (0.02 to 0.28) |
T4 | 0.14 (–0.00 to 0.29) | ||||
Interaction | p = 0.078 | p = 0.082 | |||
School relationships: class (positive effect estimate favours intervention group) | |||||
T1 | 0.07 (–0.03 to 0.17) | 0.02 (–0.11 to 0.16) | 0.12 (–0.02 to 0.26) | –0.09 (–0.23 to 0.05) | 0.24 (0.10 to 0.38) |
T2 | 0.12 (0.00 to 0.24) | 0.11 (–0.05 to 0.28) | 0.13 (–0.04 to 0.30) | 0.16 (–0.01 to 0.32) | 0.09 (–0.09 to 0.26) |
T3 | 0.19 (0.08 to 0.31) | 0.12 (–0.01 to 0.25) | 0.02 (–0.12 to 0.15) | –0.02 (–0.15 to 0.12) | 0.16 (0.03 to 0.30) |
T4 | 0.12 (–0.02 to 0.26) | ||||
Interaction | p = 0.215 | p = 0.001 | |||
Liking school (positive effect estimate favours intervention group) | |||||
T1 | 0.05 (–0.07 to 0.16) | –0.02 (–0.17 to 0.14) | 0.12 (–0.04 to 0.28) | –0.13 (–0.28 to 0.03) | 0.24 (0.08 to 0.40) |
T2 | –0.03 (–0.16 to 0.10) | –0.05 (–0.22 to 0.13) | 0.00 (–0.18 to 0.19) | –0.16 (–0.34 to 0.01) | 0.14 (–0.05 to 0.32) |
T3 | 0.14 (0.02 to 0.26) | 0.14 (–0.01 to 0.29) | 0.09 (–0.07 to 0.24) | –0.05 (–0.20 to 0.10) | 0.28 (0.13 to 0.43) |
T4 | 0.17 (–0.01 to 0.34) | ||||
Interaction | p = 0.379 | p = 0.005 | |||
School climate (positive effect estimate favours intervention group) | |||||
T1 | –0.01 (–0.09 to 0.07) | –0.07 (–0.18 to 0.04) | 0.06 (–0.06 to 0.17) | –0.09 (–0.20 to 0.02) | 0.08 (–0.03 to 0.19) |
T2 | 0.07 (–0.02 to 0.16) | 0.04 (–0.08 to 0.17) | 0.08 (–0.04 to 0.21) | –0.03 (–0.15 to 0.09) | 0.17 (0.04 to 0.30) |
T3 | 0.19 (0.10 to 0.27) | 0.14 (0.04 to 0.24) | 0.11 (0.00 to 0.21) | 0.07 (–0.04 to 0.17) | 0.18 (0.08 to 0.28) |
T4 | 0.13 (0.00 to 0.25) | ||||
Interaction | p = 0.178 | p = 0.085 | |||
Experience of antisocial behaviour (mixed-effects estimate for intervention group) | |||||
T1 | 0.04 (–0.05 to 0.13) | –0.04 (–0.16 to 0.09) | 0.13 (0.00 to 0.26) | –0.01 (–0.14 to 0.11) | 0.11 (–0.02 to 0.24) |
T2 | 0.08 (–0.02 to 0.18) | –0.01 (–0.16 to 0.13) | 0.17 (0.02 to 0.32) | 0.03 (–0.11 to 0.17) | 0.13 (–0.02 to 0.28) |
T3 | 0.01 (–0.08 to 0.12) | –0.05 (–0.17 to 0.07) | 0.11 (–0.02 to 0.23) | –0.03 (–0.15 to 0.09) | 0.09 (–0.04 to 0.21) |
T4 | –0.04 (–0.17 to 0.09 | ||||
Interaction | p = 0.173 | p = 0.513 | |||
Participation in antisocial behaviour (positive effect estimate favours intervention group) | |||||
T1 | 0.03 (–0.03 to 0.08) | –0.00 (–0.08 to 0.07) | 0.06 (–0.02 to 0.14) | 0.07 (–0.00 to 0.15) | –0.02 (–0.10 to 0.06) |
T2 | 0.04 (–0.03 to 0.10) | 0.02 (–0.07 to 0.11) | 0.07 (–0.02 to 0.16) | 0.10 (0.02 to 0.19) | –0.02 (–0.11 to 0.07) |
T3 | –0.08 (–0.14 to –0.02) | –0.10 (–0.18 to 0.03) | 0.02 (–0.05 to 0.10) | –0.02 (–0.10 to 0.05) | –0.06 (–0.13 to 0.02) |
T4 | –0.00 (–0.08 to 0.08) | ||||
Interaction | p = 0.128 | p = 0.175 |
Of the 120 between-group comparisons reported (see Table 18), 22 (18%) were statistically significant at a 5% significance level in favour of the intervention group, and three (2.5%) were significant in favour of the control group.
Pupil-reported health risk behaviours at follow-ups 3 and 4
We undertook additional exploratory analysis around pupils’ health risk behaviours when the OC reached 12–13 years (T3), and at age 15–16 years (T4). Please note that, at T3, the pupils were still relatively young regarding alcohol, tobacco, e-cigarettes and cannabis use. A further limitation is that, at T4, three secondary schools dropped out, which weakens our ability to interpret the data. For these results, see Appendix 3, Tables 46 and 47. At T3 we found significantly less use of e-cigarettes among the intervention pupils; at T4 this outcome was not significant. At T4 we found significantly less reporting of alcohol use. For all outcomes, the health risk behaviours reported were lower for the intervention arm.
School staff-reported domains
Table 19 reports the intervention effect estimates for a range of domain scores derived from teacher-completed questionnaires at each time point (see Appendix 3, Table 48, for more detail). The analyses are cross-sectional, given the relatively high turnover of staff in trial schools. Furthermore, although HTs supported participation, staff questionnaire completion was voluntary; participation rates were 53% at T1, 46% at T2 and 44% at T3.
Domain | T1 (95% CI) | T2 (95% CI) | T3 (95% CI) |
---|---|---|---|
Pupil behaviour | 0.01 (–0.09 to 0.11) | 0.07 (–0.04 to 0.18) | 0.09 (–0.02 to 0.21) |
Pupil confidence | 0.02 (–0.08 to 0.11) | 0.03 (–0.08 to 0.13) | 0.09 (–0.01 to 0.19) |
Pupil engagement | 0.06 (–0.02 to 0.14) | 0.13 (0.04 to 0.22) | 0.14 (0.05 to 0.23) |
Pupil–pupil relationships | 0.07 (–0.04 to 0.17) | 0.11 (–0.00 to 0.22) | 0.10 (–0.01 to 0.21) |
Staff–pupil relationships | 0.06 (–0.03 to 0.15) | 0.11 (0.01 to 0.21) | 0.14 (0.04 to 0.24) |
Staff–staff relationships | –0.00 (–0.08 to 0.09) | 0.07 (–0.02 to 0.17) | 0.13 (0.03 to 0.22) |
Perceptions of management | 0.07 (–0.20 to 0.15) | 0.13 (0.04 to 0.22) | 0.10 (0.01 to 0.19) |
Staff support | 0.07 (–0.01 to 0.15) | 0.14 (0.06 to 0.22) | 0.12 (0.04 to 0.20) |
School ethos | 0.06 (–0.02 to 0.14) | 0.16 (0.07 to 0.25) | 0.16 (0.08 to 0.25) |
School support for SEW | –0.01 (–0.09 to 0.07) | 0.10 (0.01 to 0.18) | 0.03 (–0.05 to 0.12) |
Valued member of staff | 0.01 (–0.10 to 0.11) | 0.07 (–0.04 to 0.19) | 0.11 (–0.01 to 0.23) |
Training opportunities | –0.02 (–0.13 to 0.08) | 0.06 (–0.05 to 0.18) | 0.76 (–0.04 to 0.19) |
Self-efficacy | –0.04 (–0.12 to 0.04) | 0.07 (–0.02 to 0.15) | –0.04 (–0.12 to 0.05) |
No statistically significant differences between the arms of the trial were found at T1. However, at T2 and T3, a number of between-group differences emerged in favour of the intervention arm. At T2, staff in intervention schools reported significantly better perceptions of management, staff support, school ethos, school support for SEW, pupil engagement and staff–pupil relationships. At T3, these benefits were retained, except for school support for SEW, and an improvement in staff–staff relationships became apparent. Caution is required when interpreting these results given the relatively low response rates.
Summary/conclusion
The intention-to-treat analysis showed significantly positive results for intervention pupils in terms of the primary outcome (SDQ-TDS) at the main outcome point (T3), which was 4 years after the baseline. Thus, the results of the trial seem to be robust over time, including over the OC’s transition to secondary school. The intention-to-treat analysis also showed significant benefits at T1 and T2. The results were particularly notable for the OC, particularly the older boys. The intervention shows a marked reduction in SDQ-TDS (by 3 points at T1) for the pupils whose SDQ-TDS was not in the normal range at baseline. This is a greater difference than that observed between children living in affluent and children living in deprived areas. 63 In terms of the SDQ subscales, the intervention worked across all five subscales, but was particularly beneficial for emotional symptoms. Fewer positive results were found in regard to our measures of other social and emotional domains: self-management and responsible decision-making showed the strongest benefit. In terms of pupils’ social contexts, positive results predominantly emerged at T2 and T3. This triangulates with the findings in regard to staff-reported data, for which all significant results emerged at T2 and T3. Together this suggests that changing social context (ethos and culture and relationships) takes time to bed in and show benefit. Chapter 7 starts by summarising these results and discusses the results in the context of other literature.
Chapter 5 Economic evaluation
Introduction
The SEED intervention is a schools-based programme that adopts a comprehensive whole-school approach with the aim of improving pupils’ SEW; it has been designed based on recommendations from NICE public health guidance. 107 Economic evaluation in school settings is a relatively novel area despite the need for consideration of optimal ‘healthcare’ resource allocation in the educational sector.
Evidence from economic evaluations of school-based interventions aiming to improve children’s emotional and social well-being is sparse. Recent (2018) systematic reviews of school based interventions40,107 identified 25 cost–utility analyses, of which only two focused on mental health and well-being: one used a dancing intervention to improve depression108 and the other implemented a nutrition and physical activity intervention for an eating disorder. 109 The remaining studies primarily dealt with physical activity, nutrition and obesity prevention. Following this, a more recently published clustered RCT reported the economic evaluation of the class-based ROE programme for improving prosocial behaviour and decreasing difficult behaviour. 40 To our knowledge, the economic evaluation in the SEED RCT is the first economic evaluation of a whole-school-based complex intervention to improve overall children’s SEW.
One of the key challenges for the evaluation of children’s interventions is the measurement and valuation of children’s health-related quality of life. Previous cost–utility analyses related to interventions for children relied primarily on adult health-related quality-of-life measures or indirect modelling methods, and few studies directly measured and valued child’s utility with a preference-based child’s utility measure. 40,107 The SEED economic evaluation adopted a validated child-specific preference-based measure, the CHU-9D, to obtain utility values that, together with costs, were used to conduct a cost–utility analysis. In addition, a within-trial cost-effectiveness analysis was conducted using the SDQ, which is the primary outcome measure for the trial.
For details of the methods used to conduct the SEED economic evaluation, see Chapter 3, Economic evaluation.
Results
Intervention cost
The SEED intervention costs are detailed in Table 20. Intervention costs included the staff costs of analysing SDQ responses, recommending activities and preparing for the feedback, and the cost of providing the feedback sessions to the schools (see Chapter 3, Intervention costs). The total average cost per pupil for the 4-year intervention across schools is estimated to be £165.19; the research staff costs of analysing SDQ responses, recommending tailored activities based on these results and preparing for the feedback account for around 90% of the total. For the details of intervention cost calculation for each component, see Appendix 4, Table 54. The total cost for each school ranged between £153.80 and £225.03, varying primarily based on the total number of pupils in a school. For example, the largest intervention cost, £225.03, came from a school cluster with very few pupils (approximately 17 pupils for the 3 intervention years), whereas the smallest intervention cost (£153.80) came from a school that had around 76 pupils for the intervention. The other factors affecting the intervention cost include feedback session duration, number of feedback sessions and school location (time and distance to travel).
Items | Intervention cost (per pupil) in 2017 financial year value (£) | |||
---|---|---|---|---|
First year | Second yearb,c | Third yearc | Fourth yearc | |
SDQ analysis and preparation for feedback sessions | ||||
Staff cost | 43.31 | 21.15 | 41.39 | 42.39 |
Feedback sessions | ||||
Staff | 4.68 | – | 1.88 | 0.24 |
Travel (staff time) | 4.97 | – | 2.20 | 0.14 |
Travel | 1.27 | – | 0.44 | 0.05 |
Material (feedback report, etc.) | 0.77 | – | 0.27 | 0.04 |
Total average per pupil | 55.00 | 21.15 | 46.18 | 42.86 |
Total average per pupil (4 years) | 165.19 |
Missing data
For the number and percentage of missing data for each collected resource use item and CHU-9D index score, see Appendix 4, Tables 52 and 53. A similar number of missing data were observed for both arms, although, for all items, the intervention group has a slightly higher percentage of missing data than the control group. For the resource use, approximately 75% of data were missing for the T0 (baseline) to T1 (follow-up 1) period and 85% of data were missing for the T1 to T3 period. For the CHU-9D index, the proportions of missing data were approximately 18%, 15%, 43% and 32% for T0, T1, T2 and T3, respectively.
Resource use and costs
Appendix 4, Table 51, shows the use of each health and societal resource item (mean number of visits, SD, median, minimum and maximum) accrued among the OC over the first 2 years (2013–2015), assessed at T1, and over the last year (2016–2017), assessed at T3. Overall, there were no statistically significant differences in the use of each resource item between the intervention and control groups, except for the GP visits between T2 and T3.
The observed and estimated mean costs of service use in both arms were compared by using ordinary least squares regression without adjusting for any covariates, and by using a multilevel mixed model of the log form of the cost of resources plus adjusting for covariates. In Figure 6, the distribution of the cost of resource use in the OC over the 4 years in its natural unit (see Figure 6a) and log form (see Figure 6b) is presented. Overall, the cost of the intervention arm was distributed more towards the lower end in Figure 6a, compared with the control group, with a few extreme high-cost values from the intervention arm. The cost of resource use is right-skewed; therefore, the adjusted model predicted the cost in its log form, which was distributed closer to a normal distribution (see Figure 6b). For both models, there were no statistically significant differences at the 0.05 level in resource use between arms (Table 21). Despite this, the adjusted model suggested that the intervention arm was estimated to have a reduced resource use of £279.20 (p = 0.063), compared with the control arm.
Trial arm | Mean | 95% CIa | Difference | p-value |
---|---|---|---|---|
Observedb,c (ordinary least squares, unadjusted) | ||||
Intervention | 1805.25 | 1144.93 to 2465.58 | 30.43 | 0.946 |
Control | 1835.69 | 1248.73 to 2422.64 | ||
Estimated (multilevel mixed effect, adjusted)b,d | ||||
Intervention | 1114.39 | 948.76 to 1308.94 | –279.20 | 0.063 |
Control | 1393.59 | 1198.00 to 1621.12 |
Child Health Utility-9 Dimensions index and quality-adjusted life-years
The CHU-9D index values for each arm of the trial at T0 and at each of the follow-up assessments, and the QALYs accrued over the 4 years, are shown in Table 22. The utility values for both groups declined (suggesting decreased quality of life) over the 4 years from 0.85 to 0.83 in the intervention group and from 0.84 to 0.82 in the control group, with the 0.01 difference at T3. However, this was not statistically significant. There were no statistically significant differences between groups at any of the assessment points, although the number of QALYs gained in the intervention group was 0.05 larger than that of the control group.
Trial arm | Participants (n) | Mean | SD | Minimum | Maximum | p-value (adjusted)a |
---|---|---|---|---|---|---|
T0 index score | ||||||
Intervention | 548 | 0.853 | 0.134 | 0.347 | 1.000 | 0.211 |
Control | 567 | 0.844 | 0.138 | 0.398 | 1.000 | |
T1 index score | ||||||
Intervention | 554 | 0.853 | 0.127 | 0.394 | 1.000 | 0.963b |
Control | 597 | 0.852 | 0.132 | 0.345 | 1.000 | |
T2 index score | ||||||
Intervention | 373 | 0.851 | 0.140 | 0.330 | 1.000 | 0.669b |
Control | 411 | 0.844 | 0.137 | 0.350 | 1.000 | |
T3 index score | ||||||
Intervention | 454 | 0.833 | 0.144 | 0.381 | 1.000 | 0.387b |
Control | 469 | 0.822 | 0.145 | 0.397 | 1.000 | |
QALYs | ||||||
Intervention | 255 | 3.353 | 0.384 | 1.810 | 3.912 | 0.308b |
Control | 292 | 3.306 | 0.407 | 1.887 | 3.912 |
Cost–utility base-case analysis
The cost–utility results for the SEED intervention in the OC are presented in Table 23. Overall, the intervention arm consumed approximately £75.58 less in resources (i.e. a total of intervention cost plus NHS, PSS and public sector resource use) per pupil and gained 0.038 more QALYs than the control group.
Treatment group | Adjusteda cost (£) | Adjusteda QALY | ||
---|---|---|---|---|
Mean | 95% CI | Mean | 95% CI | |
Intervention | 1315.25 | 1131.07 to 1529.43 | 3.498 | 3.453 to 3.539 |
Control | 1390.83 | 1206.96 to 1602.71 | 3.460 | 3.415 to 3.501 |
Difference (95% CI) | –75.58 | –575.85 to 586.18b | 0.038 | –0.023 to 0.107b |
ICER | –£1992.50 per QALYc (intervention dominant) |
The average cost per participant was £1315 (95% CI £1131 to £1529) in the intervention group and £1391 (95% CI £1207 to £1603) in the control group. The mean number of QALYs accrued over the 4-year trial period was 3.50 (95% CI 3.45 to 3.54) for the intervention group and 3.46 (95% CI 3.42 to 3.50) for the control group, leading to a 0.038 (95% CI –0.023 to 0.107) additional QALY gain in the intervention group, compared with the control group.
The 1000 iterations of non-parametric bootstrapped pairs of mean costs and QALYs are shown in the cost-effectiveness plane (Figure 7). The x-axis represents the incremental QALYs and the y-axis represents the incremental costs (£). It shows that the incremental cost is almost evenly distributed across the x-axis, which suggests that the costs across the intervention and control groups are similar. Most of the bootstrapped pairs are to the east (see Figure 7), which corresponds to the positive point estimate of the incremental QALY. The precise percentage of the bootstrapped pairs of mean cost and QALYs distributed in each of the quadrants is outlined below. The probability that the intervention achieved an incremental QALY was approximately 90% (north-east + south-east quadrants), and the probability that the intervention was cost-saving was approximately 56% (south-west + south-east quadrants).
The distribution of the bootstrapped pairs of cost difference and QALY difference in the four quadrants are as follows:
-
north-east – intervention is more costly and more beneficial (40.2%)
-
north-west – intervention is more costly and less beneficial (4.1%)
-
south-west – intervention is less costly and less effective (6.5%)
-
south-east – intervention is less costly and more effective (49.1%).
The CEAC for the base-case analysis is shown in Figure 8. The probability that the SEED intervention was cost-effective at the standard threshold of £20,000 per QALY gained was 88.5%. This corresponds to the proportion of pairs under the £20,000 per QALY threshold slope in the north-east, south-west and south-east quadrants of the cost-effectiveness plane (see Figure 7). When the threshold equals 0, the probability of the intervention being cost-effective was 55.6%, which corresponds to the proportion of cost and QALY difference pairs under the x-axis in the cost-effectiveness plane (i.e. the south-west + south-east quadrants in Figure 7). The probability increases with the increase of the cost-effectiveness threshold and remains at around 89% after the threshold is > £40,000 per QALY gained.
Sensitivity analyses
The results of the sensitivity analyses are presented in Table 24. When varying the intervention cost up and down by 30%, the incremental cost varied; however, the conclusion did not change: the intervention was still associated with a reduced incremental cost and improved QALY, with > 80% probability that the ICER was lower than the £20,000 threshold. When the missing whole-wave cost questionnaire (see Chapter 3, Economic evaluation) was assumed to be no resources used as long as the CHU-9D index at the same wave was completed, the number of observations available for cost estimation increased from 124 to 511. This scenario also generates a similar conclusion, although the cost saving became more obvious and the probability for the intervention to be under the £20,000 per QALY threshold increased to 100%. Both complete-case analysis and multiple imputation (for the cases that had both CHU-9D index and cost missing) were originally planned; however, the estimation failed to proceed for the cluster analysis after the multiple imputation procedures. Therefore, only the complete-case analysis result is presented here. When the discount rate increased from 1.5% to 3.5%, neither the incremental cost (–£69) nor QALY (0.036) varied much from the base-case analysis, and the bootstrapped probability decreased to 84.8%.
Analysis | Adjusteda incremental cost (intervention – control) (95% CIb) (£) | Adjusteda incremental QALYs (intervention – control) (95% CIb) | ICER (£/QALY gained) | Probability of being cost-effective at threshold of £20,000 per QALY gained (%) |
---|---|---|---|---|
Base-case analysis | ||||
Base case | –75.58 (–575.85 to 586.18) | 0.038 (–0.023 to 0.107) | Intervention dominant | 88.49 |
Intervention cost | ||||
Intervention cost increased by 30% | –16.51 (–525.25 to 589.30) | 0.038 (–0.026 to 0.105) | Intervention dominant | 85.14 |
Intervention cost decreased by 30% | –135.43 (–610.86 to 548.29) | 0.038 (–0.023 to 0.112) | Intervention dominant | 89.9 |
Missing data | ||||
Zero replaced missing resources when CHU-9D index is not missing to enable complete-case analysis | –462.78 (–617.67 to –287.52) | 0.038 (–0.030 to 0.106) | Intervention dominant | 95.6 |
Discount rate | ||||
3.5% for both cost and effect | –68.95 (–538.37 to 566.29) | 0.036 (–0.029 to 0.104) | Intervention dominant | 84.8 |
Perspective | ||||
Educational sector perspective (intervention cost only) | £165.19 (NA) | 0.038 (–0.026 to 0.103) | 4347.11 (–24,451.7c to 42,808.4) | 82.1 |
Cost-effectiveness analysis
Table 25 summarises the estimated additional cost per 1-point decrease in SDQ-TDS among the whole sample and the YC and OC separately over the 4 years. For the whole sample, the cost-effectiveness analysis resulted in an additional cost of £82 per 1-unit decrease in the SDQ-TDS, and an additional cost of £396 per 1-unit decrease in the SDQ prosocial score (i.e. the strengths subscale). Among the YC, the intervention was associated with a £166 increase in the cost and a 0.19 decrease in SDQ-TDS, leading to an £856 per 1-unit SDQ-TDS decrease. Costs were lower in the OC because of the smaller incremental cost and much larger treatment effect, with an ICER of £1.22 per 1-unit decrease in SDQ-TDS. The ‘strengths’ (prosocial) score did not differ between the intervention and control groups in either of the separate cohort analyses.
Outcome: SDQ score at T3 | Incremental cost (intervention – control) (95% CIa) (£) | Incremental SDQ score (intervention – control) (95% CIa) | Cost per SDQ score decrease (£) |
---|---|---|---|
Both cohorts | |||
Adjustedb SDQ-TDS (i.e. ‘difficulty score’) | 68.79 (–147.13 to 306.77) | –0.841c (–1.450 to –0.260) | 81.80 |
Adjustedb SDQ prosocial score | 68.79 (–147.13 to 306.77) | 0.174 (–0.00026 to 0.39) | Control dominant |
YC | |||
Adjustedb SDQ-TDS (i.e. ‘difficulty score’) | 166.11 (–124.79 to 477.86) | –0.194c (–0.914 to 0.464) | 855.90 |
Adjustedb SDQ prosocial score | 166.11 (–124.79 to 477.86) | –0.075c (–0.322 to 0.196) | 2214.80 |
OC | |||
Adjustedb SDQ-TDS (i.e. ‘difficulty score’) | 2.35 (–337.66 to 452.37) | –1.92c (–2.75 to –1.02) | 1.22 |
Adjustedb SDQ prosocial score | 2.35 (–337.66 to 452.37) | 0.49 (0.23 to 0.82) | Control dominant |
Discussion
The results show that the SEED intervention is likely to be the dominant option (i.e. incurs less cost and is more effective), compared with the control group, for the older pupils (P5 at baseline) from a public sector perspective over the 4-year time horizon. However, this finding should be interpreted with caution because of the statistically insignificant incremental costs and QALYs. The intervention group was estimated to consume fewer NHS, PSS and other public sector resources (–£279.2) than the control group (p = 0.069) after adjusting for covariates and the two-level cluster effect. Although there were no statistically significant differences for any of the assessments, the intervention arm had consistently higher quality-of-life values, as measured by the CHU-9D index, than the control group for all assessments. The estimated QALY difference over the 4 years, adjusted for the baseline utility values and other covariates, was small (0.038, 95% CI –0.023 to 0.107) and favoured the intervention arm. Although there was no statistically significant difference for the incremental cost and QALYs, the probability that the intervention is cost-effective at a threshold of £20,000 per QALY gained was high, at 88%. This likelihood did not vary substantially with the assumptions in the sensitivity analyses when the intervention cost was changed, the methods of handling missing data varied, the discount changed and cost elements changed.
The cost-effectiveness analysis revealed a noticeable difference in terms of the cost and outcomes between the two cohorts. The incremental cost between groups among the OC was very small, indicating the resource use savings in the intervention group. This is consistent with the findings from the cost–utility analysis. In contrast, this resource use saving was not observed in the YC. The incremental cost (£166) was close to the intervention cost (£165), indicating that the resource use between arms among the YC was very similar. In addition, the incremental 1-unit SDQ-TDS decrease in the OC was much larger than in the YC (1.92 vs. 0.19, respectively). This led to the much lower cost per 1-unit SDQ-TDS decrease in the OC (£1.22 per 1-unit SDQ-TDS decrease) than in the YC (£856 per 1-unit SDQ-TDS decrease).
Further exploration is required to investigate the resource use cost savings surrounding the intervention in the OC. The dominant result of this intervention was due to the low intervention cost (£165.19 per pupil over 4 years) and the estimated cost savings in NHS, PSS and other public sector resource use from the resource questionnaire in the intervention arm, compared with the control arm. However, there was no statistically significant difference when comparing the individual resource use items between the arms. Despite this, when the last sensitivity analysis removed the NHS, PSS and other public sector resource use from the cost equation by incorporating only the intervention cost, the probability of the SEED intervention being cost-effective at the threshold of £20,000 per QALY gained was still > 80%. This suggests that the intervention is still more likely to be cost-effective at a £20,000-per-QALY threshold, even when the resource gain is a chance finding.
Another issue arising from the analysis is the substantial number of missing cost data. Approximately 75% and 85% of data were missing for the NHS, PSS and other public sector cost questionnaire collected at T1 and T3, respectively. Potential reasons for the very low response rate are questionnaire fatigue and distrust of unfamiliar organisations (despite provision of participant information sheets). The resource questionnaire was distributed along with other questionnaires collected for the trial via pupil school bags at all follow-ups, so the completion rate also relied on either the pupils passing the questionnaire to their parents or the parent checking the school bags. An alternative method of dealing with the missing data was explored in the sensitivity analysis, which assumed that no resource was used if the CHU-9D index at the same wave was not missing. This scenario increased the number with complete cost data available for cost analysis from 10% of the sample to 40%. The results of this scenario showed an increase of resource saving due to the intervention, and a subsequent increased probability of the intervention being cost-effective (i.e. 95.6%). Therefore, it was determined that the risk that the results were biased owing to the missing data was not high. We acknowledge the limitation that the extent of missing not at random was not explored, which may potentially inform if there are any population characteristics that may systematically contribute to the missing data.
The trial data collection methods were not initially designed to involve parental data collection; however, the school-bag approach was an attempt to gather resource use data for the economic evaluation. Future studies could involve and engage with parents at the outset to garner greater study buy-in; the use of financial or voucher incentives could even be considered.
Early intervention has considerable potential to benefit the individual, their family and society as a whole. 110 Universal approaches attempt to promote well-being in all children, whereas targeted approaches are directed only at those most vulnerable and at highest risk of developing, or already experiencing, problems. Growing evidence suggests that a universal approach may be more effective and efficient in improving health and well-being in adolescence and adulthood. 62 Whole-school approaches aim to involve everyone (pupils, staff, families and community) to change the environment and culture of the school,111 and the literature seems to favour these approaches, as well as the use of multicomponent programmes. 44,112,113 Searches of the literature reveal approximately 350 cost-effectiveness analyses or economic evaluations of school-based interventions; however, there were no economic evaluations of school-based interventions specifically related to improving SEW that we are aware of. This trial investigated whether or not implementation of SEED in a school is effective and cost-effective when compared with standard school instruction, that is Scotland’s CfE. Details of the design, methods and development of the SEED RCT are given elsewhere. 66 The SEED intervention is considered a public health intervention; therefore, NICE public health guidance was employed as a basis for methods development for this economic evaluation. 114
At the time of the research design, searches of the literature had revealed no economic evaluations of school-based interventions specifically related to improving SEW. Hence this was the first of its kind. We adopted contemporary methods for conducting an economic evaluation alongside a public health intervention, including adopting a wider perspective, the use of a preference-based quality-of-life instrument and a lower discount rate (1.5%). The number of economic evaluations of interventions such as the SEED intervention are increasing; the 2020 publication by Turner et al. 115 showed that the Promoting Alternative Thinking Strategies curriculum has the potential to be cost-effective at standard UK willingness-to-pay thresholds; however, the sensitivity of the cost-effectiveness estimates to key assumptions means that decision-makers should seek further information before allocating scarce public health resources. Research priorities include the need to improve the evidence base for the cost-effectiveness of such complex school-based public health interventions; this may require the use of broader economic evaluation frameworks, such as cost–benefit analysis, as recommended by Belfield et al. 116 and Tudor et al. 117
Conclusion
The results showed that the SEED intervention is likely to be cost-effective for the OC, compared with the control group. However, this finding should be interpreted with caution because of the statistically insignificant incremental costs and QALYs, although all the sensitivity analyses indicated that this finding was unlikely to be biased. The SEED intervention was associated with greater incremental cost per SDQ score decrease in the YC than in the OC, and therefore appears to be less cost-effective for younger pupils.
Chapter 6 Process evaluation
Introduction
This chapter outlines the findings from the process evaluation, which was conducted to answer research questions related to the implementation of the SEED intervention. The main process evaluation comprised of thematic analysis of interviews, focus groups and intervention observations and selected quantitative analyses. This chapter is structured as follows:
-
mapping of research questions to qualitative and mixed-methods findings
-
presentation of qualitative findings
-
fidelity and quality of implementation of the SEED intervention
-
exploration of causal pathways within the programme theory and SEED theory of change model
-
conclusions.
Reflective statement
Members of the SEED research team undertook the process evaluation of the SEED intervention. This means that most data were collected and analysed by the same people who helped develop, design and implement the SEED intervention (in particular SB, SS, MH and DW). Although much care was taken to collect data rigorously throughout, for example through structured interview schedules and ethnographic forms, and the conduct of systematic analysis, the role of the researchers in this process, and their potential impact on results, should be acknowledged.
The involvement of the main SEED researchers in the delivery of the intervention meant that they attended many of the school visits for data collection and all the RD sessions, and so were embedded in intervention implementation. Furthermore, they maintained regular contact with the SEED schools, building up relationships over the life of the trial. Although there is a clear risk of researcher bias, we believe that the benefits to a rich process evaluation of the researchers being immersed in the experience of intervention delivery outweighed this.
Mapping of qualitative themes to research questions (original research questions from the protocol and revised process evaluation questions)
See Appendix 5, Table 55, for an illustration of how the qualitative themes and mixed-methods data sources map on to the research questions, both from the protocol and the revised research questions. It is recognised that many questions can be addressed by multiple themes and that the themes presented qualitatively are not exhaustive.
Qualitative findings
The main themes from the thematic analysis of the qualitative data collected were as follows.
-
Perceptions of the SEED intervention.
-
Staff perceptions of the SEED intervention.
-
Pupil experiences of the SEED intervention.
-
Parent experiences of the SEED intervention.
-
-
Implementation of the SEED intervention.
-
Collection of SEED data.
-
Presentation of data and RD sessions.
-
Development and maintenance of APs.
-
The SEED resource guide.
-
Displaced activities.
-
-
Mechanisms for change.
-
Raising awareness of, and increasing focus on, the importance of SEW.
-
Space/time for professional dialogue and reflection.
-
Assessing the school’s SEW profile:
-
use of data to identify needs
-
validating and reassessing existing work.
-
-
Supporting decision-making:
-
contributing to a culture of evaluating practice
-
integration with formal school improvement planning.
-
-
Implementation of activities at whole-school and curricular levels.
-
Relationships.
-
-
Study effects: the experience of control schools.
-
A raised awareness and sensitivity among school staff towards SEW.
-
Evaluating practice.
-
Lack of impact.
-
Modelling SEW assessment.
-
-
Contextual factors
-
Reasons for participation.
-
School-level barriers and facilitators:
-
cultural readiness for change and school ethos
-
relationships with EPs
-
staff and pupil turnover
-
time and resources.
-
-
Local authority and national-level barriers and facilitators:
-
CfE and national priorities for health and well-being
-
LA priorities
-
time and resources.
-
-
School identifiers comprise four digits representing the arm of the trial (C = control; I = intervention), the LA (A, B or C), a school identifier and school type (P = primary; S = secondary).
Perceptions of the Social and Emotional Education and Development intervention
This section summarises perceptions of the intervention from the perspectives of staff, pupils and parents. The following section examines the implementation of each of its core elements and considers who engaged most with each.
Staff perceptions of the Social and Emotional Education and Development intervention
First reactions to the intervention were universally positive among those who agreed to participate, reflecting a cultural positivity towards addressing SEW within schools:
I can remember thinking that it was really good that people were still prioritising social and emotional well-being and development and it was hopefully still a key factor in terms of positive outcomes for children.
SchCC4P, HT interview, T1
There is a big value to the project [SEED], it empathises the emotional part of children’s lives and how they will affect them for the rest of their lives and ours. We are building the Scottish people of the future.
HT, SchIC4P, RD observation notes, T1
However, there were misunderstandings over the process when first embarking on the SEED intervention, in particular the lack of concrete resources provided:
I think some schools maybe stepped forward thinking that … they would be given perhaps packs of … information, as opposed to actually thinking – having to think it out themselves.
SchIA3P, EP interview, T3
Pupil experiences of the Social and Emotional Education and Development intervention
In focus groups with the older pupil cohort when they had left primary school and were in the second year of secondary school, they reported being unaware of the SEED intervention in their school beyond completion of questionnaires, suggesting that the SEED intervention as a process was not visible to pupils. However, some free-text questionnaire responses suggest that the process of completing the SEED questionnaires annually encouraged reflectivity around the pupils’ own SEW:
It makes you think and truly begin to like and love who you are. It makes you aware of yourself and you get to explore yourself as a person but I’m still figuring out who I am.
SchCA2P, OC pupil, T3
It was kind of good to reflect on how I’ve been feeling/doing instead of pushing it away.
SchCB5P, OC pupil, T3
Parent experiences of the Social and Emotional Education and Development intervention
There were only minimal data on parents’/carers’ experiences of the SEED intervention, gathered from free-text responses in the parent questionnaires (approximately one-third of parents completed questionnaires at T0 and T2; only a small proportion of these completed free-text responses). There was no evidence that parents/carers were aware of any changes within the schools as a result of participation in the SEED intervention, but most who commented said that they thought that the study was worthwhile. Several parents looked forward to hearing findings resulting from the research:
Really useful research programme to help understand pupils on their journey from primary to secondary school. I have enjoyed participating in the study and am sure my child has too.
OC parent questionnaire, free-text response, T3
There was some evidence that, for both pupils and parents, completing the questionnaires may have encouraged dialogue and reflection around issues of SEW within the family:
This has been very useful, my son has enjoyed completing the questionnaire at school and speaking about his responses at home.
OC parent questionnaire, free-text response, T1
The [parent] questionnaires are a really good prompt for you to think about your child and their school. Often we take things for granted, but having to think about your child’s happiness and well-being makes you realise that these aspects can be very challenging for other children and families. It prompted me to talk to all of my children about the importance of being kind to everyone.
YC parent questionnaire, free-text response, T3
Implementation of the Social and Emotional Education and Development intervention
Collection of Social and Emotional Education and Development data
The data collection process appeared to go smoothly across all schools, particularly with the self-completed pupil survey completed in class time:
[The process of collecting the data] was very well done and well organised. I don’t think there were any issues for us at the time. You know, it wasn’t obtrusive, and you worked with the school to make sure it was the right time.
SchCC5P, HT interview, T1
Completion of SDQs for pupils by teachers also seemed to be acceptable:
It was actually reasonably quick to fill in for each child. And I didn’t feel it was an onerous task.
SchCC2P, HT interview, T1
However, there were low response rates for parents and variable response rates for staff (detailed later in this chapter), which may have, in part, have been a result of the demands of questionnaire completion:
Questionnaire far too long & time-consuming for busy/working mothers. Future questionnaires should be shorter but more frequent to ensure all information is captured.
OC parent questionnaire, free-text response, T1
Yet another piece of paperwork – Sorry!
SchIC4P, staff questionnaire, free-text comments, T3
Staff also commented on the time of year that the survey was administered, raising concerns over the representativeness of responses, and possibly accounting for lower return rates:
The questionnaire to staff is given at a time when staff are very tired and stressed.
SchIB1P, staff feedback sticky notes from RD session, T2
Response rates improved when HTs gave staff time to complete surveys in school:
Both this year and last year, I put a little bit of time aside. I think the last time it was at a staff meeting, and this time it was at an in-service day.
SchCA3P, HT interview, T1
Presentation of data and reflective discussion sessions
The role of the RD sessions and engagement with the data feedback as possible mechanisms of change will be explored in more detail in Mechanisms for change. More general perceptions of these processes and the validity of the data fed back to schools in these sessions is explored here.
Presentation of Social and Emotional Education and Development data
The data presented in sessions were received enthusiastically. Staff seemed genuinely interested in seeing data from their schools, commonly referring to data as ‘interesting’, ‘useful’, ‘encouraging’ and ‘informative’. Staff in almost all schools seemed to respect and value the data, taking them at face value and viewing them as an accurate representation of the school:
[The SEED] data collection was as good a snapshot as any that you get from the perception of children and young people, and where the schools were at. So I think that that was, that was pretty helpful, I would say.
SchIA1P, EP interview, T3
The SEED findings were seen as more detailed and more independent than alternative surveys:
Yes, it has to be triangulated with our knowledge of ourselves, but there’s something quite affirming and possibly challenging about someone else coming in and providing you with that.
SchIB1P, EP interview, T3
Reactions to the data were generally split between thinking the data reflected positive work in the school or revealed surprising or concerning issues:
The data were very strong on school ethos and staff seemed to be aware of this and expressed a commitment to creating a nurturing environment for pupils.
SchIA6P, RD observation notes, T0
Pupil responses to questions related to emotions and the ability to express these was picked up on by staff as being of concern, as well as responses to questions about feeling positive about themselves/ self-esteem.
SchIC6P, RD observation notes, T0
However, some participants questioned the validity of the pupil data, in particular. There were queries over pupils’ honesty in answering questions, their interpretation of the questions or their ability to be sufficiently self-aware to answer ‘correctly’:
It’s hard for a child at that age to know how they feel – it’s not surprising.
IB3P, RD observation notes, T1
Children might answer some questions as they think they ought to.
SchIC1P, RD observation notes, T1
Furthermore, there were concerns over the representativeness of the data. For the staff and parent data, such concerns were usually linked to low response rates:
I remember where you came and did a feedback session and it was all about, ‘oh the behaviour’s really bad, blah, blah, blah …’. And then we looked at it, and only seven people had responded.
SchIC2P, HT interview, T3
For pupil data, there were concerns that surveying only two cohorts of pupils meant that the data did not represent the whole school population. Although some respondents did say that they could use the two cohorts as a benchmark for other pupils in the school, others said that this limited the impact of the SEED intervention, particularly following the OC’s transition to secondary school at T2 and T3:
I think it would have been better had there been more children involved, definitely. Because … primary schools are strange places, you can get particular classes who are very, very different from other classes. So it is hard to come out with a generalisation, if you’re just looking … I mean as soon as the last, you know, the children left, we were only looking at one class.
SchIA2P, HT interview, T3
A final concern, relating to the presentation of data, was that there were simply too many data or that they were difficult to understand, particularly the SDQ scores:
I felt it was, if I’m being honest, it was probably too detailed? … More headline-type things would be kind of easier and probably because people are trying to think of so many things … It’d be quicker.
SchIC4P, HT interview, T1
When in small groups DHT [depute head teacher] ‘wanted to skip the SDQ as too complex’.
SchIB2P, RD observation notes, T2
Discussion of the Social and Emotional Education and Development data
There were many perceived benefits from the opportunity to discuss the SEED data in RD sessions; these are explored more thoroughly in Mechanisms for change.
The RD session was perceived as being a crucial element of the SEED intervention, both in terms of the provision of time to discuss the data and reflect on practice, and in terms of consolidating understanding of the SEED process:
I found today really interesting, thought provoking and useful. The process now makes sense to me!
SchIA1P, Staff feedback sticky notes from RD session, T2
The non-SDQ data from the SEED pupil questionnaire attracted more interest and engagement than the SDQ scores (total difficulties and subscales):
The most interesting ones [questionnaire responses] are the ones that the P5 pupils completed themselves.
SchIA5P, RD observation notes, T1
There was also evidence that the teachers of those pupils surveyed got more out of the process than other school staff:
The teacher who taught P7 at T1 was particularly engaged – she felt the data was representative of that year group.
SchIA1P, RD observation notes, T2
Engaging with the data may have been particularly challenging for staff members joining the school during the life of the trial:
Is the data clear enough? For those of us who have not been part of the process, it has been challenging to understand the relevance of the data.
SchIB4P, RD observation notes, T2
There was also evidence that the RD sessions had a negative or demoralising effect on some staff, particularly if the data contradicted existing perceptions of good work in the school:
I felt quite negative after the last feedback session as I felt we all got along really well – but the feedback did not reflect that and that has had an impact on some of my answers this time!
SchIC2P, staff questionnaire, free-text comments, T3
Anxiety or negativity was particularly evident in discussions of the staff data. The staff data often generated substantially less discussion than the pupil data. Sometimes this was accompanied by a tense atmosphere, particularly if the data focused on difficulties with management or other issues such as tensions between teaching colleagues, which had the potential to make staff feel uncomfortable:
There was generally good chat going through the report and lots of engagement when asked for initial reactions to the data. But there was very little discussion around the staff data – it was very quiet at this point. But staff were nodding when we talked about how staff well-being affects pupil well-being.
SchIA1P, RD observation notes, T2
The EP’s potential in facilitating discussions around sensitive data was optimised when there was a strong relationship between the EP and the school staff, in particular when the EP had prior sight of the SEED data (which was standard procedure):
I would say that [the data] needed to be really, really, carefully handled. And I think you did, as researchers, handle that as carefully as you could … you took the time to speak to me, and I think that that was probably very important. And I suppose what I would say is, if people are coming as, if you like, outside professionals to a situation, you also need to be liaising with those folks that are on the ground who are maybe very much aware of the dynamics, etc., that are going on.
SchIA3P, EP interview, T3
Development and maintenance of action plans
Schools varied in the way they went about formulating their SEED APs following the RD sessions, but most were led by the school’s key SEED contact, usually the HT or depute HT. Several schools designated the task of developing the AP to a working group, although predominantly these were existing groups that had a remit to look at health and well-being more broadly [e.g. Responsible.
Accountable, Consulted and Informed (RACI) groups], rather than new groups set up with the express purpose of working on the SEED intervention.
Although the HT/depute HT was usually responsible for completing the final AP, school staff also contributed to the discussions around the development of the plan:
Everyone was involved. We talked about it at various in-service days, we took the policies forward. One or two people did, sort of, wee group activities to actually pull together some of the policies and some of the thoughts that we had. But everybody had to be involved. That was something that we really wanted to make sure that all the staff, support staff and others, were actually fully involved with that.
SchIA5P, HT interview, T3
There was evidence that this process engaged some staff in discussions they might not otherwise have been part of:
I think [support staff] appreciated being involved in it, and I also heard them discussing it between themselves later.
SchIA2P, HT interview, T1
Some EPs were involved in these conversations and in the writing of APs:
Well, I took all the ideas and we took the ones that were already had been recognised from our questionnaires’cause we only had three really main things that we wanted to do. It was fairly easy to do that for the action plan. Then I sat with [EP] initially because she’s been with me through all the ups and downs and knows her stuff what-have-you. And we sort of brainstormed it together, so she did things like she came in and spoke to [the rest of the staff] about it at a CAT [collegiate activity time] night.
SchIA3P, HT interview, T1
Once APs were written, there was sometimes a lack of clarity about who delivered the interventions:
As a teacher, as part of the teaching group … yes we were part of the discussions, but I think we weren’t as clear as to then what had been put in place to try to address that … and we weren’t maybe as involved in implementing it. And, do you know, we probably – we were involved in implementing it, we just didn’t realise that’s what we were doing.
SchIC3P, class teacher interview, T3
Once written, the activities on the AP may have become part of everyday practice and not distinct from other activities in the school. Some schools did refer to informal ways of maintaining the AP, including drawing on support from the EP:
Because there’s an action plan [the EP]’s been coming in and checking how we’re doing with the action plan and she’s been in the staff room and not necessarily sitting ‘Let’s talk about the SEED action plan’ but just asked questions about how they are and how they’re going and then fed that back to me.
SchIA3P, HT interview, T1
The lack of a formal auditing or monitoring procedure for APs may have caused some schools to ‘lapse’ following the initial development of the plans. A small number of school staff also reported a lack of embeddedness in the whole-school structure as a barrier to optimising the delivery of APs:
And I would say [a barrier] would be, as a management team, ensuring that it’s distributed down and that everyone is aware; class teachers, SLAs [support for learning assistants], everyone, of our focus, of our action plan, that we’re all shared in it … Yeah. I think if I was to, you know, be involved again [in the SEED intervention], the involvement of the teaching staff would need to greatly increase.
SchIC3P, HT interview, T3
The Social and Emotional Education and Development resource guide
The SEED resource guide did not appear to play a major role in the selection of activities following RD. Only in a few cases was there a specific mention of selecting a programme from the guide:
We’d identified BounceBack! … through the list that I had got from SEED.
SchIC3P, HT interview, T3
We did look at [the SEED resource guide] and we sort of looked at what we were doing already and what more we could do and seeing whether the stuff on your resource page would be better than the things we were already using and so did change some things.
SchIC4P, HT interview, T1
We also did the Cool in School which was through the resources that had been recommended by SEED.
SchIA2P, HT interview, T3
It was suggested that the resource guide was not sufficiently distinct from the other health and well-being resources that were available to schools:
I think some of the staff have been on and looked at [the SEED resource guide] in the first year, but not the last year. I’ll have to remind them it’s there, actually. Because I have actually sent a lot of stuff at them but it’s in … we’ve got a health and well-being folder on the server and the stuff that you recommended is all in there and I’ve added, loads more has been added to it over the years. But that’s actually one of the things that I’ve said that it’s too much in it … ‘Cause everyone produces stuff on health and well-being.
SchIA3P, HT interview, T3
Some were also put off by the expense of some of the resources in the SEED guide:
I think [the SEED resource guide] is helpful. The only thing is … I had to put that against what is already within [the LA], in terms of the guides and things that we’ve got already, because we wouldn’t particularly be expecting people to spend money on resources or whatever.
SchIA1P, EP interview, T3
Displaced activities
When asked if implementing the SEED intervention resulted in the displacement of any other activities, the overwhelming response from school staff members was ‘no’. Similarly, there is little evidence that the SEED intervention precipitated the redistribution of staff or other resources; however, some HTs did talk about looking for ways to use protected (staff non-teaching) time more efficiently, which could be seen as a benefit of the SEED intervention:
No … nothing we’ve stopped doing. There’s other things we’ve added to do it better … or maybe changed the processes of some things, so it’s more – slick, if that makes sense?
SchIC4P, HT interview, T1
Mechanisms for change
This section explores the possible mechanisms by which the SEED programme achieved its impacts.
Raising awareness of, and increasing focus on, social and emotional well-being
Involvement in the SEED intervention appeared to raise the profile of SEW among staff. This was evident both in terms of the relative importance of SEW to the other main curricular areas, literacy and numeracy, and in terms of the profile of SEW within health and well-being:
Health and well-being is one of the core aspects of CfE, and I think the SEED involvement and all of those other little parts have helped staff to know that, actually, it’s probably the most important.
SchIC3P, HT interview, T3
I think [SEED]’s been the driver for us to really sit down, and although we were aware of the emotional intelligence and the need to look at social well-being, it’s actually been the driver to make us actually look at it and put things in place to improve it. Whereas before, it was an add-on to health and well-being and it was taking a smaller focus. The actual profile has increased.
SchIC1P, HT interview, T1
This improved focus helped to embed SEW within existing structures for improving health and well-being:
It’s [SEED] made people much more aware of what they’re actually doing in relation to social and emotional [well-being]. You know, they did it before but it … until you actually focus on something, it’s harder to sort of … see that, you know, how it fits in the bigger picture. And I think over the last year we’ve developed the SHANARRI [safe, healthy, achieving, nurtured, active, respected, responsible and included] thing much more than we had done before. You know, and it seems to dovetail quite nicely.
SchIC4P, HT interview, T1
Space/time for professional dialogue and reflection
One of the most commonly cited benefits of the SEED intervention was the way in which the RD sessions in particular provided time and space to reflect on pupils’ and staff SEW needs as a whole staff group. Many comments implied that, prior to the SEED intervention, these opportunities were rare:
Most interesting point from today – the fact that we all had time to sit and discuss these issues.
SchIA6P, staff feedback sticky notes from RD session, T1
Valuable time to discuss and decide on priorities as a whole-school staff.
SchIB4P, staff feedback sticky notes from RD session, T2
The surveys and feedback have allowed us to think about health and well-being as a staff across all areas. This is not something we would normally do.
SchIC2P, staff feedback sticky notes from RD session, T3
The data presented in the sessions were recognised as important in shaping discussion and allowing exploration of issues that may not have otherwise been raised or addressed:
Data has helped focus on aspects that we may not have been aware of if we had just had a general discussion.
SchIA6P, staff feedback sticky notes from RD session, T1
Good opportunity to discuss issues that have been lingering under the surface!
SchIC3P, staff feedback sticky notes from RD session, T3
Although the majority of comments referred to the benefits of professional dialogue and reflection at the structured SEED RD sessions, there was also some evidence of discussion of the SEED data outside the RD sessions:
The ripple effects of what … you did were helpful, because the research was there, the data was there, the looking at it was there, it provided then further conversation opportunities to move things forward.
SchIA3P, EP interview, T3
[The data] was very useful for both provoking discussion but also for evidence … and after you were away, I mean, it wasn’t just – you know, we discussed it on our own … it was and has been brought up in the staff room just sort of informally as well.
SchIA6P, HT interview, T3
In addition to providing space for dialogue at a whole-school level, the SEED intervention also appeared to stimulate personal reflection on individuals’ own practice:
A good opportunity to view the changes in staff/pupil attitudes and feelings over the years. Also good to reflect on our own practice.
SchIB4P, staff feedback sticky notes from RD session, T2
Useful to be aware of staff feelings so that I can adjust my own behaviour/relationships if needed.
SchIC3P, staff feedback sticky notes from RD session, T3
Assessing the school’s social and emotional well-being profile
Use of data to identify needs
As discussed in Discussion of the Social and Emotional Education and Development data, the provision of concrete data gave evidence for staff to validate existing priorities for SEW, as well as highlighting areas of strength. The data helped schools explore needs around SEW generally, but they also helped to identify new or unexpected areas of need.
There was evidence that this engagement with the SEED data was crucial in facilitating the co-production of activities tailored to a school’s individual needs:
I suppose in hindsight, the fact that you didn’t give us something [an ‘off-the-shelf’ package], meant that we looked at your data and tailormade, I suppose, what we did for our children. Does that make sense? … And I don’t think that the process of looking at children, doing the action plan, etc., would be different if you hadn’t been involved … I think what you added, the value in terms of, was the data, because you gave us a lot of data about our children.
SchIA1P, HT interview, T3
Validating and reassessing existing work
Although the data did highlight new or unexpected areas of need, in more instances the data validated existing concerns:
I think there was always an understanding that there were issues around attachment, but SEED clarified that for us … it was nice to get that kind of affirmation that yeah, we are on the right track, these are the kind of things that we’re needing to address. And actually to make us sit down and focus on a plan.
SchIA3P, HT interview, T3
This often prompted schools to revisit or reprioritise existing activities:
We were thinking of formalising Circle Time a bit more – it was something, actually, again, that came out of our SEED project, that perhaps we needed to formalise it a bit more.
SchIC3P, HT interview, T1
And so then, this year, we’ve been running, like, a working group has been looking at nurture and then looking and just on the last closure day, had a whole session on what does nurture look like? What does it look like, you know, and talking about Boxall Profiles and … And I really do think that if we hadn’t been a part of the SEED programme, it may not have come to us as quickly. Does that make sense?
SchIC2P, HT interview, T3
Evidence from the data also provided some extra ‘clout’ to support an increased focus on particular issues:
It [pupils’ voice] was an issue already, but because it was validated by your study it meant it wasn’t me banging at my drum.
SchIA3P, HT interview, T3
Useful data today about the P4 class I currently have – gives me evidence to back up request for extra support.
SchIB2P, staff feedback sticky notes from RD session, T2
Staff participants frequently spoke of how the SEED intervention allowed them to focus on their strengths and helped them consider how to maintain the positive work going on in the school, as well as addressing areas of need:
SEED has been incredibly helpful in allowing us to celebrate our successes and reflect on what we can do to further help our children.
SchIA2P, staff feedback sticky notes from RD session, T2
Supporting decision-making
Contributing to a culture of evaluating practice
Beyond raising awareness and encouraging reflection around SEW, the SEED programme appears to have been instrumental in providing a vehicle for evaluating practice:
I think [SEED]’s given us more of a focus and, perhaps, again, because you can get a wee bit complacent, you can say, ‘Oh, we’re good at this. They said 6 years ago when we were inspected that we’re good at this’, so you don’t revisit things. Whereas when you’re involved in a process it does make you evaluate what you’re doing and say, ‘Maybe that’s time for a wee look at that. Maybe that’s not as effective as we thought.’, I think that’s probably the value of it for us.
SchIA7P, HT interview, T1
SEED has been the catalyst to make us do that reflection and say, ‘well, how are we doing this?’, you know, ‘What, how are we meeting the needs?’. And then saying, ‘well we were doing it, kind of, but we maybe need to do it better.’.
SchIC2P, HT interview, T3
In support of self-evaluation, the SEED programme was seen as an opportunity to track improvements over time:
I was delighted [to be chosen as an intervention school] because [of] the opportunity to actually do that questionnaire and to actually get that baseline, and then measure to see what we put in place and see what that impact has … although you’d have school improvement plans you don’t have such a tight measurable tool.
SchIC1P, HT interview, T1
In practice, participants reported benefits in seeing changes to the data over time. This was empowering for staff and provided validation for any changes implemented as a result of the SEED intervention:
If I look at the first time you came and sent back the data, and the second time? I think staff were able to see that the work they had done had actually made a difference to the children. Because the children’s answers were more positive. So I think from that point of view, it gives them encouragement that they’re along the right lines with what they’re doing.
SchIA1P, HT interview, T3
As noted in Implementation of the Social and Emotional Education and Development intervention, however, not all change was positive, and the detailed, stark nature of the SEED data sometimes meant that schools had to address why scores fell over time. The following quotation is from a discussion between a member of the SEED team and a school’s key contact just before the RD session took place. The HT had seen the data in advance:
Looking at the staff data, I take it almost all of these mean the 2015 results are ‘worse’ than the 2013 ones? I just want to be clear so I’m prepared … We would never want to demoralise the staff but as a SMT [senior management team], we need to address this if it is agreed that these areas are key issues for staff. We as SMT also do not want to feel demoralised!
SchIB2P, RD observation notes, T2
Integration with formal school improvement planning
The evidence suggests that the SEED intervention contributed to a culture of evaluating practice, but its role was commonly spoken of in the context of supporting, or feeding into, existing formal structures of school improvement planning:
So, I think what we did was look to see what are we doing in school, but what was highlighted in the [SEED] report, and almost we kinda triangulated them and said, and what was already in our school improvement … or what was gonna be in our school improvement plan. So, they all kind of worked together.
SchIC2P, HT interview, T1
Because it is about, well, how do you know? And [SEED] gave us concrete information and data that, well, we were then able to use that to inform our next improvement plan … We don’t at this point have a great health and well-being programme of study. And that was really shown up through the work that we’ve done with you so far. So that’s our – practically everything that’s in our improvement plan for next year [has] stemmed from your work.
SchIA2P, HT interview, T1
Indeed, some schools saw the SEED intervention as integral to the cyclical process of school improvement planning:
[SEED] really needs to be part of the school improvement planning process. It really needs to be, it’s got to be a high priority.
SchIC3P, HT interview, T1
The SEED action planning process appeared to complement established improvement planning. In Scotland, schools create a formal school improvement plan that is submitted to their LA. Together, these processes had the potential to provide a focus and structure for addressing SEW issues and streamlining practice:
[SEED]’s made … how we work probably smoother, better, less chance of people falling through the gaps.
SchIC2P, HT interview, T1
However, the alignment of the SEED intervention to schools’ improvement plans was limited, partly because of the timing of the SEED cycles. Not all LAs adopted annual improvement planning cycles, and if the SEED APs did not coincide with the production of improvement plans, this may have limited their impact. Furthermore, although most schools referred to the SEED intervention contributing to the development of improvement plans, only a few HTs reported explicitly integrating their APs into their school improvement plan. Thus, in terms of where the SEED intervention added value to existing improvement planning processes, the evidence points to the role of the data to aid discussion and reflection. The structure of the SEED AP may have added value though, as a small number of respondents reported perceived benefits in looking at actions in the short, medium and long terms:
We looked at [goals] both for short term, the medium term, the longer term. I felt that was quite good, actually, to actually be thinking of that thing in terms of … thinking about the cycle of schools.
SchIA1P, EP interview, T3
Implementation of activities at whole-school and curricular levels
There was evidence that there were tangible and long-term changes to activities implemented both at a whole-school level and within the curriculum, although these were discussed less frequently in interviews with key respondents than the more cultural changes outlined previously:
The very fact that we have a core [SEW] programme in place that’s delivered, can be partly attributed to SEED as well.
SchIC2P, HT interview, T3
Some of the things that we did through the work with SEED are already part of the school now. I mean we looked at our entire health and well-being programmes of study because of the project. They are, they are changed. So that is forever going to be part of the school.
SchIA2P, HT interview, T3
Thus, the SEED intervention seemed to facilitate the embedding of SEW activities into the curriculum. In addition to activities focusing on pupil well-being, some respondents talked of concrete activities implemented to improve staff well-being resulting from the SEED intervention:
SEED raised issues around staff relationships – they requested more social meetings and we do this now, and have team-building days … At the last [SEED] feedback there were questions about staff morale … staff feel more valued, are working together.
SchIB1P, notes from HT interview, T3
However, it seemed easier to address pupil issues than staff issues:
Management reaction to the data shared after each [school pupil and staff] survey has been confrontational with no obvious effort made to improve issues affecting staff well-being. On the other hand lots of thought has been given to the pupils so some positive impact.
SchIC3P, staff questionnaire, free-text comments, T3
It is also noteworthy that different schools appeared to adopt different approaches to activities implemented, depending on their own circumstances, underlining the benefits of co-producing activities rather than imposing a one-size-fits-all approach. The following quotation is from an EP who worked in two SEED schools:
My experience of [SEED] in the two schools, it was quite different in that [SchIB1P] went for more of a bigger focus on particular areas that they did as a whole school. You know, so they’ve looked at their outdoor learning environment, they’ve looked at growth mindset, and they’ve done that as a whole school. So, [HT]’s really embedded that within assemblies, you know, teachers reinforce that all the time. They’ve developed a language across the school … And maybe that’s where the impact’s come in. I don’t know whether it will be picked up or not, in the tool, but, you know, certainly anecdotally, [the secondary school] are saying they’re noticing a difference in terms of the children coming from [SchIB1P] … Whereas I think [SchIB4P], my experience was that they went for more small, practical things, you know, like buddy benches in the playground, identifying – you know, children didn’t know where the enhanced support area was, so, putting footprints on the floor and all that kind of thing.
SchIB1P, EP interview, T3
Some respondents reported that involvement in the SEED intervention led to more staff being interested in, and wanting to participate in, relevant existing training opportunities through an increased awareness of SEW:
[Being involved in SEED] has certainly maybe encouraged them [staff], you know, there’s two or three of us who’ve been in quite … intensive like nurture training with some educational psychologists and things like this and more want to do it this year. So I think it has made staff realise, you know, this is going to help.
SchIC5P, HT interview, T3
For some schools, however, the lack of resources or clear guidance as to how to address issues raised as part of the SEED intervention was a barrier to selection and implementation of whole-school or curricular initiatives. The following quotations represent similar views from a staff member and the EP of a school that faced significant challenges, including a socially and economically deprived catchment area, a high caseload of pupils with emotional and behavioural needs and multiple changes in senior management. They suggest that it is very difficult to change a culture and work at a whole-school, ‘systemic’ level when the needs of individual pupils and caseloads are so high:
I feel we have spent a lot of time completing questionnaires and discussing/analysing results. More practical help to address issues would have been beneficial.
SchIA8P, staff questionnaire, free-text comments, T1
They’ve got just so many children with emotional and behavioural needs that … it’s kind of firefighting, and I’m always exhausted when I come out because they throw so much at you because they need to. They need to offload themselves and get advice and so they are very case-driven, but I know the reason for it as well and I can’t really blame them for it or … I try and steer them away sometimes, or have systemic stuff as well, but it does always tend to revert back, I’ve got to say, to individual cases.
SchIA8P, EP interview, T3
Relationships
The evidence from the qualitative data of direct impact on whole-school relationships resulting from the SEED intervention was sparse; however, there was tentative evidence that pupil relationships improved resulting from activities implemented because of the SEED intervention [see Improving pupil social and emotional well-being through school connectedness, relationships and embedding social and emotional well-being activities into the curriculum (quadrant 3: short-term outcomes for children)]:
When I was looking at the [SEED] feedback, one of the things on the first page was that the support staff had a lot of problems with pupils in the playground … It used to be that we could have two or three football games running concurrently and there was a lot of behaviour difficulties associated with that with foul play and things like that. Well, we now only have football on 1 day a week for each class … we have had almost practically no arguments about football, I mean, it’s dramatically changed the behaviour and I think that that is dramatically then having an impact on behaviour out in the playground altogether.
SchIC2P, HT interview, T1
There was slightly more evidence of improved relationships between staff in some schools:
The questionnaire that you put out at the beginning was in sections and some of it was about staff relationships and our relationships with each other and as well as being with the pupils, so because it broke it down it meant that we could see where we weren’t doing very well, and then focus on that, and then the second time you came with the results, we’d gone off the chart on some of the feedback, you know, so that was really pleasing’cause everyone made a real effort.
SchIA3P, HT interview, T3
The mechanisms used appeared to be both the implementation of concrete activities (above) and an increased openness about addressing SEW issues:
More [staff] think about well-being because of [SEED] and may be more willing to discuss their feelings.
SchIA1P, staff questionnaire free-text comments, T3
I have noticed that Senior Management are now taking staff health and well-being more seriously and are far more approachable when I have a concern or feel ill.
SchIC2P, staff questionnaire free-text comments, T3
The SEED intervention also had a role in creating an ethos of discussion and dialogue within the school and in making staff feel more valued:
You [SEED] helped to create an environment where people felt they were able to express their opinions then, and it created points of discussion, even with the kids, you know, in both areas. It enabled it, it opened up dialogue to be had, you know, in an easier way.
SchIC3P, class teacher interview, T3
The support staff feel happier. They feel they’re more involved in decision-making and being invited out to things [explicitly in the context of support staff having been involved in the SEED intervention].
SchIA3P, HT interview, T1
The importance of relationships in creating a positive school ethos and underpinning learning was highlighted frequently in key respondent interviews. It is noteworthy, however, that, although staff relationships were often cited as a focus of attention or a priority for action following the SEED RD sessions, there were limitations to the process’s ability to address these:
The process has shone a spotlight on a huge problem with staff relationships.
SchIC3P, RD observation notes, T3
Good to bring issues to the forefront but often not acted upon. I would like to see issues tackled re staff.
SchIA8P, RD observation notes, T2
Study effects: the experiences of control schools
Examining the experiences of control schools can highlight potential effects of participating in the research that may contaminate and dilute the effects of the intervention. Around half of the HTs in control schools that took part in an interview said that participation in the SEED RCT had an impact on their approach to SEW. This was supported by a small number of qualitative responses from the staff questionnaires.
Raised awareness of and sensitivity among school staff towards social and emotional well-being
The following quotations from control school HTs are in response to the question ‘Do you think that being involved in SEED will have, or has had, any impact on how the school prioritises social and emotional well-being?’:
I think probably a bit, purely because it certainly, for me, raised my … it kind of brought it to the forefront of my thoughts for a good while and I think it probably triggered in some way, gently, you know, moving to things like Rights Respecting Schools […] even although, obviously, we don’t know the results of things, the questionnaires, I think even doing that for the staff, the questionnaires in particular, I think would maybe make some of them just think, you know? I’m not sure it would have a long-term effect, but at that point it definitely, I think, made some of them kind of think ‘right, how do I interact with this?’ … ‘And how do I communicate?’. And that’s got to be a good thing.
SchCA1P, HT interview, T1
This has been a very worthwhile project which has kept us focused on promoting the emotional health and well-being of our pupils.
SchCC4P, staff questionnaire free-text comments, T3
Evaluating practice
There was evidence that some control schools used the data to reflect on their practice in a way similar to that intended in the SEED programme theory, although it was not clear if, or how, any increased reflection on practice translated into action:
When we were looking at your questions, we were saying, ‘well how does that fit in to our programme for our social and emotional well-being within the school?’. What could we do better? What are we doing really well? And actually, if we’re doing it really well, how are we going to keep that momentum up?
SchCA7P, HT interview, T3
Lack of impact
On the other hand, around half of the HTs from control schools reported that the SEED RCT had no impact on their perceptions, priorities or actions towards promoting pupils’ SEW. Many schools made reference to ‘doing it anyway’. The following quotations from control school HTs are in response to the question ‘Do you think that being involved in SEED has made any difference in terms of how you prioritise social and emotional well-being?’:
No. I don’t think so, no. I think it might have been different if we had been one of the schools you were working with, but as one of the control groups, no, because obviously we saw the importance of it anyway.
SchCA5P, HT interview, T1
I don’t think so to be honest … that’s something we would’ve been doing anyway.
SchCC5P, HT interview, T1
Modelling social and emotional well-being assessment
Three of the control school HTs mentioned that being involved in the SEED surveys heightened their appreciation of how to gather meaningful data on SEW. In one school there was an indication that this extended to using the SEED questions as a model to develop their own assessment tools:
We are surveying our primary 5s, 6s and 7s at the moment, about how they feel about school and, you know, their readiness to learn and, you know, if they feel the need to be able to learn, and who supports them in the school … So we’ve been able to use the practice that you had shown in the questionnaires. Not taking any questions word for word, but it’s certainly just given us a wee bit of food for thought.
SchCA3P, HT interview, T3
Possible benefits for pupils of completing the questionnaires were raised in Pupil experiences of the Social and Emotional Education and Development intervention. It should be recognised here that any impact of completing questionnaires on pupils’ SEW applies to both arms of the trial.
Contextual factors
Reasons for participation in the Social and Emotional Education and Development trial
In interviews, there were three main reasons cited as to why the decision was taken to participate in the SEED RCT.
Fit with existing priorities, ongoing work or personal interest
The SEED intervention was often aligned with a school’s existing work on SEW or general ethos:
The main thing was, it came and we thought, you know, this is exactly, you know, the kinda thing we talk about and think about.
SchCC3P, HT interview, T1
It’s the kind of thing I am interested in personally and professionally … so I would’ve liked the fact that it [SEED] was looking at emotional well-being of the children.
SchCC2P, HT interview, T1
The salience of SEW in the national curriculum (CfE) and the need to evidence progress in SEW outcomes also seemed to be important factors in deciding to take part:
And health and well-being, because that had become one of the, you know, literacy, numeracy, health and well-being, responsibility of all and we’re all going ‘Oh, what’s that going to mean?’ and we were starting to look up a curriculum and how we were embedding it and then we get the sort of contact letter. You know, ‘Do you want to be involved in this health and well-being project?’. Everyone was like ‘Yeah!’.’ Cause that’ll help us tick boxes …
SchIA3P, HT interview, T1
Support for addressing a recognised need to improve pupils’ social and emotional well-being
Others identified more specific issues around SEW within their schools and a hope that the SEED intervention could help address these:
We had concerns at that time about the sort of [social, emotional and behavioural] issues in terms of a lot of the children. But we really just didn’t feel we had enough information to go on.
SchIA5P, HT interview, T1
I was very keen to take part in this because a lot of the children don’t know how to express themselves, express their feelings, and use their hands and feet to lash out.
SchIC1P, HT interview, T1
The provision of data to support existing work
The element of data collection within the SEED intervention appealed to HTs, in its ability to assess the needs of pupils and, to a lesser extent, staff:
I kinda wanted … for people to come in and sorta say to us … validate where we are and maybe give us some ideas on how we could sort of improve further. Plus the anonymous audit side of it is obviously very … is always interesting because you can think everybody’s with you and you’re all singing from the same hymn sheet, but sometimes you’re not. So, that kind of anonymous audit, I think, was really useful too. I also was really interested to chart the progress of children through the school.
SchCC4P, HT interview, T1
The SEED data were also seen as a way to augment or add value to existing surveys, or to validate existing concerns/issues:
I think the self-evaluation that we had been doing had been more along the lines of looking at learning and teaching, but we also felt that, as far as the pastoral side of life in the school was concerned, we wanted to know as much as we could. We had done surveys with parents, with staff, with pupils, on a kind of … roughly a, kind of, 2- to 3-yearly basis, as part of our improvement planning process. And that had been going on for quite a wee while, but when you came along, we thought that this was really going to take it to a totally different level.
SchIA6P, HT interview, T1
Less commonly cited reasons included the financial ‘thank you’ for their time and effort (all schools were offered £1000 for taking part) and raising awareness of the importance of, and knowledge about, SEW with school staff. The decision to take part in the SEED RCT was usually taken at a HT/senior management team level although there was some evidence of wider staff being consulted (at staff meetings) before a final decision was taken.
School-level barriers and facilitators
Cultural readiness for change and school ethos
One of the most important contextual factors appeared to be a readiness to change, largely in the control of leadership or senior management, particularly when there were potentially difficult issues to address:
I’d just come through a not very good inspection, and part of the inspection where they failed badly on was relationships, so it was a bit of a raw nerve. And then when the SEED thing came along it meant that I had a context, which is why I was really for it as well, it gave me a context for building relationships that was outwith HMI [Her Majesty’s Inspectorate of Education], so, but yeah, they [staff] did find it really hard and that’s what I mean.
SchIA3P, HT interview, T3
This suggests that the SEED intervention worked best when work was already under way to initiate change:
I think that the SEED information – although a wee bit painful – was actually, probably, just able to bring … It helped to continue to bring staff together. But it was within a context where other things were happening, if you like, really.
SchIA1P, EP interview, T3
In almost all schools, there was evidence that there was a positive pre-existing culture and ethos within the school, which may have shaped the ability of the SEED intervention to affect the organisational culture:
It was generally felt that the school ethos and positive atmosphere was an area that the school was doing well in and was contributed to by staff relationships, modelling of behaviour and motivational and supportive attitudes toward pupils and knowledge of individual pupil background and needs.
SchIB2P, RD observation notes, T1
In contrast, respondents from a minority of schools talked about difficulties or challenges within the school ethos. A perceived lack of direction and a lack of openness and communication between staff could be barriers to fully engaging in the SEED process, as could resistance to the SEED process itself:
A group of young teachers/classroom assistants … went on to discuss that problems had arisen the previous year because there had been a culture of ‘no discussion’ and ‘no feedback’. There was also some concern about how SEED interventions would be implemented within the school because ‘we sometimes launch into new things with no drive and no direction’ … It generally felt that the staff group weren’t ready, as a whole, to really scrutinise the issues raised and commit to meaningful change.
SchIA8P, RD observation notes, T1
Part of the barrier was the staff. People’s different attitudes towards it. There was also a few people who didn’t understand the value and see the value in doing it, so, again, if people are feeling that, have a negative attitude towards it, it’s harder to get them on board.
SchIC3P, HT interview, T3
Although the evidence suggests that the willingness and/or ability to change the culture is a prerequisite for receptiveness to, and engagement with, the SEED intervention, it should be noted that this is not necessarily easy to create. One EP who was involved with three SEED intervention schools stressed that change is ‘professionally challenging’ and felt that this was only really successful in one out of the three schools that they worked in. This willingness to change is interlinked with factors within the school and with demands from district and national education bodies, as explored further in this chapter.
Relationships with educational psychologists
The relationships between the HT (or key contact), school staff and EP could also be seen to directly affect the implementation of the SEED intervention by laying the groundwork for change and facilitating difficult conversations. One EP stated that this was unlikely to happen without a strong relationship characterised by trust and experience:
Had you not had an involved psychologist on the ground who knew the school well, who knew the background of the school, I think if you’d just had someone new in post, or you’d had someone that was unfamiliar, who didn’t have the trust level that I had with the school, and with individuals within the school, that I could go in, have a cup of tea with them, and then have a difficult conversation with them …
SchIA3P, EP interview, T3
As with teaching staff, in some LAs there was a shortage and high turnover of EPs, which could affect the development of these relationships with the school:
This’ll be the third educational psychologist in 3 years … when you only see somebody for half a morning once a month, it takes a while to build up these relationships.
SchIA7P, HT interview, T1
In other cases, there was a barrier to EPs optimising their role in the SEED process when the school was reluctant to use what was perceived as precious caseload time for SEED-related, whole-school working:
I have to say that, in these needy schools, it reverts back to casework quite quickly, and some of the senior management staff in the schools can be quite vocal about that, because that’s where they see us making a difference, whereas we’re maybe having the view that we can make more of a, kind of, long-term difference, if we’re doing that less direct work.
SchIA8P, EP interview, T3
There may also have been a lack of clarity in terms of expectations of the EP’s role of taking the SEED intervention forward in schools. This appeared more likely if the SEED link person was not the EP’s regular contact in the school or if there was a lack of continuity in the senior management team causing disruption to the longevity and implementation of the AP. Two EPs also made comments to suggest that the SEED research team could have done more to encourage this interaction by meeting more regularly:
I know there’s time elements, travelling and whatnot, but something that might have been helpful was, you know, for someone from the SEED team, the educational psychologist, and the head teachers, to get together.
SchIB1P, EP interview, T3
Staff and pupil turnover
Head teachers repeatedly spoke of the problem of staff turnover throughout the study. In one school, over the life of the SEED RCT, there was practically a complete change of staff, which appeared to affect the ability to engage with the SEED process, as well as with curricular developments:
SchIA2P, HT interview, T3: [Over] these last 4 or 5 years … we’ve had so many staff changes that it really has been, we’ve got like practically a new school.
Interviewer: What challenges does that present to you?
SchIA2P, HT interview, T3: Well it means that, in some ways, you’re starting again with people maybe come in, and they’ve got a different view … of – of nurture and attachment and health and well-being, and you’re kind of starting over again.
However, this staff turnover was also perceived as a facilitator, as younger staff were more likely to be receptive to a strong focus on health and well-being. This links in with a shift in national priorities for SEW, discussed further below:
We have a totally new staff from when you started, no disrespect to the previous staff, but they were, you know, they were very experienced, they’ve been teaching a long time, and the focus on mental health and well-being being a bit more formal, was quite different to them, whereas the younger staff that I’ve got now, to have that focus on emotional and mental health and well-being, it’s just seen as the norm … I think the training at uni[versity] is obviously a lot more child-centred.
SchIA3P, HT interview, T3
Pupil turnover was also seen as a barrier to meaningful interpretation of changes in the data over time:
[SEED] was useful to a point, but the biggest issue is to do with consistency. Over the life of the study the school has seen huge changes. There have been changes in staffing and constant pupil changes. The overall roll has increased, lots of pupils are leaving, more classes starting at P1, composite/straights are changing so much – the general dynamics of the classroom set-up and class structure changes mean that looking at data across different time points doesn’t give a true reflection of changes over time.
SchIB1P, notes from interview with HT, T3
Time and resources
Lack of time was universally mentioned as a potential barrier to SEED intervention implementation. This was often seen in the context of staffing and resource cuts. This highlights the vulnerability of the SEED intervention in terms of time assigned to it, and raises implications for its future sustainability if not seen as a priority:
Time is something that we discuss all the time, and we’re having to make cuts and … yeah, I mean, I suppose very realistically, those [projects like SEED] are the things to go because they’re not the absolute have-tos, even though we all know how they’re going to make a difference long term.
SchIA8P, EP interview, T3
Time was also seen as a barrier to implementing the specific elements of the SEED RCT, particularly running RD sessions:
There are very, very few occasions in a session in the life of a school, where you can get – especially in a school our size – where you can get everybody together. To find a time where there are no kids in the school, and when nobody is away out at meetings, and everybody can be there – it’s pretty difficult. Not so much, I would have to say, with regards to the children and with regards to coming into the class to do the surveys and that kind of thing. Even as far as the teachers are concerned, with regard to the, you know, the questionnaires. They have a big workload.
SchIA6P, HT interview, T3
Resource and staff cuts also placed limitations of the selection of activities in response to the needs assessment:
No point in it really nothing is ever done about anything as there is no money.
SchCB4P, staff questionnaire free-text comments, T3
The school is trying to access more support for nurture. But there are staffing issues – ‘we need to think about how to do this without staff resources’.
SchIA8P, RD observation notes, T1
Local authority-/national-level facilitators and barriers
Curriculum for Excellence
The SEED programme was developed within the policy context of the CfE. As discussed previously, schools participating in the trial were frequently motivated to do so by the need to meet the CfE’s requirements to address SEW. Even if this was not the initial primary motivation, the national shift in focus in education towards prioritising SEW within the CfE meant that the SEED programme was viewed as a good fit with policy and the requirements of the curriculum, thus facilitating its implementation:
Along with the SEED we were also kind of getting our heads round GIRFEC [Getting it Right for Every Child] and SHANARRI [safe, healthy, achieving, nurtured, active, respected, responsible and included] and I think the two things were very similar. You know, if you think in the SHANARRI indicators … I think the two things [SEED and SHANARRI] were kind of hand in hand and we needed, as a school, to, you know, be getting much more familiar with safe, healthy, nurtured, respected, you know?
SchIA7P, HT interview, T1
Thus, the SEED programme was seen as being of direct value in helping schools meet CfE requirements to place health and well-being on an equal footing with literacy and numeracy:
With everything that’s happening now with the national improvement framework, you know, there’s a very high emphasis on health and well-being. And I think that because of the work that we’ve done with SEED, we’re a wee bit ahead of the game. Because we have put so much thought and effort in to improving our health and well-being education, due to the work that we’ve done with yourselves, you know – we’re in a good place as far as that goes.
SchIA2P, HT interview, T3
There was also evidence in control schools of a change in approach to SEW over the life of the trial that resulted from the increased priority given to SEW in the school’s wider cultural and political context:
We have done things differently since we started being a control school because it was on my plan to do things differently anyway … It was a priority, and when we, you know, were chosen as a control school, we just kinda got on and have done things ourselves and continued to build on that.
SchCC6P, HT interview, T3
Local authority priorities
Inclusion of SEED activity in the school improvement plan was critical both in providing space within the curriculum for SEED activities and in legitimising SEED activity itself. However this raises questions about sustainability if social and emotional development slips down LA priorities:
Obviously SEED was a major part of our improvement plan the year that we started. And people at headquarters … I don’t think they would be impressed by seeing the same thing year after year … And we wouldn’t be able to give it the time it required, if it wasn’t part of our improvement plan.
SchIA2P, HT interview, T3
If you got quite a number of, if you like, council priorities coming on you … you have to take this initiative forward or that initiative forward, then that would influence the ability of a school to be able to spend the time on a bespoke research project like [SEED]. The bespoke research project then becomes almost more remote from the day-to-day reality, as to what a head is taking forward.
SchIB1P, EP interview, T3
Staff from several schools talked about using the SEED data in conjunction with other data, gathered at either a LA or an individual school level. Although the SEED data were felt to be useful in validating perceptions of the school supporting improvement planning, further evidence suggests that other data sources may compete with, and lessen the impact of, SEED data:
Well, I think around the same time as SEED the council paid for the [regionally adopted] survey. They gathered information from young people and from parents, and used that to shape the local plans. I suppose that’s taken over a bit for schools in relation to the SEED data. I think if we could think of a way of triangulating that data so that people are seeing the link, it would make what you guys have been doing even more powerful.
SchIB1P, EP interview, T3
Integration of qualitative and quantitative findings on process
The next section combines a discussion of the aforementioned qualitative themes with additional quantitative analyses (see Appendix 5, Table 55) to examine the fidelity of intervention delivery and the evidence for the multiple pathways proposed in the SEED intervention’s theory of change. It is recognised that process evaluations are not able to evaluate all causal assumptions;118 this, coupled with variability in the availability and quality of data, means that only selected pathways are explored here. The discussion below is structured into quadrants 1–3 of the SEED intervention’s theory of change (i.e. inputs, activities and outputs, and short-term outcomes for children) (see Figure 2).
Raising awareness of social and emotional well-being and fostering a collective commitment towards addressing needs (quadrant 1: inputs)
Pupil assessment
In all schools, comprehensive data were collected from both cohorts of pupils at each wave, indicating ongoing support and commitment to the trial from schools in both arms of the trial from the start. Only one intervention school dropped out completely, shortly after baseline data collection, citing lack of time. There were a few times when it was not possible to collect some elements of the data from schools at individual time points. Two primary schools declined to complete the teacher-completed SDQs at T2, one because of severe but temporary staffing cuts, the other because a temporary HT was in place. In another intervention school that was particularly hard to engage, three changes in HT over the life of the trial led to being unable to collect staff-, parent- or teacher-completed SDQ data at T2. The qualitative data suggested that the process of completing teacher-completed SDQs was not onerous, however, and overall response rates for the teacher-completed SDQ were strong throughout the trial: 95% of the total eligible sample at T0, 98% at T1, 95% at T2 and 93% at T3.
For the older pupils who self-completed the SEED pupil questionnaire, return rates were not as high, but remained very respectable, following transition to secondary school between T1 and T2. Of the total eligible sample, 92% completed the questionnaire at T0, 97% at T1, 75% at T2 and 88% at T3. The drop at T2 was because three secondary schools did not agree to participate; these schools did come on board for T3.
Teaching and non-teaching staff completed questionnaires in all schools at all waves, apart from the exception detailed above and one further school that did not return staff questionnaires. At T0 the average staff return rate was 60%, ranging from 17% to 96% across individual schools. The average was 53% (range 20–95%) at T1, 46% (range 0–94%) at T2 and 44% (range 0–89%) at T3. There was no noticeable difference in average return rates between control and intervention schools. This demonstrates a marked difference between school staff in their willingness or ability to complete the SEED survey, including the length of survey and time administered (see Collection of Social and Emotional Education and Development data) and the embeddedness of the SEED RCT within the school (see School-level barriers and facilitators). The fall in overall completion rates suggests a degree of questionnaire fatigue over the life of the trial.
Parent response rates were lower than our cautious estimate, varying from an average of 34% at T0 (ranging from 17% to 70% across individual schools) to 20% (OC) and 27% (YC) at T3. This meant that there was limited or no discussion of parent data in many of the RD sessions.
Therefore, the data collection element of the intervention was implemented with fidelity, but participation and retention rates for staff and parents meant that this element of the intervention was less successful than the collection of pupil data.
Data analysis and feedback to schools in presentation and reflective discussion sessions
The data analysis was completed as intended and schools were presented with reports giving individual item-level feedback on T0 data and domain-level data showing changes over time for later waves. There were logistical challenges in preparing reports quickly following data collection, which affected scheduling of RD sessions in some instances. This was particularly difficult in later waves when the data collection and feedback cycle was yearly, which gives credence to the notion that a two-yearly data collection and feedback cycle would be more acceptable to schools.
Educational psychologists were invited to all data feedback and RD events, and across all LAs they attended over half the sessions, although less so in later waves. The main explanation was not being able to prioritise this over other commitments, primarily individual caseloads. In about two-thirds of schools, support staff, as well as teaching staff, participated in RDs. Support staff found it difficult to attend because they worked fewer hours than teaching staff and did not have as much allocated staff development time. It was hoped that QIOs would participate in RD sessions, but their role was peripheral. Only in a very small number of RD sessions were they present, and there was no evidence that they contributed to the writing or implementation of APs.
In the theory of change, it was proposed that the RD sessions would provide an opportunity to discuss SEW issues, leading to a greater awareness of pupil and staff needs, improved collegiate relationships and a collective commitment and improved motivations to tackle and prevent problems. In practice, the SEED programme provided a clear, and often rare, platform, through the discussion of data, to discuss issues around SEW, clearly leading to both a greater awareness of issues and improved collegiate relationships (see Mechanisms for change). This may also have led to a collective commitment to tackle and prevent problems, but the lack of all-staff involvement following RD sessions suggests that this was less important.
This speaks to the socioecological model,76 in particular at the individual (raising individuals’ awareness of SEW issues) and interpersonal (engagement of staff in discussions) levels. It is less clear if this pathway worked at the organisational level. This is also in line with principles of co-production79 and the framing of a programme within the community.
It is possible that elements within this first quadrant were evident in control schools, suggesting possible contamination. Approximately half of respondents from control schools did suggest that participating in the SEED data collection may have had unintended benefits (see Study effects: the experiences of control schools), which mirrored the mechanisms of change outlined in Mechanisms for change). In particular, completing the SEED staff questionnaire appeared to raise awareness of the importance of SEW and to keep in mind the importance of evaluating practice. Furthermore, pupils reported potential benefits from completing the questionnaires, which may have affected their well-being (see Pupil experiences of the Social and Emotional Education and Development intervention). Although it is difficult to draw firm conclusions on the basis of the limited qualitative data presented here, there is a possibility that completion of the SEED questionnaires has a therapeutic benefit for pupils that could dilute the observable impact of the SEED intervention in the intervention group.
The first quadrant of the theory of change was well evidenced, but there was less evidence to support the pathways from this to the identification, selection and implementation of suitable initiatives.
Identifying suitable activities and implementing initiatives (quadrant 2: activities and outputs)
Identifying the most suitable and effective activities
There is good evidence (see Assessing the school’s social and emotional well-being profile) that appropriate suggestions to address need were raised at the RD stage; however, the mechanisms by which these suggestions were translated into implementation of actions is much less clear. For instance, the process of completing APs was carried out as intended and did not seem to be a difficult task for schools (themes 2.3 and 3.4: development and maintenance of APs, and supporting decision-making); however, there are doubts as to whether or not the function of the AP was as intended.
To explore this, we examined the coherence between the summaries of the RD sessions (prepared by the research team for the schools) and the schools’ SEED APs. The analysis judged whether or not schools wrote actions into their plans that addressed the issues they had raised in the RD sessions, rating this on a four-point scale from ‘good’ to ‘very poor’. Out of all 18 intervention schools, only four were rated as ‘good’. A further eight were rated as ‘fair’, three ‘poor’ and one ‘very poor’. It is important to note that this does not mean that the suggested actions themselves did not have the potential to improve staff and/or pupil well-being, but that most were not aligned to the conclusions reached at the data presentation and feedback discussions.
There may be several reasons for this. It is possible that the AP itself was not seen as an important document. This is supported by the lack of distinction between the SEED APs and other planning documents (see Development and maintenance of action plans and Supporting decision-making). A possible explanation for this is that SEED initiatives might have been incorporated into a school’s improvement plan, making a separate SEED AP redundant. Similarly, it is possible that the APs were written to support existing work or plans, rather than to look at any innovative or unexplored ways to promote SEW that may have been raised in RD sessions (theme 3.3: assessing the school’s SEW profile). A third possibility is that there was not enough time and support built into the SEED intervention to help schools translate possible actions into a tangible plan. This hypothesis is supported by some staff feeling overwhelmed by the number and detail of data presented (see Presentation of data and reflective discussion sessions).
The role of the resource guide is also relevant here. The SEED theory of change suggested that the resource guide would be an important link between identification of possible actions and selection of relevant evidence-based interventions. However, the qualitative findings suggest that the SEED resource guide had limited utility (see The Social and Emotional Education and Development resource guide) and that staff preferred to draw from known banks of resources. The relatively low use of the resource guide to select activities also suggests that the teachers did not really value the guide’s assessment of how well evidenced the effectiveness of programmes was. For those who did select initiatives from the resource guide, these may have been one entry in the AP alongside several other activities that may or may not have been linked to discussions in the RD session. This could explain why an AP might not be coherent with discussion in the RD sessions, but still address an important need identified in the feedback process.
The lack of congruence between RD sessions and APs also raises the possibility that the types of initiatives written into APs were not the mechanism for change, and that the cultural benefits of creating space for discussion (see Space/time for professional dialogue and reflection), coupled with small, whole-school changes in ethos and relationships (see Relationships), were the drivers to improve well-being. The implications of this would be that the formal role of action planning is peripheral to the elements in the first quadrant of the SEED programme’s theory of change.
Improved curricular and whole-school initiatives
The qualitative data provide limited evidence on the type and extent of initiatives implemented as a direct result of the SEED intervention (see Implementation of activities at whole-school and curricular levels). To help explore this pathway further, we analysed how the intervention and control schools differed in terms of the SEW activities and initiatives they adopted over the trial period, using staff questionnaire responses at T0, T2 and T3. These surveys included questions about school activities, programmes or initiatives aimed at promoting SEW for pupils and staff. Initiatives were coded as follows: (1) included in the SEED resource guide, (2) other known SEW initiatives (not included in the resource guide because of a lack of evidence, theoretical framework or specificity to a LA), (3) non-SEW-specific health initiatives or (4) non-SEW/health initiatives.
A Wilcoxon signed-rank test was carried out, and no significant difference was found between the two arms of the trial at baseline, either in the number of SEW initiatives (coded 1 and 2) reported in total or the number of resource guide initiatives (coded 1). Similarly, there was no statistically significant difference between the number of initiatives reported by the two arms of the trial at T3, suggesting that intervention schools did not report doing or adopting more SEWactivities following implementation of the SEED intervention than control schools. The implication of this is that the implementation of activities to address SEW was not the key mechanism by which SEW was improved.
We further analysed whether or not intervention schools increased their reporting of SEW initiatives over the life of the trial more than control schools. Although there was no significant difference between the total number of resources used or the number of resource guide initiatives for intervention schools between T0 and T3, there were significantly fewer resources in total (p = 0.034) and significantly fewer resource guide initiatives reported (p = 0.041) between T0 and T3 for control schools. It is difficult to account for a drop in reporting of resources in the control group. It is possible that, if resource use generally dropped across all schools (perhaps as a result of national or regional priorities), the SEED RCT may have protected against this. It may be more likely, however, that being in the control group reduced motivation to complete this section of the questionnaire in full as the trial progressed. Although there was a variable response rate for the staff questionnaire, this did not vary significantly between arms of the trial and so was unlikely to influence these results.
Staff were also asked to rate the amount of each specified resource used (‘a few exercises’, ‘about half’, ‘most of it’, ‘the whole package’) and state how carefully they followed the programme (‘followed carefully’, ‘some adaptations’, ‘considerable adaptations’). Preliminary analysis of these data at T3 showed a statistical trend (p = 0.072) that more intervention staff reported using all or most of at least one classroom-based SEW resource/package than control staff (40% vs. 38%). Similarly, there was a trend that 16% of intervention staff reported following at least one package carefully, whereas 14% of control staff did so (p = 0.09). These are only tentative findings, but could suggest that staff in intervention schools may not have used more resources, but those that they did use, they used with greater fidelity.
Staff were asked in their questionnaires to say how often they or other school staff used or engaged in each resource (‘rarely’, ‘occasionally’, ‘at least weekly’). At T3, there was no significant difference between arms in how often staff reported themselves or their colleagues using the SEW resources. Staff were also asked how often they engaged in staff well-being initiatives available to them in their school (‘once’, ‘more than once’, ‘a lot’) and how worthwhile each of them has been for their own well-being. There was a trend towards more use of staff SEW activities and resources in the intervention schools at T3, with a mean of 1.7 activities being reported by intervention school staff, compared with 1.3 in the control schools (p = 0.07). Finally, logistic regression showed that intervention school staff were significantly more likely than control school staff to find the staff activities/initiatives/resources worthwhile for their own SEW at T3: odds of 1.71 (95% CI 1.15 to 2.53). This provides further, tentative, evidence that it is not the quantity of resources implemented that is important, but the quality, relevance and fidelity of those resources.
The theory of change proposed a pathway to improved staff SEW and reduced stress via the selection of suitable activities and a collective commitment to tackling problems. However, there was no evidence that staff in the intervention arm experienced improvements in mental health and well-being or stress. There were no significant differences in scores on the Short Warwick–Edinburgh Mental Wellbeing Scale119 between the two trial arms at any of the follow-up visits [T1: 0.313 (95% CI –0.411 to 1.037; p = 0.397), T2: 0.593 (95% CI –0.156 to 1.343; p = 0.121) and T3: 0.556 (95% CI –0.18 to 1.291; p = 0.139)]. There were similar non-significant results for staff health, job stress, stress outside work and work–life balance. However, those in the intervention arm reported, at T1 only, more positive responses to ‘do you actively consider how to improve your own well-being in work?’ (p = 0.012), which resonates with the findings on staff finding activities worthwhile to improve their own SEWat work.
It is notable that the implementation of activities directed at improving SEW did not appear to displace other activities (see Displaced activities), although some HTs said it led to more efficient use of staff development time.
Improving pupil social and emotional well-being through school connectedness, relationships and embedding social and emotional well-being activities into the curriculum (quadrant 3: short-term outcomes for children)
Regulatory and instructional order
The theory of change proposed that pupils’ well-being, connectedness to school and stress could be influenced via improvements in a school’s ‘instructional’ and ‘regulatory orders’. 83 A school’s regulatory order refers to the way in which a school encourages norms of behaviour and belonging, and the instructional order refers to the formal and informal structures for learning.
The theory of change suggests that delivering classroom interventions should lead to an increased awareness among both staff and pupils of the importance of SEW and provision of space to practise SEW skills. There was minimal evidence from the main outcome data to suggest that pupils’ SEW skills improved for all pupils (apart from self-management), but there were significant differences for the older pupils between intervention and control arms for emotional regulation (p = 0.018), social awareness/ empathy (p = 0.025), self-management (p < 0.001) and responsible decision-making (p < 0.001).
It is difficult, from the available data, to determine the mechanisms for such change, but there seems to be more evidence for cultural and environmental changes than curricular ones. There is little evidence available (as opposed to weak evidence) to suggest that the SEED intervention helped to promote the rewarding of SEW skills, but there was evidence for a raised awareness of SEW among staff (see Raising awareness of, and increasing focus on, social and emotional well-being). There was also evidence for improved relationships, which may give support to the SEED intervention acting on the regulatory order via increased staff respect for, and interest in, pupils. However, this may have primarily been achieved via staff relationships: the available evidence for improved school connectedness among pupils was weaker. The evidence that the SEED intervention acted to improve relationships is explored in more detail next.
Improving relationships
There was a significant difference between arms for both cohorts of pupils at T3 for the domain ‘school relationships – whole’ (0.149, 95% CI 0.05 to 0.24; p = 0.001). This was made up of only two items: ‘children in this school are kind to each other’ and ‘most adults in this school are kind to pupils’, providing tentative evidence for improvement in staff–pupil and pupil–pupil relationships. However, there were non-significant differences for pupil reports of ‘happy friendships’ and ‘school relationships – class’ and perceptions of pupil–pupil relationships by staff. See Appendix 2, Domain summary, for a breakdown of items included in all domain scores.
There were significant improvements for several secondary staff outcomes (see Table 19), including school ethos (p < 0.001), staff–staff relationships (p = 0.011), staff–pupil relationships (p = 0.005) and staff support (p = 0.005). This supports the qualitative findings suggesting an improvement in staff relationships (see Relationships).
Cumulatively, this suggests a pathway for the SEED intervention to affect pupil outcomes through improvements in staff relationships and school ethos.
Embedding social and emotional well-being activities into the curriculum
In most schools, the SEED intervention was seen as integral to the cyclical process of school improvement planning and a useful way to structure SEW practices efficiently (see Supporting decision-making). The lack of strong evidence of improvements in curricular activities around SEW means that it is difficult to conclude that the SEED intervention worked as intended in modifying tailored activities. This further supports the suggestion made previously that the activities in the first quadrant are the key mechanisms leading to any improvements in practice, rather than the action planning process.
The Social and Emotional Education and Development intervention as a cycle
The SEED intervention was intended to function as an iterative process with yearly cycles of data collection, feedback and action planning. Following feedback on baseline data, no feedback sessions were planned for year 2 of the trial as this was intended to be when schools would focus on implementing their APs. It was the original intention that schools would have further RD sessions on T1 and T2 data, but in practice this was not always possible. Of the 18 intervention schools, only two had three cycles of data feedback including RD sessions. Four schools had only one cycle and the rest (n = 14) had two. When RD sessions were not possible, schools were offered electronic versions of their feedback to use in their own time. The main reasons as to why schools did not follow the expected cycle were practical limitations in finding time to schedule feedback sessions within a school calendar that has very limited staff collegiate time and a loss of commitment to the process due to staff changes and school dynamics. Evidence from EPs also suggests that more regular contact and meetings between EPs and schools would strengthen the ability to maintain the SEED intervention in schools for the longer term.
Conclusions
-
There was great variability in the degree to which the SEED intervention was implemented. The most consistently implemented elements were the data collection and the process of feeding back data to whole-school staff groups at RD sessions. There were fewer RD sessions than had been anticipated in most schools, implying that a 2-year rather than a 1-year cycle would be optimal.
-
There was a lack of congruence between the RD sessions and schools’ APs and minimal use of the SEED resource guide to select activities. Furthermore, there was no indication that intervention schools implemented more SEW initiatives than the control schools. The SEED data may have been overwhelming for schools, resulting in a lack of direction on what to aim for and/or too many possible priorities or actions to really make an observable difference.
-
The SEED intervention appeared to work as intended in its provision of the following: time to reflect on pupils’ and teachers’ SEW needs, an opportunity to discuss SEW and the fostering of a collective commitment to tackle and prevent problems. The SEED data were a crucial part of this process, but primarily acted to reinforce existing priorities, rather than invent new ones.
-
The SEED intervention did not appear to be instrumental in increasing curricular activity around SEW, although it may have increased the quality of activity. The SEED intervention may have improved the quality of initiatives related to staff SEW more than pupils’ SEW.
-
The SEED intervention contributed to a culture of evaluating practice. Action planning appeared to support and feed into existing school improvement planning cycles, rather than being seen as a distinct process. Although school staff were involved in this process, the bulk of action planning was carried out by senior staff members. EPs could play a central role in facilitating the SEED intervention, but there was variability in the extent to which this happened.
-
Existing relationships between staff, and with EPs, shaped the adoption of the SEED intervention. Relationships were also shown to be a widespread priority, and the SEED intervention seemed to improve school ethos and relationships between staff in some schools. However, only the quantitative data showed a marginal improvement in staff–pupil and pupil–pupil relationships. This may reflect a stronger pathway for improving the school’s ‘regulatory order’ than its ‘instructional order’.
-
The SEED intervention was seen to be of direct value in helping schools meet national curricular requirements to place health and well-being on an equal footing with literacy and numeracy (e.g. the Scottish Government’s CfE). The SEED intervention was, therefore, generally seen as complementary to, rather than competing against, other established initiatives, helping to embed health and well-being and other SEW programmes and policies in schools.
-
The school’s pre-existing climate and readiness for change were important in determining the degree of engagement with the SEED intervention and any likely improvements in outcome.
-
Outcomes in control schools may have been affected by involvement in the SEED trial, either through a raised awareness and sensitivity among school staff towards SEW or through the benefits of completing the questionnaires for pupils.
Chapter 7 Discussion and conclusions
This chapter summarises the key findings presented in the previous three chapters and considers their implications for future research, practice and policy.
Key findings
The prespecified primary outcome was the SDQ-TDS at T3 (4 years after baseline). This was analysed first using analysis of covariance, adjusted for baseline values, then using repeated measures models of data collected at all trial assessment points, either restricted to those pupils exposed to the intervention from the start of the trial or including any pupils who joined the schools during the trial period. Results from the three sets of analyses were similar, showing modest improvements in SDQ-TDSs at T3 for pupils attending intervention schools, compared with controls. There was no evidence of different intervention effects according to deprivation. Additional sensitivity analyses, using last observation carried forward or return to baseline to deal with missing outcomes, gave consistent results.
However, the intervention effects varied by cohort and gender. The intervention effect at T3 was restricted to the OC of pupils (starting at P5, aged 8–10 years), with none evident for the YC (starting at P1, aged 4–6 years). The pattern of intervention effects over time varied between cohorts, with an intervention benefit emerging over time among older pupils until T3, whereas, in the YC, there was an immediate intervention effect, but this disappeared at later time points. Intervention effects were also stronger for boys than for girls, particularly for the OC (effect sizes of 0.42 and 0.24 for boys and girls, respectively, at T3 for the OC). The balance of evidence suggested that the intervention effect was diminished at 6 years post baseline, T4, when pupils were facing national, external examinations.
Secondary analysis showed that the intervention had significant effects in the desired direction across all five SDQ subscales (evident at different time points in our trial and differential by gender) and demonstrated a range of positive effects on social and emotional skills, and school ethos and relationships. Only boys showed significant results for conduct problems and prosocial behaviour; all the other subscales showed effects for both genders. Even for the secondary and auxiliary results that did not reach significance, they were, with one exception, in a favourable direction. Both pupil and staff data showed modest improvements in school climate and relationships in the intervention arm at T2 and T3 for both pupils and staff, that is at least 2 years after baseline. This is in line with our expectations, as we did not anticipate being able to detect effects in < 2 years, as informed by our formative evaluation. There was some evidence of reduced alcohol and tobacco use among the intervention group at T3 and self-reported alcohol use at T4.
There was agreement between the outcome and economic evaluations of the SEED RCT. The economic evaluation concluded, with 88% probability, that the intervention was cost-effective at a £20,000 per QALY threshold and was robust to sensitivity analysis. NICE suggest that interventions costing < £20,000 per 1-unit increase in QALY are cost-effective,120 fitting with our interpretation. Furthermore, in keeping with the main outcomes, cost-effectiveness was particularly high for the OC, particularly for the older boys, and a reduction in the OC’s NHS costs is suggested. The economic evaluation’s sensitivity analysis was robust to the NHS cost savings and missing data, increasing our confidence in the findings.
The process evaluation showed that there was an appreciation for the timing of the SEED intervention in terms of its alignment to the Scottish CfE,16,121 and its ability to support the requirement to place health and well-being on an equal footing with literacy and numeracy. School staff understood the rationale of the SEED RCT and saw its aims as important, namely to improve pupil and staff SEW. School staff broadly cooperated with the data collection and engaged with the reflective feedback sessions, which were delivered by the research team in collaboration with EPs. Most found these sessions useful.
Strengths and limitations of the intervention and its evaluation
Intervention
The SEED intervention and its evaluation were timely in their strong fit with the CfE,16,121 which was a rationale for testing the concept in Scotland. The co-production and adaptability of the reflective process was seen as a strength by school staff. The ability of the SEED intervention to adapt to local needs suggests a likely fit with other policy contexts. The SEED intervention was seen as integral to the cyclical process of school improvement planning and as a useful way to structure SEW practices efficiently, suggesting intensiveness of integration into routine practice.
The process and outcome evaluations point to the importance of the quality of staff–pupil and staff–staff relationships and trust; however, we experienced a relatively high turnover of school staff, which may have reduced the efficacy of the intervention. Furthermore, resource and staffing cuts, and LAs’ competing demands on schools, inhibited implementation of SEW activities. Within LAs, the need to avoid ‘contamination’ of the control schools, and the temporary research status of the programme, prevented the SEED intervention from becoming embedded in LA structures.
The weakest element of the SEED intervention was the creation of co-produced APs. First, the APs did not always reflect the priorities identified in the RD sessions. Second, the plans made little use of the SEED resource guide to select evidence-based SEW interventions. Although some schools implemented those packages identified as having strong evidence, few schools used the resource guide to shape their AP. Third, parents or pupils were not involved in the production of APs, as had originally been intended. In some schools these were produced solely by senior staff. We had also hoped for greater pupil involvement in the action planning process; however, this was difficult as most schools chose staff in-service days for their RD sessions. Pupils are not present during in-service days.
On the other hand, engagement with the pupil needs assessment data motivated all staff to improve SEW in the schools. Specifically, the reflective element of the intervention supported existing improvement planning processes and contributed to a culture of evaluating practice. The SEED RCT gave space for professional dialogue in which whole-school activities were prioritised over individual-level programmes. The intervention schools offered a similar amount of SEW-related activities to the control schools, but the activities may have been selected and implemented with greater consensus and fidelity in the intervention arm.
Given that the SEED intervention encourages whole-school initiatives, it would have been helpful to involve all year groups of pupils, and this is supported by process evaluation findings concerning the representativeness of the data. However, that would have been costly in terms of time and money. We opted instead to recruit two cohorts of pupils starting at P1 (aged 4–6 years) and P5 (aged 8–10 years). The relatively small proportion of pupils surveyed could be addressed in future roll-out of the SEED intervention (see Sustainability and implications for roll-out).
Existing relationships between staff, and with the EP, shaped the adoption of the SEED intervention. Although all EPs supported the objectives of the SEED RCT, their intended role as a ‘critical friend’ worked best when this fitted with their current working relationship with the school, and when this did not conflict with existing individual caseload priorities. The contexts that facilitated the SEED intervention were closely related to relationships, for example strong leadership, readiness for change and good relationships with the school EP. Conversely, there were some school contexts in which the SEED intervention did not work as well, for example if the SEW needs of pupils were too high, if there were fractious staff relationships to start with or if there was a poor relationship with the EP. Thus, relationships both have predictive value in determining the success of the intervention and are an important mechanism for change.
Evaluation
The SEED questionnaires contain a rich range of reliable, validated scales and some novel items selected to ensure suitability for the ages of pupils included in the trial. These questionnaires served the dual purpose of a needs assessment for intervention schools and of providing outcome, process and economic measures for the evaluation, as recommended by MRC guidelines for the evaluation of complex interventions. 122
The three participating LAs had heterogeneous characteristics and demographics, strengthening the generalisability of our findings (see Generalisability). Overall, we had strong pupil retention and, despite inevitable attrition, the trial had sufficient statistical power. We anticipated attrition particularly around the transition from primary to secondary education, and so we over-recruited by one school in each arm to compensate for this, in addition to our expected 25% dropout. We allowed for school clustering within the statistical analysis. A further strength was that contamination was minimised by clustering at the school learning community level (the secondary school together with its primary feeder schools). EPs are allocated to learning communities, so, in general, each EP was associated with either the intervention or control arm only. Furthermore, in general, intervention and control primary school pupils graduated to their linked secondary schools.
The randomisation was conducted by a registered clinical trials unit, and demonstrated good balance between the arms of the trial (see Table 2) in terms of school demographics. The pupil-level (based on self-reported postcodes), not school-level, SIMD was the only measure by which arms differed. The intervention arm contained more pupils (self-reported) from deprived areas, which we would expect would decrease the chance of showing an effect, given that the literature suggests that interventions are less effective for deprived pupils. 123 In line with this, our intervention arm pupils started out with statistically significant higher SDQ-TDSs. Thus, there was no bias in favour of the SEED intervention and adjustment was made for deprivation in the outcome analyses. The trial was registered with the ISRCTN registry, and a protocol was available in the public domain before data were collected and analysed. There were no significant deviations from our protocol, although the process evaluation’s data collection was less comprehensive than intended.
There were, however, limitations to the evaluation. For instance, as mentioned previously, for time and cost reasons we involved only two cohorts of pupils, starting at P1 (aged 4–6 years) and P5 (aged 8–10 years), in the intervention and its evaluation. Given that the intervention focused on a whole-school approach, not involving pupils from every school year may have weakened the intervention and thus its evaluation. Furthermore, interpretation of T2 and T4 data is problematic because three schools temporarily withdrew from data collection; we believe that this was due to the burden of data collection, particularly at T4, when our pupils and schools were focusing on national external examinations. In addition, as mentioned previously, although we invited all staff to complete questionnaires at all time points, the high turnover of staff meant that we had to use those data cross-sectionally (rather than repeated measures longitudinally).
We anticipated a challenge in collecting data from parents, both for the intervention and its evaluation, as has been found in other studies,124 but the level of parental involvement was disappointing. This had two impacts. First, parent data were not sufficiently representative to give confidence in the analysis for parent-reported outcomes. Second, it meant that the parent data fed back at the RD sessions were unlikely to be a fair basis for influencing decision-making (we communicated that to schools). Ideally, as described previously, we would have also welcomed pupil involvement in the feedback and planning process.
It was not possible to conduct stakeholder interviews at all planned time points. Therefore, the experiences of some SEED RCT schools, particularly intervention schools, were poorly represented in the process evaluation. It is also possible that those who were more positive about the SEED RCT were more likely to agree to be interviewed. Furthermore, key staff (SB and SS) had the combined roles of programme designers, implementers and data collectors. This was likely to create two kinds of bias. First, they were highly invested in the success of the programme, and therefore might have sought favourable data and/or interpreted them positively. Second, their interviewees were aware that they had helped design, and had implemented, the programme, and so might have been inclined to report positive aspects of it, and withhold negative aspects, through courtesy.
The funding for the SEED trial was allocated during an economic recession. We had hoped to offer schools core staff training in SEW, in collaboration with Hawkins et al. ,125 based on the Seattle Social Development Project, but the necessary additional funding was not available. NIHR does not fund interventions and, given the then-impending national referendum on Scottish independence, the Scottish Government was unable to commit to additional intervention costs. The Scottish Government did offer in-kind support for training. The strength of this is that the SEED intervention was able to demonstrate its effectiveness, while being very cost-effective and fitting with the current funding model for schools.
Generalisability
The achieved SEED RCT sample included a good range (see Chapter 3) of urban and rural, denominational (Roman Catholic) and non-denominational schools, and a range of more and less deprived areas. As a result, we can be reasonably confident that these results can be generalised to other Scottish schools that might participate in the SEED intervention. The SEED intervention process is compatible with the Welsh proposals for school health and well-being. 126 In England, small but significant reductions in bullying were also identified after implementation of an intervention involving whole-school processes. 127 Thus, the SEED intervention may be worth extending to the rest of the UK and beyond, especially because its co-produced component strengthens its generalisability, as it encourages adaptation to local contexts. We did not find any differential effectiveness by LA, further suggesting that the SEED intervention is transferable across geographical contexts. We acknowledge that Scotland has a unique policy context (CfE) that has facilitated implementation of the SEED intervention (see Chapter 6, Contextual factors), but the mechanisms for change (see Chapter 6, Mechanisms for change) in the process findings suggest that the SEED intervention may work in alternative policy environments.
Participation in the trial was voluntary, so recruited schools in both arms may have been more motivated to address SEW than the 53 out of 91 schools that declined to participate. There was some attrition of schools, particularly at T2 and T4 (longer-term follow-up). There was also, as expected, attrition at individual pupil level, especially when the OC transitioned to secondary school (see Figure 4). However, all analyses were adjusted for cohort (age), gender and deprivation, and were clustered at school level, and attrition followed the same pattern for both arms of the trial, so this should not affect the findings of this trial.
Interpretation
This trial has provided evidence that the SEED intervention had a small, but statistically significant, positive effect on pupils’ SEW. For the YC this lasted only 2 years from the start of the intervention. However, for the OC, the benefits became stronger over time, peaking at the primary outcome point, T3 (4 years after the start of the intervention). At T3, the effect size was greatest for boys, with a value of 0.42, which is close to moderate.
The older cohort effect
The SEED intervention effects were stronger for the OC (those that started in P5, aged 8–10 years) than for the YC (those that started in P1, aged 4–6 years). Although this may be counterintuitive, as the older pupils were least exposed to the intervention, this could be explained by the relative richness of data collected for the older pupils. Throughout the trial, data were collected for older pupils by self-completed questionnaires, as well as teacher-completed surveys at baseline, which meant that RDs and APs were more focused on needs identified by older pupils. It is also possible that the OC’s self-reported data may have been more sensitive to change than the YC’s teacher-completed SDQs. However, as the SDQ is not validated for self-completion by the YC, we had no option but to use the teacher-completed SDQs. SDQ-TDSs increase with age, so older pupils had more need for action and intervention for their SEW, plus more capacity for change.
This stronger effect size for the OC might be linked to the greater number of intervention options available for older pupils. More packages have been developed and evaluated for that age group, partly because the YC cannot yet read and write. This is reflected in the curricular packages contained in the resource guide being skewed towards the OC.
Gender effect
Among the OC, the effect was significant for both boys and girls, but strongest for boys. This resonates with findings from other studies that boys can benefit most from school-based health promotion interventions. 90,128,129 Bonell et al. 127 reported a reduction in SDQ scores for boys, but not for girls, resonating with our finding of a stronger effect for boys. This warrants further study.
Longer-term outcomes
Our prespecified primary outcome, SDQ-TDS at T3, demonstrated encouraging results, particularly for the older boys (effect size of 0.42), at 4 years post baseline. This is important as very few studies have conducted long-term outcome assessments of environmental interventions that span the transition from primary to secondary school. A notable example is the Seattle Social Development Project (see Chapter 1). 36 The Gatehouse Project, which also influenced this work, had some longer-term outcomes, but did not commence in primary school. 53
Although the primary outcome was the SDQ-TDS, when pupils reached secondary school we also explored health risk behaviours. Descriptive statistics show that intervention arm pupils exhibited lower levels of risk taking for all health risk behaviours explored. Statistically significant reductions were found for e-cigarettes (T2 and T3) and for alcohol (T4). The intervention was a process to improve SEW and not directly aimed at health risk behaviours, but an association with improved SEW reducing health risk behaviour is expected. 26 These findings provide further evidence of the potential benefits of investing in young people’s SEW.
Unfortunately, by T4, 6 years post baseline, the intervention effect had been diminished. Although it was disappointing not to retain the level of effect reported at T3, on reflection, pupils with higher SDQ-TDSs may have been more likely to drop out the longer we ran the trial, as they are likely to be the pupils least engaged with school, leaving fewer pupils with capacity for reduction in their SDQ-TDS. 130 The T4 data collection was possible only because of additional funding; there was uncertainty and delay in securing this funding, necessitating an extension to our ethics permission. Unfortunately, by the time we were able to collect the T4 data, some pupils in some schools in the OC may have left school, particularly those disengaged with education, who may also have higher SDQ-TDSs. Three out of 18 secondary schools declined to take part in T4, reducing our statistical power. This also highlights the challenges that schools face in engaging with research among school years facing national examinations. It may be worth noting that secondary schools were not part of our intervention, rather they were co-operating to help us collect longer-term data; perhaps this affected their motivation to stay in the trial during a challenging school year.
Fit with the theory of change
We used the evidence to test the theory of change, and a number of the proposed mechanisms were supported by the outcome and process evaluations (see Chapters 4 and 6). These mechanisms of change are explored in detail in Chapter 6 and include the following: (1) improved staff–staff relationships, attributed to participation in the SEED process and RD sessions, leading to a collective commitment to, and awareness of, SEW; (2) improved pupil–staff relationships, perceptions of climate, and school connectedness, which may emerge from point 1; and (3) enhancing the process of identifying and evidencing SEW needs in school and implementing appropriate activities to address these needs.
The SEED intervention encouraged school staff to think about what they could do within their own school context to address both pupil and staff SEW. It seems that differences in pupil SEW cannot be attributed to the implementation of evidence-based curricular packages recommended in the SEED resource guide. Notably there was not more uptake of evidence-based SEW curricular packages in the intervention arm. This may have been due to the high financial and time costs of many of these initiatives and competing less costly resources from alternative sources that were not evidence based. It is also possible that intervention schools’ selection of packages was more attuned to the needs of their pupils, as intended by the SEED intervention process. However, this also suggests that staff may recognise that there are alternative responses to problems than implementation of curricular packages, such as focusing on school culture and relationships (see Chapter 6). There was evidence that the SEED intervention improved the quality of school relationships and the provision of space to discuss issues around SEW, which may have directly affected staff relationships, as well as pupil SEW.
Some schools opted to prioritise staff–staff relationships, rather than addressing pupil SEW directly, which may have improved the school environment. Social learning theory80,81 proposes that improved staff–staff relationships would also provide a healthier model of relationships for pupils, but there is limited evidence from our findings to support a strong role for modelling. Rather, by creating an iterative reflective space for all school staff to meet and discuss ways to improve their own and pupils’ SEW, the SEED intervention may have amplified the importance of SEW and stimulated an improvement in social relationships within schools. Certainly, the importance of school connectedness is reflected in the literature,57,131 and has been shown to be protective of SEW over the transition to secondary school,132 which may help explain the stronger benefits for the OC. Further evidence that improving school culture and ethos was a stronger mechanism than improving pupils’ social and emotional skills is demonstrated in the secondary outcomes. The SEED intervention did not appear to affect several of the pupil SEW core competencies, including self-awareness (ability to recognise and talk about emotions) and social awareness (empathy and helping others). However, there were small improvements in whole-school relationships (pupils’ perceptions of kindness between pupils and from staff to pupils), liking school and perceptions of school climate (feeling safe and fairly treated at school).
Echoing the discussion above, the process evaluation suggested a stronger pathway for improved regulatory, rather than instructional, order. Markham and Aveyard’s83 theory of human functioning and school organisation proposes that student commitment to the instructional (formal and informal learning) and regulatory orders (behaviour and belonging) leads to student affiliation (the ability to form relationships) and student practical reasoning and self-management, ultimately leading to health-promoting behaviours. This resonates with the CASEL domains secondary outcomes: the most significant effects were in self-management (all pupils), followed by responsible decision-making (boys only). 33,73 It also may help explain the beneficial e-cigarettes finding at T3 (see above and also Chapter 4). This theory adds that commitment to school may be achieved by fostering collaborative staff–staff and staff–pupil relationships, focusing on pupil SEW rather than solely academic progress, and distributing authority among staff. The SEED findings chime with this theory, for instance, the significant improvements in perceptions of management and the process evaluation findings of an increased focus on SEW.
That the SEED intervention appears to work on several levels resonates with Bonell et al. ’s47 integrated theory of how school environment influences health. This review explored 24 existing theories and grouped these into ‘upstream’ theories (i.e. those focused around a school’s organisation, teaching structures, discipline or pastoral care, or physical environment); ‘downstream’ theories, relating to students’ health-related cognitions or behaviour; and ‘medial’, those that combine elements of both upstream and downstream pathways. The resulting integrated theory proposes four pathways through which the school environment can influence student health: (1) student commitment to school [related to the school’s instructional and regulatory orders (see previous description of Markham and Aveyard’s83 theory)]; (2) students’ peer commitments; (3) student cognitions (both through the social ecology of peer groups and through formal social, emotional and health education); and (4) student behaviours, either health promoting or health harming, influenced, in part, by relationships with staff and peers.
The process evaluation suggests that the findings relate to the socioecological model. 76 The SEED intervention appeared to work as a true ‘whole-school’ intervention, with impact observed at several levels of the ecological framework, which is again compatible with Bonell et al. ’s47 upstream theories. We observed improved pupil well-being in some core competencies, particularly for the OC, suggesting that the SEED intervention can act on attitudes and cognitions. The SEED intervention also promoted positive relationships (interpersonal), contributed to a culture of evaluating practice, provided space for professional dialogue (organisational) and supported schools in delivering national policy (macro).
We were able to demonstrate small, but significant, intervention effects without additional resources beyond a school’s own resources, devolved budgets and the available research funding for data collection. It is possible that larger effects may have been found if we had been able to offer some additional core training based on the Seattle Social Development Project. 125
What does the Social and Emotional Education and Development evaluation add to existing knowledge?
This evaluation adds to existing global knowledge in a number of key ways. Few rigorous evaluations have considered the cost-effectiveness of SEW initiatives in schools;116 the SEED evaluation adds important evidence when considering the economic value of such initiatives, showing, on balance of probability, that the SEED intervention and other complex social interventions can be cost-effective.
This study adds to a growing consensus that school-based SEW interventions show long-term benefits when employing a whole-school, multicomponent approach addressing the curriculum, school environment and school community. 7,10,28–32,34,52 In addition to its multimodal delivery, the most important elements of the SEED intervention are its co-production process, similar to that of the Gatehouse Project,53 and the long duration of its effect for the OC. 33,133 A 2019 review on the effectiveness of whole-school interventions for SEW34 suggested that whole-school interventions demonstrated significantly higher effect sizes in the USA than in other countries; the review authors suggest that this is due to the less prescriptive nature of national support for social and emotional skills development in countries other than the USA. It may also reflect the comparison arms used in studies. The needs analysis contained in the SEED intervention may help bridge this gap by giving direction on what may be useful to implement. 34,134 Like the Seattle Social Development Project125 and the Zero Tolerance Respect trial,129 the SEED intervention influenced young people’s SEW starting during middle childhood, suggesting that this is an optimal age for intervention.
The SEED intervention was presented as a universal intervention process in schools, but it is compatible with proportionate universalism. 26,135 Proportionate universalism was used in ChildSmile136 and succeeded in beating the UK government targets for dental health among young people. Our findings suggest that the SEED intervention has the ability to effect system-level change in the school setting, and this can be interpreted through Hawe et al. ’s137 theory of an intervention as an ‘event within a system’. Their complex systems perspective argues against looking at interventions as linear models working in isolation, instead focusing on how an intervention interacts with, and affects, the dynamic context into which it is introduced. The value of this perspective in the design and evaluation of interventions has received increasing support in recent years. 47,138,139 The ability of an intervention to effect change within a complex system can be investigated through its reach across the organisation, its integration into routine practice, changes in relationships and changes in distribution and transformation of resources. 137
The SEED evaluation reinforces the need for quality implementation. The intervention worked best when combined with recognised factors affecting programme implementation, such as the school’s capacity and openness for change, fit with existing priorities, effective leadership and perceived fit for pupil needs. 140 The flexibility of the SEED process means that adaptation is possible to ensure a fit with the local culture and context. 141
The SEED intervention did not appear to displace other activities. The work of the SEED intervention was directly aligned with other national health and well-being priorities, so rather than being in competition with other initiatives, the SEED intervention was seen as supporting a process that schools were going through anyway. This also resonates with the theory of co-production. 67
Sustainability and implications for roll-out
The feasibility study for the SEED RCT took place in 2011;68 at that stage, the pilot primary schools did not have the capacity to support online data collection. During the course of the trial, we developed the technology to administer surveys online and schools’ capacity to complete surveys changed. We therefore successfully implemented the longer-term, final follow-up wave of data collection (T4) online. This leads to two conclusions. First, the overall cost of collecting, processing, analysing and reporting data reduced substantially, thus making the SEED intervention even more cost-effective. Second, the new technology context is helpful for the roll-out and sustainability of the SEED intervention process and would allow surveying of a larger proportion of pupils in the school.
A strength of the intervention is its adaptability to local context. Future implementation of the SEED intervention should, however, ensure that key features are present. The RD sessions are essential: presentation of school-specific data is important at these, but the space for dialogue is critical. We anticipated that the school EP would play an essential role in the delivery of the SEED intervention. Although a strong relationship between the EP and the school certainly facilitates the SEED intervention, this is not a prerequisite; however, strong leadership from someone with knowledge of the school is needed. There are benefits to having an external ‘critical friend’,53 but a committed HT or other member of the senior management team could take this role. Committing to, and documenting plans for, improvement is an essential element of the SEED intervention, but the APs could be adapted to integrate into existing improvement planning processes.
We are currently working with the Scottish Government to explore how the SEED intervention process can be integrated into their initiative to undertake a regular pupil census. Similarly, we are working with English, Welsh and Scottish groups who are developing research-ready school networks, which aim to help schools identify their key health and well-being needs by facilitating closer collaborations between health researchers and schools. 126,142
To scale up the SEED intervention, several adaptations are recommended. Data collection and report production should move online, and data collection should be extended to include all primary school cohorts when possible. Rather than yearly cycles (of surveys, feedback, reflection, and action selection and implementation), a 2-year cycle would improve feasibility, acceptability and engagement. We expect that online administration of questionnaires and a 2-year cycle would compensate for any increased financial or time burden of increasing the number of cohorts surveyed. Consideration should be given to optimising presentation of data reports. There needs to be a balance between presenting the data as simply as possible and retaining the richness of data that are not available elsewhere. A strong finding from the process evaluation was that the richness and depth of data were seen as one of the unique and valued aspects of the SEED intervention; however, this needs to be presented in a structured and accessible way.
Further research
We recommend that future research tries to understand the gender differences illustrated by the SEED RCT outcomes. The impact of adding core training elements to the intervention process, for instance as in the Seattle Social Development Project36 and the initiating change locally in bullying and aggression through the school environment (INCLUSIVE) intervention,127 should be evaluated. Future steps for the SEED intervention will be to explore its transferability to other contexts, including other countries and other settings. Considering the diminished outcomes at T4, it is unfortunate that the SEED RCT involvement with secondary schools was limited to collecting data to evaluate a primary school intervention. There is scope to extend the SEED model for delivery in secondary schools, given the general absence of whole-school approaches targeting staff well-being at secondary school. 143 Extending the intervention into secondary schools may help retain the benefits for pupils facing national examinations, which is a stressful time for them. Finally, we are keen to explore extending the SEED concept to other organisational settings, for instance preschool establishments, homes for the elderly and workplaces.
In the future, funding permitting, we hope to conduct a data linkage to SEED RCT pupils’ external examination results (aggregated linkage rather than individual pupil level) to assess whether or not the SEED intervention had any beneficial impact on academic attainment.
Conclusions
This cluster RCT provides robust evidence about the effects of the SEED intervention, a school-based intervention to promote SEW in children. We have demonstrated that the SEED intervention is an acceptable, cost-effective way to modestly improve pupil well-being and improve school climate, particularly for older pupils and, among them, particularly for boys and those with greater levels of psychological difficulties of both genders. There is no evidence of the intervention being more beneficial for advantaged pupils, so it should not widen the inequalities gap. It was beneficial during the transition from primary to secondary school, but longer-term analysis suggests that the effect is diminished after 6 years. The SEED intervention can be implemented alongside existing systems for addressing pupil well-being and can be complementary to other intervention initiatives.
Acknowledgements
There are many people to thank for the SEED Phase III trial being able to happen:
-
The pupils, staff and pupils’ parents of the 59 schools that participated in the main trial or in the feasibility or pilot trial; without them we would have no data to present or discuss.
-
Our conscientious and dedicated SEED RCT fieldworkers, who travelled to schools, collected the data and supported our SEED RCT pupils.
-
The funders of our scoping and feasibility-to-pilot trial, namely the Scottish Collaboration for Public Health Research and Policy (MRC and CSO) and the NIHR Public Health Research programme for funding the Phase III trial.
-
The Scottish Government and participating LA education departments, for a range of in-kind support for the SEED intervention, development, pilot and evaluation.
-
The EPs in the LA areas who supported and facilitated the reflective sessions with school staff.
-
The PHRF, currently based at MRC/CSO SPHSU. Its staff supported everything, including artwork; questionnaire design; production; barcoding; logging questionnaires out and in, and their transfer to the RCB (Clinical Trials Unit); data entry of personal information and textual data; online questionnaires; day-to-day management of the SEED RCT fieldworkers; and scheduling with schools. Thanks in particular to Marcela Gavigan, Elaine Hindle, Kate Campbell, Kaye Ross, Sharon Mitchell, Julie Watson, Andrew Jackson, Matthew Tolan, John Kelly and John Gibbons.
-
The MRC/CSO SPHSU at the University of Glasgow for creating a supportive and conducive working environment for conducting our research. Thanks in particular to Professor Laurence Moore, Professor Kate Hunt, Professor Dame Sally Macintyre, Dr Helen Sweeting, Dr Alice McLachlan, Carol Nicol, Crawford Nelson, Enni Pukkinen, Iain Taylor, Brenda Butler and Susan Wilkie.
-
The RCB (Clinical Trials Unit) at the University of Glasgow for working with us on our statistical analysis plan, project and data management, data entry, and data analysis of our outcome evaluation. In particular, we would like to thank Dr Sarah Weeden, John McHugh and Dr Martina Messow.
-
Dr Nikki Boyer and Professor Elisabeth Fenwick for their input to the economic evaluation.
-
Dr Tony Axon for his support with proofreading, EndNote [Clarivate Analytics (formerly Thomson Reuters), Philadelphia, PA, USA] and long-document challenges in Microsoft Word (Microsoft Corporation).
-
The SEED RCT Trial Steering Committee, which provided expert advice and feedback and was chaired very effectively by Professor Chris Bonell (London School of Hygiene & Tropical Medicine). The members were Maggie Fallon (Education Scotland), Dr Graham Connolly (University of Strathclyde), Professor Gillian Raab (University of Edinburgh), Dr Jeremy Segrott (University of Cardiff) and Barry Syme (Glasgow Education Psychology Service).
-
Last but not least, our anonymous external peer reviewers for their detailed, constructive and careful comments.
Contributions of authors
Sarah Blair (https://orcid.org/0000-0001-5045-2636) contributed to the design of the intervention and study; was a researcher on the scoping and feasibility study and then the pilot trial; was project manager and a researcher for the main trial; co-led the writing of this report; and led the process evaluation. Sarah Blair was previously known as Sarah Tweedie.
Marion Henderson (https://orcid.org/0000-0001-7582-9516), chief investigator, led the conception and design of the SEED trial, and directed the pilot and main evaluation. She contributed to the collection of quantitative and qualitative data for the main study and co-led the writing of this report.
Alex McConnachie (https://orcid.org/0000-0002-7262-7000), statistical lead, co-led the outcome evaluation, produced the school reports that fed into the school RD sessions, and contributed to the writing of this report.
Emma McIntosh (https://orcid.org/0000-0001-6340-3083) was the economic evaluation lead and contributed to the writing of this report.
Susie Smillie (https://orcid.org/0000-0003-4748-0710) was a researcher on the project for 5 years; she contributed to the day-to-day running of the trial, the collection of both qualitative and quantitative data, and drafts of this report.
Kirsty Wetherall (https://orcid.org/0000-0003-0547-3184) conducted the statistical analysis for the outcome evaluation and contributed to the writing of the methodology (see Chapter 3) and outcome evaluation (see Chapter 4).
Daniel Wight (https://orcid.org/0000-0002-1234-3110) co-led the development and design of the intervention and its evaluation from scoping to Phase III trial, co-led the process evaluation, and contributed to the collection of qualitative data for the main study and to the writing of this report.
Yiqiao Xin (https://orcid.org/0000-0001-5856-3103) conducted the economic evaluation analysis and co-led the writing of the economic evaluation chapter (see Chapter 5) and relevant sections of the methodology chapter (see Chapter 3).
Lyndal Bond (https://orcid.org/0000-0003-1693-5508) co-led the development and design of the intervention and its evaluation from scoping to Phase III trial, and supported the interpretation of the results.
Lawrie Elliott (https://orcid.org/0000-0001-7727-231X), co-investigator, conducted some process evaluation analysis and commented across all chapters of this report.
Sally Haw (https://orcid.org/0000-0001-7844-0362) co-investigator, conducted some process evaluation analysis and commented across all chapters of this report.
Caroline Jackson (https://orcid.org/0000-0002-2067-2811) was a co-investigator and contributed to the writing of this report, particularly to the introduction (see Chapter 1).
Kate Levin (https://orcid.org/0000-0002-7889-3289), co-investigator with statistical expertise, commented across chapters of this report.
Philip Wilson (https://orcid.org/0000-0002-4123-8248) co-led the design of the evaluation, supported the statistical analysis plan, and contributed to and commented on all chapters of this report.
All authors commented on drafts of this report.
Data-sharing statement
After the SEED data set is completed and the co-applicants have had key outcome papers published, the data will be deposited with ReShare, a UK online data repository. While that is happening, interested parties can contact the corresponding author and follow the MRC/CSO SPHSU data-sharing policy.
Ethics approval and trial monitoring
Ethics approval was granted by the University of Glasgow College of Medical, Veterinary and Life Sciences Ethics Committee for Non-Clinical Research Involving Human Subjects on 22 November 2012 (reference number 2012087). The MRC/CSO SPHSU, University of Glasgow, the lead applicant’s employer during the trial, sponsored the trial.
A steering committee was established to provide independent expert advice. This committee included experts in the field of school-based interventions, an EP, members of Education Scotland, a parent representative (identified by the Scottish Parent Teacher Council) and a teacher representative (identified by the Educational Institute of Scotland). In addition, involvement of school staff, pupils and parents is integral to the SEED co-production process.
Disclaimers
This report presents independent research funded by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the PHR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the PHR programme or the Department of Health and Social Care.
References
- UNICEF Innocenti Research Centre . Child Poverty in Perspective: An Overview of Child Well-Being in Rich Countries. Innocenti Report Card No. 7 2007.
- UNICEF Office of Research . Child Well-Being in Rich Countries: A Comparative Overview. Innocenti Report Card 11 2013.
- Office for National Statistics . Conceptions in England and Wales 2016.
- NHS National Services Scotland . Teenage Pregnancy: Year of Conception, Ending 31 December 2016 2018.
- National Institute for Health and Care Excellence . Social and Emotional Wellbeing for Children and Young People Overview 2020. https://pathways.nice.org.uk/pathways/social-and-emotional-wellbeing-for-children-and-young-people (accessed 1 November 2021).
- Department for Children, Schools and Families . Your Child, Your Schools, Our Future: Building a 21st Century Schools System n.d. www.gov.uk/government/publications/department-for-children-schools-and-families-report-june-2009 (accessed 1 April 2022).
- Hagelskamp C, Brackett MA, Rivers SE, Salovey P. Improving classroom quality with the RULER approach to social and emotional learning: proximal and distal outcomes. Am J Community Psychol 2013;51:530-43. https://doi.org/10.1007/s10464-013-9570-x.
- Lee RC, Tiley CE, White JE. The Place2Be: measuring the effectiveness of a primary school-based therapeutic intervention in England and Scotland. Couns Psychother Res 2009;9:151-9. https://doi.org/10.1080/14733140903031432.
- McIntosh E, Barlow J, Davis H, Stewart-Brown S. Economic evaluation of an intensive home visiting programme for vulnerable families: a cost-effectiveness analysis of a public health intervention. J Public Health 2009;31:423-33. https://doi.org/10.1093/pubmed/fdp047.
- Roffey S, McCarthy F. Circle Solutions, a philosophy and pedagogy for learning positive relationships: what promotes and inhibits sustainable outcomes?. Int J Emot Educ 2013;5:36-55.
- Wolke D, Waylen A, Samara M, Steer C, Goodman R, Ford T, et al. Selective drop-out in longitudinal studies and non-biased prediction of behaviour disorders. Br J Psychiatry 2009;195:249-56. https://doi.org/10.1192/bjp.bp.108.053751.
- Department for Education and Skills . Social and Emotional Aspects of Learning (SEAL) for Secondary Schools: Guidance Booklet 2007.
- Scottish Government . Early Years and Early Intervention n.d. www.scotland.gov.uk/Publications/2008/03/14121428/0 (accessed 1 April 2022).
- Scottish Government . Equally Well: Report of the Ministerial Task Force on Health Inequalities n.d. www.scotland.gov.uk/Publications/2008/06/25104032/0 (accessed 1 April 2022).
- Scottish Government . Getting It Right for Every Child (GIRFEC) n.d. www.gov.scot/policies/girfec/ (accessed 1 April 2022).
- Scottish Government, Her Majesty’s Inspectorate of Education, Scottish Qualifications Authority, Learning and Teaching Scotland . The Curriculum for Excellence: Health and Wellbeing Experiences and Outcomes n.d. https://education.gov.scot/Documents/health-and-wellbeing-eo.pdf (accessed 1 April 2022).
- Scottish Executive . A Curriculum for Excellence: Building the Curriculum 3–18 (1). The Contribution of Curriculum Areas n.d. https://education.gov.scot/Documents/btc1.pdf (accessed 1 April 2022).
- World Health Organization, Council of Europe, Commission of European Communities . The European Network of Health Promoting Schools: A Joint World Health Organization–Council of Europe–Commission of European Communities Project 1993.
- Foxcroft DR, Ireland D, Lister-Sharp DJ, Lowe G, Breen R. Longer-term primary prevention for alcohol misuse in young people: a systematic review. Addiction 2003;98:397-411. https://doi.org/10.1046/j.1360-0443.2003.00355.x.
- Spoth R, Greenberg M, Turrisi R. Preventive interventions addressing underage drinking: state of the evidence and steps toward public health impact. Pediatrics 2008;121:311-36. https://doi.org/10.1542/peds.2007-2243E.
- Wiehe SE, Garrison MM, Christakis DA, Ebel BE, Rivara FP. A systematic review of school-based smoking prevention trials with long-term follow-up. J Adolesc Health 2005;36:162-9. https://doi.org/10.1016/j.jadohealth.2004.12.003.
- Faggiano F, Minozzi S, Versino E, Buscemi D. Universal school-based prevention for illicit drug use. Cochrane Database Syst Rev 2014;12. https://doi.org/10.1002/14651858.CD003020.pub3.
- DiCenso A, Guyatt G, Willan A, Griffith L. Interventions to reduce unintended pregnancies among adolescents: systematic review of randomised controlled trials. BMJ 2002;324. https://doi.org/10.1136/bmj.324.7351.1426.
- Scher LS, Maynard RA, Stagner M. Interventions intended to reduce pregnancy-related outcomes among adolescents. Campbell Systematic Reviews 2006;2:1-70. https://doi.org/10.4073/csr.2006.12.
- Oringanje C, Meremikwu MM, Eko H, Esu E, Meremikwu A, Ehiri JE. Interventions for preventing unintended pregnancies among adolescents. Cochrane Database Syst Rev 2009;4. https://doi.org/10.1002/14651858.CD005215.pub2.
- Jackson CA, Haw SJ, Frank JW. Environmental scan of adolescent and young adulthood health in Scotland: interventions that address multiple risk behaviours or take a generic approach to risk in youth. Eur J Public Health 2010;20.
- Durlak JA, Weissberg RP, Dymnicki AB, Taylor RD, Schellinger KB. The impact of enhancing students’ social and emotional learning: a meta-analysis of school-based universal interventions. Child Dev 2011;82:405-32. https://doi.org/10.1111/j.1467-8624.2010.01564.x.
- Adi Y, Killoran A, Janmohamed K, Stewart-Brown S. Systematic Review of the Effectiveness of Interventions to Promote Mental Wellbeing in Children in Primary Education. Report 1: Universal Approaches, Non-Violence-Related Outcomes 2007.
- Adi Y, Killoran A, Schrader McMillian A, Stewart-Brown S. Systematic Review of Interventions to Promote Mental Wellbeing in Primary Schools. Report 3: Universal Approaches With Focus on Prevention of Violence and Bullying 2007.
- Batty GD, Deary IJ, Macintyre S. Childhood IQ in relation to risk factors for premature mortality in middle-aged persons: the Aberdeen Children of the 1950s study. J Epidemiol Community Health 2007;61:241-7. https://doi.org/10.1136/jech.2006.048215.
- Currie C, Molcho M, Boyce W, Holstein B, Torsheim T, Richter M. Researching health inequalities in adolescents: the development of the Health Behaviour in School-Aged Children (HBSC) family affluence scale. Soc Sci Med 2008;66:1429-36. https://doi.org/10.1016/ j.socscimed.2007.11.024.
- Green H, McGinnity Á, Melzer H, Ford T, Goodman R. Mental Health of Children and Young People in Great Britain, 2004 2005. https://sp.ukdataservice.ac.uk/doc/5269/mrdoc/pdf/5269technicalreport.pdf (accessed 31 October 2022).
- Greenberg MT, Weissberg RP, O’Brien MU, Zins JE, Fredericks L, Resnik H, et al. Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. Am Psychol 2003;58:466-74. https://doi.org/10.1037/ 0003-066x.58.6-7.466.
- Goldberg JM, Sklad M, Elfrink TR, Schreurs KM, Bohlmeijer ET, Clarke AM. Effectiveness of interventions adopting a whole school approach to enhancing social and emotional development: a meta-analysis. Eur J Psychol Educ 2019;34:755-82. https://doi.org/10.1007/ s10212-018-0406-9.
- Lonczak HS, Abbott RD, Hawkins JD, Kosterman R, Catalano RF. Effects of the Seattle Social Development Project on sexual behavior, pregnancy, birth, and sexually transmitted disease outcomes by age 21 years. Arch Pediatr Adolesc Med 2002;156:438-47. https://doi.org/10.1001/archpedi.156.5.438.
- Hawkins JD, Kosterman R, Catalano RF, Hill KG, Abbott RD. Promoting positive adult functioning through social development intervention in childhood: long-term effects from the Seattle Social Development Project. Arch Pediatr Adolesc Med 2005;159:25-31. https://doi.org/10.1001/archpedi.159.1.25.
- Manca A, Hawkins N, Sculpher MJ. Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility. Health Econ 2005;14:487-96. https://doi.org/10.1002/hec.944.
- Hawkins JD, Guo J, Hill KG, Battin-Pearson S, Abbott RD. Long-term effects of the Seattle Social Development Intervention on school bonding trajectories. Appl Dev Sci 2001;5:225-36. https://doi.org/10.1207/S1532480XADS0504_04.
- Kosterman R, Hawkins JD, Hill KG, Bailey JA, Catalano RF, Abbott RD. Effects of Social Development Intervention in childhood on adult life at ages 30 to 39. Prev Sci 2019;20:986-95. https://doi.org/10.1007/s11121-019-01023-3.
- Connolly P, Miller S, Kee F, Sloan S, Gildea A, McIntosh E, et al. A cluster randomised controlled trial and evaluation and cost-effectiveness analysis of the Roots of Empathy schools-based programme for improving social and emotional well-being outcomes among 8- to 9-year-olds in Northern Ireland. Public Health Res 2018;6. https://doi.org/10.3310/phr06040.
- Gordon M. Roots of Empathy: Changing the World Child by Child. Toronto, ON: Dundurn Press; 2005.
- Santos RG, Chartier MJ, Whalen JC, Chateau D, Boyd L. Effectiveness of school-based violence prevention for children and youth: a research report. Healthc Q 2011;14:80-91. https://doi.org/10.12927/hcq.2011.22367.
- Weare K, Gray G. What Works in Developing Children’s Emotional and Social Competence and Well Being?. London: Department for Education and Skills; 2003.
- Bonell C, Fletcher A, McCambridge J. Improving school ethos may reduce substance misuse and teenage pregnancy. BMJ 2007;334:614-16. https://doi.org/10.1136/bmj.39139.414005.AD.
- Fletcher A, Bonell C, Hargreaves J. School effects on young people’s drug use: a systematic reivew of intervention and observational studies. J Adolesc Health 2007;42:209-20. https://doi.org/10.1016/j.jadohealth.2007.09.020.
- Shackleton N, Jamal F, Viner RM, Dickson K, Patton G, Bonell C. School-based interventions going beyond health education to promote adolescent health: systematic review of reviews. J Adolesc Health 2016;58:382-96. https://doi.org/10.1016/j.jadohealth.2015.12.017.
- Bonell CP, Fletcher A, Jamal F, Wells H, Harden A, Murphy S, et al. Theories of how the school environment impacts on student health: systematic review and synthesis. Health Place 2013;24:242-9. https://doi.org/10.1016/j.healthplace.2013.09.014.
- Bonell C, Parry W, Wells H, Jamal F, Fletcher A, Harden A, et al. The effects of the school environment on student health: a systematic review of multi-level studies. Health Place 2013;21:180-91. https://doi.org/10.1016/j.healthplace.2012.12.001.
- Jamal F, Fletcher A, Harden A, Wells H, Thomas J, Bonell C. The school environment and student health: a systematic review and meta-ethnography of qualitative research. BMC Public Health 2013;13. https://doi.org/10.1186/1471-2458-13-798.
- Naghieh A, Montgomery P, Bonell CP, Thompson M, Aber JL. Organisational interventions for improving wellbeing and reducing work-related stress in teachers. Cochrane Database Syst Rev 2015;4. https://doi.org/10.1002/14651858.CD010306.pub2.
- Bonell C, Jamal F, Harden A, Wells H, Parry W, Fletcher A, et al. Systematic review of the effects of schools and school environment interventions on health: evidence mapping and synthesis. Public Health Res 2013;1. https://doi.org/10.3310/phr01010.
- Langford R, Bonell CP, Jones HE, Pouliou T, Murphy SM, Waters E, et al. The WHO Health Promoting School framework for improving the health and well-being of students and their academic achievement. Cochrane Database Syst Rev 2014;4. https://doi.org/10.1002/14651858.CD008958.pub2.
- Bond L, Patton G, Glover S, Carlin JB, Butler H, Thomas L, et al. The Gatehouse Project: can a multilevel school intervention affect emotional wellbeing and health risk behaviours?. J Epidemiol Community Health 2004;58:997-1003. https://doi.org/10.1136/jech.2003.009449.
- Bond L, Thomas L, Coffey C, Glover S, Butler H, Carlin JB, et al. Long-term impact of the Gatehouse Project on cannabis use of 16-year-olds in Australia. J Sch Health 2004;74:23-9. https://doi.org/10.1111/j.1746-1561.2004.tb06597.x.
- Toumbourou JW, Rowland B, Jefferies A, Butler H, Bond L. Preventing Drug-Related Harm Through School Re-Organisation and Behaviour Management 2004. www.researchgate.net/ publication/237619004_Preventing_drug-related_harm_through_school_re-organisation_and_behaviour_management (accessed 1 April 2022).
- Libbey HP. Measuring student relationships to school: attachment, bonding, connectedness, and engagement. J Sch Health 2004;74:274-83. https://doi.org/10.1111/j.1746-1561.2004. tb08284.x.
- Bond L, Butler H, Thomas L, Carlin J, Glover S, Bowes G, et al. Social and school connectedness in early secondary school as predictors of late teenage substance use, mental health, and academic outcomes. J Adolesc Health 2007;40:357.e9-18. https://doi.org/10.1016/ j.jadohealth.2006.10.013.
- Patton GC, Bond L, Carlin JB, Thomas L, Butler H, Glover S, et al. Promoting social inclusion in schools: a group-randomized trial of effects on student health risk behavior and well-being. Am J Public Health 2006;96:1582-7. https://doi.org/10.2105/AJPH.2004.047399.
- Shucksmith C, Summerbell C, Jones S, Whittaker V. Mental Wellbeing of Children in Primary Education (Targeted/Indicated Activites). London: National Institute for Health and Care Excellence; 2007.
- Stewart-Brown S, McMillan AS. Home and Community Based Parenting Support Programmes and Interventions: Report of Workpackage 2 of the DataPrev Project n.d. http://wrap.warwick.ac.uk/3239/ (accessed 1 April 2022).
- National Institute for Health and Care Excellence (NICE) . Social and Emotional Wellbeing in Primary Education 2008. www.nice.org.uk/ guidance/ph12 (accessed 1 April 2022).
- Jackson C, Haw S, Frank J. Adolescent and Young Adult Health in Scotland. Interventions That Address Multiple Risk Behaviours or Take a Generic Approach to Risk in Youth n.d. https://blogs. ed.ac.uk/scphrp/2014/03/20/adolescent-and-young-adult-health-in-scotland/ (accessed 1 April 2022).
- Barry SJ, Marryat L, Thompson L, Ellaway A, White J, McClung M, et al. Mapping area variability in social and behavioural difficulties among Glasgow pre-schoolers: linkage of a survey of pre-school staff with routine monitoring data. Child Care Health Dev 2015;41:853-64. https://doi.org/10.1111/cch.12237.
- Klanš̌cek HJ, Ziberna J, Korošec A, Zurc J, Albreht T. Mental health inequalities in Slovenian 15-year-old adolescents explained by personal social position and family socioeconomic status. Int J Equity Health 2014;13. https://doi.org/10.1186/1475-9276-13-26.
- West P, Sweeting H, Young R. Transition matters: pupils’ experiences of the primary–secondary school transition in the West of Scotland and consequences for well-being and attainment. Res Pap Educ 2010;25:21-50. https://doi.org/10.1080/02671520802308677.
- Henderson M, Jackson C, Bond L, Wilson P, Elliot L, Levin K, et al. Social and Emotional Education and Development (SEED): A Stratified, Cluster Randomised Trial of a Multi-Component Primary School Intervention That Follows the Pupils’ Transition into Secondary School n.d. www.journalslibrary.nihr.ac.uk/programmes/phr/10300613/#/ (accessed 1 April 2022).
- Boyle D, Harris M. The Challenge of Co-Production: How Equal Partnerships Between Professionals and the Public Are Crucial to Improving Public Services 2009. www.nesta.org.uk/report/the-challenge-of-co-production/ (accessed 1 April 2022).
- Wight D, Henderson M, Connelly G, Wilson P, Haw S, Scott C, et al. Piloting and Refining the Social and Emotional Education and Development (SEED) Programme. MRC/CSO Scottish Collaboration for Public Health Research and Policy; 2012.
- Di Riso D, Salcuni S, Chessa D, Raudino A, Lis A, Altoe G. The Strengths and Difficulties Questionnaire (SDQ). Early evidence of its reliability and validity in a community sample of Italian children. Pers Individ Differ 2010;49:570-5. https://doi.org/10.1016/j.paid.2010.05.005.
- Goodman R. Psychometric properties of the Strengths and Difficulties Questionnaire. J Am Acad Child Adolesc Psych 2001;40:1337-45. https://doi.org/10.1097/00004583-200111000-00015.
- Van Roy B, Veenstra M, Clench-Aas J. Construct validity of the five-factor Strengths and Difficulties Questionnaire (SDQ) in pre-, early, and late adolescence. J Child Psychol Psychiatry 2008;49:1304-12. https://doi.org/10.1111/j.1469-7610.2008.01942.x.
- Mohr BJ, Watkins JM. The Essentials of Appreciative Inquiry: A Roadmap for Creating Positive Futures 2002. www.academia.edu/42960986/EssentialsOfAppreciativeInquiry (accessed 1 April 2022).
- Collaborative for Academic, Social and Emotional Learning (CASEL) . Safe and Sound: An Educational Leader’s Guide to Evidence-Based Social and Emotional Learning (SEL) Programs 2003. https://casel.org/safe-and-sound-guide-to-sel-programs/ (accessed 1 April 2022).
- Littlefield L, Cavanagh S, Knapp R, O’Grady L. Social and Emotional Learning in Australia and the Asia-Pacific: Perspectives, Programs and Approaches. New York, NY: Springer; 2017.
- Scottish Government . Curriculum for Excellence: Health and Wellbeing, Principles and Practice n.d. https://education.gov.scot/Documents/health-and-wellbeing-pp.pdf (accessed 1 April 2022).
- McLeroy KR, Bibeau D, Steckler A, Glanz K. An ecological perspective on health promotion programs. Health Educ Q 1988;15:351-77. https://doi.org/10.1177/109019818801500401.
- Dahlgren G, Whitehead M. Policies and Strategies to Promote Social Equity in Health. Stockholm: Institute for Future Studies; 1991.
- Bronfenbrenner U. Ecological Systems Theory. London: Jessica Kingsley Publishers; 1992.
- Greenhalgh T, Jackson C, Shaw S, Janamian T. Achieving research impact through co-creation in community-based health services: literature review and case study. Milbank Q 2016;94:392-429. https://doi.org/10.1111/1468-0009.12197.
- Bandura A. Social Foundations of Thought and Action: A Social Cognitive Theory. Englewood Cliffs, NJ: Prentice Hall; 1986.
- Pajares F, Johnson MJ. Self-efficacy beliefs and the writing performance of entering high school students. Psychol Schools 1996;33:163-75. https://doi.org/10.1002/(SICI)1520-6807 (199604)33:2<163::AID-PITS10>3.0.CO;2-C.
- Barber BK, Schluterman JM. Connectedness in the lives of children and adolescents: a call for greater conceptual clarity. J Adolesc Health 2008;43:209-16. https://doi.org/10.1016/ j.jadohealth.2008.01.012.
- Markham WA, Aveyard P. A new theory of health promoting schools based on human functioning, school organisation and pedagogic practice. Soc Sci Med 2003;56:1209-20. https://doi.org/10.1016/S0277-9536(02)00120-X.
- Smith M. School attendance in Hertfordshire. Educ Res 1996;38:226-36. https://doi.org/10.1080/0013188960380209.
- Scottish Council of Independent Schools . Annual Census 2017. www.scis.org.uk/facts-and-figures (accessed 8 October 2018).
- Scottish Government . Summary Statistics for Schools in Scotland 2017. www.gov.scot/ publications/summary-statistics-schools-scotland-8-2017-edition/pages/3/ (accessed 1 April 2022).
- Scottish Government . Included, Engaged and Involved: Part 2: A Positive Approach to Managing School Exclusions n.d. www.gov.scot/publications/included-engaged-involved-part-2-positive-approach-preventing-managing-school/ (accessed 1 April 2022).
- Marryat L, Thompson L, Minnis H, Wilson P. Primary schools and the amplification of social differences in child mental health: a population-based cohort study. J Epidemiol Community Health 2018;72:27-33. https://doi.org/10.1136/jech-2017-208995.
- Junghans C, Feder G, Hemingway H, Timmis A, Jones M. Recruiting patients to medical research: double blind randomised trial of ‘opt-in’ versus ‘opt-out’ strategies. BMJ 2005;331. https://doi.org/10.1136/bmj.38583.625613.AE.
- Elliott L, Henderson M, Nixon C, Wight D. Has untargeted sexual health promotion for young people reached its limit? A quasi-experimental study. J Epidemiol Community Health 2013;67:398-404. https://doi.org/10.1136/jech-2012-201034.
- Parkes A, Sweeting H, Wight D. What shapes 7-year-olds’ subjective well-being? Prospective analysis of early childhood and parenting using the Growing Up in Scotland study. Soc Psychiatry Psychiatr Epidemiol 2016;51:1417-28. https://doi.org/10.1007/s00127-016-1246-z.
- Stewart-Brown S, Tennant A, Tennant R, Platt S, Parkinson J, Weich S. Internal construct validity of the Warwick–Edinburgh Mental Well-being Scale (WEMWBS): a Rasch analysis using data from the Scottish Health Education Population Survey. Health Qual Life Outcomes 2009;7. https://doi.org/10.1186/1477-7525-7-15.
- Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015;350. https://doi.org/10.1136/bmj.h1258.
- National Institute for Health and Care Excellence . The NICE Public Health Guidance Development Process (Third Edition) 2012. www.nice.org.uk/process/pmg5/chapter/the-role-of-evidence-providers (accessed 1 April 2022).
- Ratcliffe J, Flynn T, Terlich F, Stevens K, Brazier J, Sawyer M. Developing adolescent-specific health state values for economic evaluation: an application of profile case best–worst scaling to the Child Health Utility 9D. PharmacoEconomics 2012;30:713-27. https://doi.org/10.2165/ 11597900-000000000-00000.
- Ratcliffe J, Stevens K, Flynn T, Brazier J, Sawyer M. An assessment of the construct validity of the CHU9D in the Australian adolescent general population. Qual Life Res 2012;21:717-25. https://doi.org/10.1007/s11136-011-9971-y.
- Stevens K. Valuation of the Child Health Utility 9D Index. PharmacoEconomics 2012;30:729-47. https://doi.org/10.2165/11599120-000000000-00000.
- Stevens K, Ratcliffe J. Measuring and valuing health benefits for economic evaluation in adolescence: an assessment of the practicality and validity of the Child Health Utility 9D in the Australian adolescent population. Value Health 2012;15:1092-9. https://doi.org/10.1016/ j.jval.2012.07.011.
- Curtis L, Burns A. Unit Costs of Health and Social Care 2017. Canterbury: PSSRU, University of Kent; 2017.
- NHS Improvement . National Health Service Reference Costs 2017. https://improvement.nhs.uk/resources/reference-costs/ (accessed 3 August 2018).
- Brazier J, Ratcliffe J, Salomon JA, Tsuchiya A. Measuring and Valuing Health Benefits for Economic Evaluation. Oxford: Oxford University Press Inc.; 2017.
- Schafer JL. Multiple imputation: a primer. Stat Methods Med Res 1999;8:3-15. https://doi.org/10.1177/096228029900800102.
- Ramsey S, Willke R, Briggs A, Brown R, Buxton M, Chawla A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA Task Force report. Value Health 2005;8:521-33. https://doi.org/10.1111/j.1524-4733.2005.00045.x.
- Jakobsen JC, Gluud C, Wetterslev J, Winkel P. When and how should multiple imputation be used for handling missing data in randomised clinical trials – a practical guide with flowcharts. BMC Med Res Methodol 2017;17. https://doi.org/10.1186/s12874-017-0442-1.
- Donner A, Birkett N, Buck C. Randomisation by cluster: sample size requirements and analysis. Am J Epidemiol 1981;114:906-14. https://doi.org/10.1093/oxfordjournals.aje.a113261.
- Paton NI, Stöhr W, Oddershede L, Arenas-Pinto A, Walker S, Sculpher M, et al. The Protease Inhibitor Monotherapy Versus Ongoing Triple Therapy (PIVOT) trial: a randomised controlled trial of a protease inhibitor monotherapy strategy for long-term management of human immunodeficiency virus infection. Health Technol Assess 2016;20. https://doi.org/10.3310/hta20210.
- Boyer N. Economic Evaluation of Population Health Interventions Aimed at Children and Delivered at School. PhD Thesis 2018.
- Philipsson A, Duberg A, Möller M, Hagberg L. Cost–utility analysis of a dance intervention for adolescent girls with internalizing problems. Cost Eff Resour Alloc 2013;11. https://doi.org/10.1186/1478-7547-11-4.
- Wang LY, Nichols LP, Austin SB. The economic effect of Planet Health on preventing bulimia nervosa. Arch Pediatr Adolesc Med 2011;165:756-62. https://doi.org/10.1001/archpediatrics.2011.105.
- Cheney G, Schlösser A, Nash P, Glover L. Targeted group-based interventions in schools to promote emotional well-being: a systematic review. Clin Child Psychol Psychiatry 2014;19:412-38. https://doi.org/10.1177/1359104513489565.
- Wells J, Barlow J, Stewart-Brown S. A systematic review of universal approaches to mental health promotion in schools. Health Educ 2003;103:197-220. https://doi.org/10.1108/ 09654280310485546.
- Humphrey N, Lendrum A, Wigelsworth M. Social and Emotional Aspects of Learning (SEAL) Programme in Secondary Schools: National Evaluation n.d. www.gov.uk/government/ publications/social-and-emotional-aspects-of-learning-seal-programme-in-secondary-schools-national-evaluation (accessed 1 April 2022).
- Durlak JA, Wells AM. Primary prevention mental health programs for children and adolescents: a meta-analytic review. Am J Community Psychol 1997;25:115-52. https://doi.org/10.1023/ a:1024654026646.
- National Institute for Health and Care Excellence . Methods for the Development of NICE Public Health Guidance (Third Edition) 2012. www.nice.org.uk/process/pmg4/chapter/incorporating-health-economics (accessed 3 August 2018).
- Turner AJ, Sutton M, Harrison M, Hennessey A, Humphrey N. Cost-effectiveness of a school-based social and emotional learning intervention: evidence from a cluster-randomised controlled trial of the promoting alternative thinking strategies curriculum. Appl Health Econ Health Policy 2020;18:271-85. https://doi.org/10.1007/s40258-019-00498-z.
- Belfield C, Bowden AB, Klapp A, Levin H, Shand R, Zander S. The economic value of social and emotional learning. J Benefit-Cost Anal 2015;6:508-44. https://doi.org/10.1017/bca.2015.55.
- Tudor Edwards R, McIntosh E. Applied Health Economics for Public Health Practice and Research. Oxford: Oxford University Press; 2019.
- Bevington TJ. Appreciative evaluation of restorative approaches in schools. Pastor Care Educ 2015;33:105-15. https://doi.org/10.1080/02643944.2015.1046475.
- Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, et al. The Warwick–Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation. Health Qual Life Outcomes 2007;5. https://doi.org/10.1186/1477-7525-5-63.
- McCabe C, Claxton K, Culyer AJ. The NICE cost-effectiveness threshold. PharmacoEconomics 2008;26:733-44. https://doi.org/10.2165/00019053-200826090-00004.
- Kane J, Lloyd G, McCluskey G, Maguire R, Riddell S, Stead J, et al. Generating an inclusive ethos? Exploring the impact of restorative practices in Scottish schools. Int J Incl Educ 2007;11:1-21.
- Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Medical Research Council Guidance. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008;337. https://doi.org/10.1136/bmj.a1655.
- White M, Adams J, Heywood P. Social Inequality and Public Health. Bristol: Policy Press; 2009.
- Scottish Government, NHS Health Scotland . Evaluation of Healthy Respect Phase Two: Final Report n.d. www.healthscotland.com/uploads/documents/12159-HealthyRespectEvaluationReport_3342.pdf (accessed 1 April 2022).
- Hawkins JD, Smith BH, Hill KG, Kosterman R, Catalano RF, Abbott RD. Promoting social development and preventing health and behavior problems during the elementary grades: results from the Seattle Social Development Project. Vict Offender 2007;2:161-81. https://doi.org/10.1080/15564880701263049.
- Moore GF, Anthony RE, Hawkins J, Van Godwin J, Murphy S, Hewitt G, et al. Socioeconomic status, mental wellbeing and transition to secondary school: analysis of the School Health Research Network/Health Behaviour in School-aged Children survey in Wales. Br Educ Res J 2020;46:1111-30. https://doi.org/10.1002/berj.3616.
- Bonell C, Allen E, Warren E, McGowan J, Bevilacqua L, Jamal F, et al. Effects of the Learning Together intervention on bullying and aggression in English secondary schools (INCLUSIVE): a cluster randomised controlled trial. Lancet 2018;392:2452-64. https://doi.org/10.1016/S0140-6736(18)31782-3.
- Henderson M, Ecob R, Wight D, Abraham C. What explains between-school differences in rates of smoking?. BMC Public Health 2008;8. https://doi.org/10.1186/1471-2458-8-218.
- Raab GM, Burston A, Henderson M, Storkey H, Davis J, Elliott L. Short-Term Evaluation of the Zero Tolerance Respect Package in Midlothian Primary Schools, A Randomized Controlled Trial. Report to Chief Scientist Office (Ref No. CZG2313) 2008.
- European Commission . Work Together to Stop Truancy Among Youth 2013. https://cordis. europa.eu/article/id/91454-reduced-truancy-improved-mental-health (accessed 1 April 2022).
- Lamblin M, Murawski C, Whittle S, Fornito A. Social connectedness, mental health and the adolescent brain. Neurosci Biobehav Rev 2017;80:57-68. https://doi.org/10.1016/j.neubiorev.2017.05.010.
- Lester L, Cross D. The relationship between school climate and mental and emotional wellbeing over the transition from primary to secondary school. Psychol Well Being 2015;5. https://doi.org/10.1186/s13612-015-0037-8.
- Sklad M, Diekstra R, Ritter MD, Ben J, Gravesteijn C. Effectiveness of school-based universal social, emotional, and behavioral programs: do they enhance students’ development in the area of skill, behavior, and adjustment?. Psychol Schools 2012;49:892-909. https://doi.org/10.1002/pits.21641.
- Santalahti P, Kampman M, Liski A, Appelqvist-Schmidlechner K, Bjorklund K, Solantaus T. School programme to promote children’s social and emotional competences and well being. Eur Child Adolesc Psych 2013;1:S179-80.
- Jackson CA, Henderson M, Frank JW, Haw SJ. An overview of prevention of multiple risk behaviour in adolescence and young adulthood. J Public Health 2012;34:i31-40. https://doi.org/10.1093/pubmed/fdr113.
- Kidd JB, McMahon AD, Sherriff A, Gnich W, Mahmoud A, Macpherson LM, et al. Evaluation of a national complex oral health improvement programme: a population data linkage cohort study in Scotland. BMJ Open 2020;10. https://doi.org/10.1136/bmjopen-2020-038116.
- Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol 2009;43:267-76. https://doi.org/10.1007/s10464-009-9229-9.
- Rutter H, Savona N, Glonti K, Bibby J, Cummins S, Finegood DT, et al. The need for a complex systems model of evidence for public health. Lancet 2017;390:2602-4. https://doi.org/10.1016/S0140-6736(17)31267-9.
- Moore GF, Evans RE, Hawkins J, Littlecott H, Melendez-Torres GJ, Bonell C, et al. From complex social interventions to interventions in complex social systems: future directions and unresolved questions for intervention development and evaluation. Evaluation 2019;25:23-45. https://doi.org/10.1177/1356389018803219.
- Durlak JA. Programme implementation in social and emotional learning: basic issues and research findings. Cambridge J Educ 2016;46:333-45. https://doi.org/10.1080/0305764X.2016.1142504.
- Evans R, Murphy S, Scourfield J. Implementation of a school-based social and emotional learning intervention: understanding diffusion processes within complex systems. Prev Sci 2015;16:754-64. https://doi.org/10.1007/s11121-015-0552-0.
- SHINE . Schools Health and Wellbeing Improvement Research Network 2020. https://shine.sphsu.gla.ac.uk/about/ (accessed 1 April 2022).
- Kidger J, Gunnell D, Biddle L, Campbell R, Donovan J. Part and parcel of teaching? Secondary school staff’s views on supporting student emotional health and well-being. Br Educ Res J 2009;36:919-35. https://doi.org/10.1080/01411920903249308.
- Curtis L, Burns A. Unit Costs of Health and Social Care 2015. Canterbury: PSSRU, University of Kent; 2015.
- NHS Digital . 2006/07/UK/General/Practice/Workload/Survey 2007.
- Curtis L. Unit Costs of Health and Social Care 2014. Canterbury: PSSRU, University of Kent; 2014.
- Scottish Office, Home and Health Department . HIV and AIDS in Scotland, Prevention the Key 1992.
- Tones K. Why theorise? Ideology in health promotion. Health Educ J 1986;49:2-6. https://doi.org/10.1177/001789699004900101.
- Tones K. Preventing drug misuse: the case for breadth, balance and coherence. Health Educ J 1986;45:223-30. https://doi.org/10.1177/001789698604500411.
Appendix 1 Literature search strategy
Academic databases from multiple disciplines were searched to inform the literature review. The following searches were conducted.
Database | Search terms |
---|---|
Campbell Collaboration | “Social emotional” |
“Primary school” | |
“SEL” | |
“restorative” | |
Cochrane Library | “school social emotional” in Reviews |
“School emotional” in Reviews | |
“social emotional” in Reviews | |
EBSCOhost: | “Social and emotional wellbeing” |
|
“SEL” AND “primary school” |
“wellbeing” AND “primary school” | |
“social emotional” AND “school” AND “intervention” AND “UK” [until 2014] | |
“SU transition” AND “SU school” AND “wellbeing” | |
“SU School transition” AND “primary school” | |
“SU School transition” AND “Scotland” | |
“SU School transition” AND “strengths and difficulties questionnaire” |
The literature was also informed by additional sources from Google Scholar (Google Inc., Mountain View, CA, USA) and the reference lists of relevant publications. The search for studies was carried out in January 2018.
Appendix 2 Primary and secondary outcome measures
Domain summary
Domain items from pupil self-completed and staff questionnaires, follow-up 2 (2016)
Pupils
SDQ | Question number | Question |
---|---|---|
Conduct problems | I get very angry and often lose my temper | |
I usually do as I am told | ||
I fight a lot. I can make other people do what I want | ||
I am often accused of lying or cheating | ||
I take things that are not mine from home, school or elsewhere | ||
Emotional symptoms | I get a lot of headaches, stomach aches or sickness | |
I worry a lot | ||
I am often unhappy, downhearted or tearful | ||
I am nervous in new situations. I easily lose confidence | ||
I have many fears, I am easily scared | ||
Hyperactivity/inattention | I am restless, I cannot stay still for long | |
I am constantly fidgeting or squirming | ||
I am easily distracted, I find it difficult to concentrate | ||
I think before I do things | ||
I finish the work I’m doing. My attention is good | ||
Peer problems | I am usually on my own. I generally play alone or keep to myself | |
I have one good friend or more | ||
Other people my age generally like me | ||
Other children or young people pick on me or bully me | ||
I get on better with adults than with people my own age | ||
Prosocial scale | I try to be nice to other people. I care about their feelings | |
I usually share with others (food, games, pens, etc.) | ||
I am helpful if someone is hurt, upset or feeling ill | ||
I am kind to younger children | ||
I often volunteer to help others (parents, teachers, children) |
2015 domain name | Question number | Question |
---|---|---|
Self awareness/resilience | 20 | I am sometimes confused about the way I feel |
23 | I am easily hurt by what others say about me | |
25 | When I feel upset, I don’t know how to talk about it | |
26 | I often do not know why I am angry | |
37 | At times I do not know how I am feeling | |
39 | At times I just don’t have words to describe how I feel | |
40 | When I fall out with friends I worry for days | |
90 | I find it difficult making new friends | |
Social awareness/empathy | 34 | I like to help people with their problems |
44 | I know when my friend is sad even if they don’t say so | |
45 | I try to cheer people up | |
46 | It makes me sad to see someone who is lonely | |
47 | I try to be nice to other people. I care about their feelings | |
50 | I usually share with others (food, games, pens, etc.) | |
55 | I am helpful if someone is hurt, upset or feeling ill | |
Self-esteem | 38 | I feel good about myself |
42 | I am happy with how I look | |
43 | I like myself | |
Emotional regulation | 18 | I can calm myself down quickly after getting angry |
31 | I do not like to talk about how I feel | |
51 | I get very angry and often lose my temper | |
Self-management | 21 | I give up trying if I find something hard |
30 | I work hard at school | |
33 | I complete my schoolwork regularly | |
36 | I can get on with my schoolwork even when I’m feeling sad or angry | |
53 | I usually do as I am told | |
56 | I am constantly fidgeting or squirming | |
61 | I am easily distracted, I find it difficult to concentrate | |
67 | I think before I do things | |
71 | I finish the work I’m doing. My attention is good | |
111 | At school I am well behaved | |
Responsible decision-making | 17 | I think I have good ideas |
22 | I can usually work out the way to deal with problems | |
24 | Once I have a goal, I make a plan to reach it | |
27 | I know what I’m good at and what I’m not so good at | |
28 | I try to learn from my mistakes | |
29 | I avoid things that are dangerous or unhealthy | |
32 | I would feel bad if I couldn’t finish something I’d promised to do | |
35 | I can say no to activities that I think are wrong | |
Materialism | 99 | Do you think it’s important to own expensive things? |
100 | Does buying expensive things make you happy? | |
101 | Do you like children who have expensive things more than you like other children? | |
Happy friendships | 41 | Most things work out OK in the end |
91 | I can talk to my friends about most things | |
94 | I enjoy spending time with my friends | |
97 | My friends are nice to me | |
98 | I would like to have more friends | |
School relationships (whole school) | 118 | Children in this school are kind to each other |
123 | Most adults in this school are kind to pupils | |
School relationships (in class) | 125 | Helps me when I need help |
126 | Understands my problems and worries | |
127 | Lets me choose how to do my school work | |
128 | Makes me feel better when I am upset | |
129 | Is happy to see me when I come to school | |
130 | Notices when I do things well | |
131 | Talks with me about how I can make my work better | |
Experiences of antisocial behaviour (by others) | 65 | Other children or young people pick on me or bully me |
95 | Other children hit or hurt me | |
96 | Other children pick on me | |
114 | Other children in my class make it hard for me to do my work | |
Participation in antisocial behaviour (towards others) | 58 | I fight a lot. I can make other people do what I want |
64 | I am often accused of lying or cheating | |
68 | I take things that are not mine from home, school or elsewhere | |
92 | I pick on other children | |
93 | I hurt or hit other children | |
Liking school | 112 | I look forward to going to school |
113 | I wish I didn’t have to go to school | |
115 | I enjoy learning at school | |
School climate | 116 | It is important to do well at school |
117 | I enjoy being at school | |
120 | In this school we are safe | |
121 | In this school we care | |
122 | Most adults in this school treat us fairly | |
Family relationships | 73 | My family get along well together |
74 | I can talk to my parents about most things | |
75 | My parents treat me fairly | |
76 | My parents and I do fun things together | |
77 | My parents listen to me | |
78 | My parents help me as much as I need | |
79 | My parents tell me if I behave well | |
80 | My parents help me with my homework | |
81 | My parents tell me when I do something well | |
Family time | 82 | Watch TV or DVDs together |
83 | Play games together | |
84 | Eat a meal together | |
85 | Go places together | |
86 | Go for a walk or play sports together | |
87 | Visit friends or relatives together | |
89 | Do other activities like music or cooking together |
Staff
Staff | Question number | Question |
---|---|---|
Perception of pupils – pupil behaviour | 63 | Pupils in my class/school … are well behaved |
78 | Pupils in my class/school … engage in antisocial behaviour | |
79 | Pupils in my class/school … behave in a way that gets me down | |
Perception of pupils – pupil confidence | 66 | Pupils in my class/school … are reluctant to try new things |
70 | Pupils in my class/school … are confident individuals | |
71 | Pupils in my class/school … have confidence in relation to learning activities | |
Perception of pupils – pupil engagement | 69 | Pupils in my class/school … are motivated to learn |
72 | Pupils in my class/school … have a sense of belonging to the school | |
73 | Pupils in my class/school … have a voice in the class/school | |
75 | Pupils in my class/school … have chances to help decide and plan activities | |
77 | Pupils in my class/school … have the opportunity to engage in creative activities at this school (e.g. sports, arts, drama and music) | |
Perception of pupils – pupil relationships | 64 | Pupils in my class/school … have respect for themselves |
68 | Pupils in my class/school … respect each other | |
76 | Pupils in my class/school … have developed good peer relationships | |
School ethos | 21 | I enjoy working at my school |
23 | There is a sense of co-operation and trust in my school | |
24 | There is a positive ethos in my school | |
27 | At my school staff care about each other | |
29 | My school is a safe place for pupils | |
31 | Staff share a common vision about the school | |
34 | There is a strong sense of morale among school staff | |
41 | Staff accomplish their jobs with enthusiasm | |
Staff–pupil relationships | 38 | Staff respect pupils |
65 | Pupils in my class/school … have respect for school staff | |
67 | Pupils in my class/school … are asked by me for their views on how their learning experiences could be improved | |
74 | Pupils in my class/school … are provided with feedback on what they need to do to improve | |
Staff relationships | 32 | I get on with most staff in my school most of the time |
33 | The staff room is a positive and encouraging place to be | |
36 | My relationships with others at work are good | |
37 | Staff communicate effectively with each other | |
39 | Staff respect each other | |
40 | Staff exhibit friendliness to each other | |
Perceptions of management | 26 | The demands made of me are manageable |
44 | In my school, change is managed effectively | |
46 | School management ask for staff views | |
47 | School management act on the views of staff | |
51 | All staff are involved in the decision-making at my school | |
53 | Leadership at all levels is effective in this school | |
Staff support | 25 | My working environment allows me to carry out my job comfortably and safely |
43 | I have appropriate support from my line manager | |
45 | I find it hard to ask for support when I need it | |
48 | Support is available to us as employees in ways of dealing with grievances, conflicts or harassment | |
49 | Support is available to us as employees in terms of counselling and advice | |
54 | I feel confident asking for support | |
55 | This school cares about my well-being | |
56 | This school does enough to look after my well-being | |
57 | Staff are protected from unreasonable community and parental demands | |
Training opportunities | 13 | I get the training I need to help me do my job well |
52 | I have good opportunities to take part in continuing professional development | |
Valued team member | 12 | I feel part of the team at my school |
20 | I feel valued at school | |
Learning community involvement | 60 | The school optimises links with other cluster primary schools to improve pupil well-being |
61 | The school optimises links with our associated secondary schools to improve pupil well-being | |
Community involvement | 58 | The school has appropriate involvement from parents |
59 | The school has appropriate involvement from the community | |
Self-efficacy | 7 | I feel confident that I have the skills to promote positive behaviour in my school |
8 | I believe I make a positive difference to the children I work with | |
9 | I feel confident when working with professionals from other agencies | |
14 | I am confident I am meeting my responsibilities under CfE to promote the health and well-being of all pupils | |
17 | I am confident that I can identify pupils who have social and emotional well-being problems | |
Role definition | 6 | I have a clearly defined role at my school |
11 | I have a clear understanding of how I should carry out my job | |
School support for SEW | 15 | I believe that teaching social and emotional skills is just as important as any other subject |
16 | My school believes that teaching social and emotional skills is just as important as any other subject | |
28 | My school aims to help pupils with emotional and behavioural difficulties | |
35 | Most staff in my school consistently model the social, emotional and behavioural skills that we want to develop in our children | |
SEW training | 18 | I have received initial teacher training on personal development in terms of my own social and emotional well-being |
19 | I have received in-service training on personal development in terms of my own social and emotional well-being |
Appendix 3 Outcome evaluation
SDQ score | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
Baseline SDQ | p = 0.003 | |||||||||||
Normal | 4.93 (4.46) | 4.38 (4.20) | 7.79 (6.36) | 8.26 (6.39) | −1.15 (−1.68 to −0.62)** | 7.89 (6.78) | 7.31 (6.14) | −0.10 (−0.67 to 0.47) | 7.88 (6.39) | 7.95 (6.86) | −0.84 (−1.40 to −0.28)* | |
Not normal | 20.44 (4.02) | 19.99 (3.80) | 12.92 (7.31) | 16.20 (7.09) | −3.83 (−5.47 to −2.19)** | 12.22 (7.87) | 15.03 (6.68) | −2.45 (−4.32 to −0.59)* | 13.23 (7.42) | 13.77 (6.02) | −0.93 (−2.76 to 0.90) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
Pupils with data at T0 (on treatment) | 1.62 (2.28) | 1.11 (1.78) | 2.24 (2.43) | 2.40 (2.45) | −0.62 (−0.82 to−0.43)** | 2.09 (2.37) | 2.00 (2.25) | −0.27 (−0.51 to −0.05)* | 2.22 (2.40) | 2.18 (2.38) | −0.48 (−0.71 to −0.52)** | |
All available data (intention to treat) | 1.62 (2.28) | 1.11 (1.78) | 2.24 (2.43) | 2.43 (2.45) | −0.64 (−0.84 to −0.45)** | 2.12 (2.38) | 2.04 (2.26) | −0.28 (−0.51 to −0.06)* | 2.24 (2.40) | 2.22 (2.39) | −0.48 (−0.71 to −0.26)** | |
Additional subgroup analysis | ||||||||||||
Cohort | p = 0.056 | |||||||||||
YC | 1.67 (2.34) | 1.04 (1.68) | 1.41 (2.09) | 1.69 (2.17) | −0.84 (−1.12 to −0.56)** | 1.58 (2.17) | 1.41 (1.96) | −0.34 (−0.63 to −0.06)* | 1.57 (2.03) | 1.41 (2.01) | −0.45 (−0.73 to −0.16)* | |
OC | 1.55 (2.20) | 1.18 (1.88) | 3.19 (2.45) | 3.23 (2.49) | −0.36 (−0.65 to −0.07)* | 2.93 (2.45) | 2.95 (2.36) | −0.08 (−0.41 to 0.25) | 3.02 (2.57) | 3.26 (2.44) | −0.46 (−0.77 to −0.15)* | |
Gender | p = 0.781 | |||||||||||
Male | 1.60 (2.28) | 1.10 (1.80) | 2.04 (2.31) | 2.24 (2.33) | −0.65 (−0.94 to −0.37)** | 1.90 (2.20) | 1.75 (2.04) | −0.22 (−0.53 to 0.09)** | 1.90 (2.20) | 1.97 (2.24) | −0.55 (−0.85 to −0.24)** | |
Female | 1.64 (2.28) | 1.12 (1.77) | 2.44 (2.53) | 2.62 (2.56) | −0.65 (−0.95 to −0.36)** | 2.35 (2.54) | 2.32 (2.42) | −0.31 (−0.63 to 0.01)*** | 2.58 (2.55) | 2.46 (2.51) | −0.39 (−0.70 to −0.08)* | |
Deprivation level | p = 0.001 | |||||||||||
Deprived | 1.75 (2.35) | 1.25 (1.95) | 2.17 (2.35) | 2.48 (2.44) | −0.70 (−0.98 to −0.42)** | 2.36 (2.52) | 1.88 (2.20) | 0.09 (−0.22 to 0.39) | 2.36 (2.39) | 2.19 (2.34) | −0.32 (−0.32 to −0.02)* | |
Not deprived | 1.46 (2.18) | 0.97 (1.57) | 2.31 (2.52) | 2.38 (2.47) | −0.60 (−0.91 to −0.30)** | 1.80 (2.14) | 2.19 (2.31) | −0.67 (−1.0 to −0.34)** | 2.11 (2.41) | 2.24 (2.45) | −0.63 (−0.95 to −0.32)** | |
SDQ score at baseline | p = 0.635 | |||||||||||
Normal | 1.09 (1.50) | 0.87 (1.33) | 2.12 (2.35) | 2.33 (2.42) | −0.46 (−0.67 to −0.26)** | 2.01 (2.30) | 1.94 (2.22) | −0.09 (−0.31 to 0.13) | 2.14 (2.36) | 2.12 (2.35) | −0.26 (−0.47 to −0.04)* | |
Not normal | 7.32 (1.32) | 7.06 (1.24) | 3.59 (2.91) | 4.14 (2.54) | −0.71 (−1.58 to 0.17) | 3.15 (2.96) | 3.95 (2.41) | −0.49 (−1.44 to 0.47) | 3.15 (2.71) | 3.85 (2.56) | −0.87 (−1.82 to 0.07)** |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
Pupils with data at T0 (on treatment) | 0.93 (1.75) | 0.82 (1.60) | 1.38 (1.80) | 1.46 (1.86) | −0.21 (−0.35 to −0.07)* | 1.36 (1.87) | 1.29 (1.74) | −0.09 (−0.25 to 0.08) | 1.38 (1.18) | 1.50 (1.97) | 0.31 (−0.47 to −0.15)** | |
All available data (intention to treat) | 0.93 (1.75) | 0.82 (1.60) | 1.40 (1.84) | 1.49 (1.88) | −0.22 (−0.36 to −0.08)* | 1.40 (1.89) | 1.30 (1.76) | −0.06 (−0.22 to 0.01) | 1.40 (1.83) | 1.83 (2.02) | −0.31 (−0.47 to −0.15)** | |
Additional subgroup analysis | ||||||||||||
Cohort | p < 0.001 | |||||||||||
YC | 0.85 (1.68) | 0.72 (1.39) | 0.84 (1.56) | 0.96 (1.78) | −0.29 (−0.49 to −0.10)* | 0.97 (1.79) | 0.79 (1.53) | 0.01 (−0.19 to 0.21) | 1.01 (1.81) | 0.90 (1.78) | −0.07 (−0.27 to 0.14) | |
OC | 1.02 (1.82) | 0.92 (1.79) | 2.05 (1.91) | 2.05 (1.83) | −0.13 (−0.34 to 0.07) | 2.05 (1.84) | 2.05 (1.79) | −0.12 (−0.25 to 0.12) | 1.87 (1.73) | 2.34 (2.02) | −0.58 (−0.80 to −0.36)** | |
Gender | p = 0.085 | |||||||||||
Male | 1.26 (2.02) | 1.07 (1.79) | 1.80 (1.97) | 1.90 (2.08) | −0.34 (−0.54 to −0.14)** | 1.77 (2.05) | 1.66 (1.95) | −0.11 (−0.32 to 0.11) | 1.78 (2.05) | 2.07 (2.31) | −0.51 (−0.72 to −0.29)** | |
Female | 0.58 (1.33) | 0.56 (1.33) | 1.00 (1.59) | 1.08 (1.57) | −0.15 (−0.35 to 0.06) | 1.00 (1.60) | 0.95 (1.46) | −0.03 (−0.25 to 0.19) | 1.02 (1.47) | 1.01 (1.50) | −0.13 (−0.34 to 0.09) | |
Deprivation level | p = 0.020 | |||||||||||
Deprived | 1.14 (1.95) | 0.91 (1.71) | 1.49 (1.89) | 1.62 (1.96) | −0.35 (−0.55 to −0.15)* | 1.66 (2.04) | 1.41 (1.83) | −0.02 (−0.24 to 0.19) | 1.53 (1.92) | 1.74 (2.15) | −0.49 (−0.70 to −0.28)** | |
Not deprived | 0.66 (1.42) | 0.72 (1.47) | 1.30 (1.77) | 1.35 (1.79) | −0.13 (−0.34 to 0.09) | 1.06 (1.59) | 1.20 (1.68) | −0.13 (−0.36 to 0.10) | 1.27 (1.71) | 1.31 (1.84) | −0.13 (−0.35 to 0.09) | |
SDQ score at baseline | p = 0.031 | |||||||||||
Normal | 0.46 (0.85) | 0.42 (0.84) | 1.21 (1.67) | 1.28 (1.70) | −0.15 (−0.29 to −0.01)* | 1.24 (1.77) | 1.14 (1.60) | −0.02 (−0.18 to 0.13) | 1.25 (1.69) | 1.40 (1.91) | −0.27 (−0.42 to −0.12)** | |
Not normal | 5.51 (1.64) | 5.29 (1.31) | 3.20 (2.19) | 3.68 (2.22) | −0.84 (−1.32 to −0.35)* | 3.02 (2.43) | 3.23 (2.19) | −0.33 (−0.88 to 0.22) | 3.07 (2.36) | 2.99 (2.20) | −0.24 (0.78 to 0.30) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
Pupils with data at T0 (on treatment) | 3.14 (3.13) | 2.80 (3.07) | 3.36 (2.88) | 3.44 (2.83) | −0.44 (−0.65 to −0.23) | 3.36 (2.88) | 3.44 (2.83) | −0.44 (−0.65 to −0.23) | 3.44 (2.89) | 3.23 (2.92) | −0.23 (−0.49 to 0.22) | |
All available data (intention to treat) | 3.14 (3.13) | 2.80 (3.07) | 3.41 (2.90) | 3.50 (2.83) | −0.45 (−0.66 to −0.24)** | 3.53 (2.90) | 3.28 (2.91) | −0.20 (−0.45 to 0.51) | 3.44 (2.85) | 3.41 (3.02) | −0.37 (−0.62 to −0.11) | |
Additional subgroup analysis | ||||||||||||
Cohort | p < 0.001 | |||||||||||
YC | 3.29 (3.21) | 2.88 (3.07) | 3.12 (3.22) | 3.20 (3.13) | −0.59 (−0.90 to −0.28)** | 3.20 (3.08) | 2.68 (2.95) | −0.01 (−0.32 to 0.31) | 2.99 (2.99) | 2.52 (3.05) | 0.00 (−0.32 to 0.32) | |
OC | 2.98 (3.04) | 2.72 (3.07) | 3.76 (2.45) | 3.82 (2.41) | −0.30 (−0.62 to 0.02)*** | 4.03 (2.54) | 4.16 (2.61) | −0.44 (−0.80 to 0.07)* | 3.97 (2.58) | 4.57 (2.57) | −0.84 (−1.18 to −0.50)** | |
Gender | p = 0.683 | |||||||||||
Male | 4.09 (3.38) | 3.72 (3.30) | 4.24 (2.93) | 4.14 (2.94) | −0.36 (−0.67 to −0.04)* | 4.33 (2.91) | 4.01 (3.03) | −0.22 (−0.56 to 0.12) | 4.27 (2.89) | 4.21 (3.11) | −0.40 (−0.73 to −0.06)* | |
Female | 2.14 (2.49) | 1.87 (2.50) | 2.56 (2.61) | 2.87 (2.56) | −0.59 (−0.91 to −0.27)** | 2.68 (2.64) | 2.57 (2.60) | −0.22 (−0.56 to 0.13) | 2.60 (2.55) | 2.64 (2.72) | −0.39 (−0.72 to −0.05)* | |
Deprivation level | p = 0.412 | |||||||||||
Deprived | 3.47 (3.20) | 2.98 (3.16) | 3.61 (2.93) | 3.63 (2.88) | −0.47 (−0.78 to −0.16)* | 3.84 (2.91) | 3.44 (2.98) | −0.11 (−0.44 to 0.22) | 3.84 (2.83) | 3.53 (3.12) | −0.22 (−0.55 to 0.11) | |
Not deprived | 2.74 (3.00) | 2.62 (2.96) | 3.19 (2.85) | 3.35 (2.76) | −0.48 (−0.81 to −0.15)* | 3.12 (2.84) | 3.12 (2.83) | −0.34 (−0.70 to 0.02)*** | 3.02 (2.81) | 3.29 (2.92) | −0.57 (−0.92 to −0.23)* | |
SDQ score at baseline | p = 0.091 | |||||||||||
Normal | 4.45 (4.54) | 3.53 (3.67) | 7.45 (6.33) | 7.85 (6.29) | −0.46 (−0.70 to −0.23)** | 7.56 (6.81) | 6.87 (6.06) | −0.07 (−0.32 to 0.18) | 7.59 (6.22) | 7.41 (6.57) | −0.26 (−0.51 to −0.01)* | |
Not normal | 16.37 (5.77) | 14.72 (5.43) | 12.25 (6.73) | 13.38 (7.05) | −0.13 (−0.61 to 0.35) | 11.76 (6.96) | 12.28 (6.51) | −0.48 (−1.01 to 0.05)*** | 12.30 (7.49) | 12.94 (7.00) | −0.44 (−0.96 to 0.09) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
Pupils with data at T0 (on treatment) | 1.30 (1.89) | 0.96 (1.64) | 1.44 (1.84) | 1.57 (1.84) | −0.47 (−0.63 to −0.31) | 1.47 (1.97) | 1.31 (1.78) | −0.19 (−0.37 to −0.01) | 1.49 (1.84) | 1.31 (1.77) | −0.20 (−0.38 to −0.03) | |
All available data (intention to treat) | 1.30 (1.89) | 0.96 (1.64) | 1.47 (1.85) | 1.62 (1.86) | −0.48 (−0.63 to −0.33) | 1.47 (1.95) | 1.36 (1.80) | −0.23 (−0.41 to −0.05) | 1.52 (1.85) | 1.35 (1.79) | −0.19 (−0.37 to −0.02) | |
Additional subgroup analysis | ||||||||||||
Cohort | p = 0.018 | |||||||||||
YC | 1.19 (1.77) | 0.85 (1.45) | 1.09 (1.67) | 1.34 (1.75) | −0.58 (−0.80 to −0.36)** | 1.27 (1.93) | 1.05 (1.67) | −0.10 (−0.32 to 0.13) | 1.32 (1.92) | 0.98 (1.63) | −0.05 (−0.28 to 0.17) | |
OC | 1.43 (2.02) | 1.06 (1.81) | 1.91 (1.94) | 1.91 (1.93) | −0.32 (−0.55 to −0.09)* | 1.78 (1.94) | 1.81 (1.88) | −0.25 (−0.51 to 0.01)*** | 1.75 (1.74) | 1.82 (1.87) | −0.31 (−0.55 to −0.06)* | |
Gender | p = 0.208 | |||||||||||
Male | 1.48 (2.00) | 1.05 (1.74) | 1.48 (1.83) | 1.66 (1.86) | −0.60 (−0.82 to −0.37)** | 1.60 (1.97) | 1.47 (1.88) | −0.27 (−0.51 to −0.04)* | 1.63 (1.91) | 1.52 (1.92) | −0.34 (−0.57 to −0.10)* | |
Female | 1.11 (1.75) | 0.86 (1.53) | 1.45 (1.86) | 1.57 (1.86) | −0.33 (−0.56 to −0.11)* | 1.34 (1.92) | 1.25 (1.71) | −0.06 (−0.31 to 0.18) | 1.40 (1.79) | 1.17 (1.64) | −0.01 (−0.45 to 0.23) | |
Deprivation level | p = 0.003 | |||||||||||
Deprived | 1.52 (2.03) | 0.95 (1.67) | 1.54 (1.96) | 1.67 (1.89) | −0.67 (−0.88 to −0.45)** | 1.65 (2.06) | 1.34 (1.80) | −0.18 (−0.41 to 0.05) | 1.63 (1.99) | 1.43 (1.83) | −0.41 (−0.65 to −0.18)** | |
Not deprived | 1.03 (1.67) | 0.96 (1.61) | 1.39 (1.70) | 1.55 (1.82) | −0.24 (−0.47 to −0.01)* | 1.24 (1.76) | 1.38 (1.79) | −0.15 (−0.40 to 0.10) | 1.40 (1.69) | 1.26 (1.74) | 0.09 (−0.15 to 0.33) | |
SDQ score at baseline | p = 0.011 | |||||||||||
Normal | 5.24 (5.18) | 4.78 (5.04) | 7.86 (6.36) | 8.48 (6.64) | −0.33 (−0.49 to −0.17)** | 7.96 (6.84) | 7.39 (6.23) | 0.03 (−0.14 to 0.20) | 8.05 (6.50) | 8.07 (6.91) | −0.05 (−0.22 to 0.11) | |
Not normal | 17.35 (6.45) | 15.21 (7.05) | 11.97 (7.64) | 13.13 (6.80) | 0.03 (−0.44 to 0.49) | 11.13 (7.71) | 13.15 (7.03) | −0.40 (−0.92 to 0.12) | 11.52 (7.54) | 11.45 (6.78) | 0.39 (−0.12 to 0.90) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
Pupils with data at T0 (on treatment) | 7.83 (2.45) | 8.08 (2.26) | 7.93 (2.26) | 7.93 (2.18) | 0.23 (0.03 to 0.43)* | 7.64 (2.25) | 7.83 (2.19) | 0.08 (−0.15 to 0.31) | 7.66 (2.23) | 7.65 (2.28) | 0.26 (0.03 to 0.51)* | |
All available data (intention to treat) | 7.83 (2.45) | 8.08 (2.26) | 7.88 (2.28) | 7.89 (2.24) | 0.23 (0.04 to 0.43)* | 7.60 (2.26) | 7.78 (2.24) | 0.08 (−0.14 to 0.31) | 7.62 (2.26) | 7.63 (2.30) | 0.22 (−0.00 to 0.45)*** | |
Additional subgroup analysis | ||||||||||||
Cohort | p < 0.001 | |||||||||||
YC | 7.88 (2.43) | 8.10 (2.14) | 7.69 (2.60) | 7.63 (2.57) | 0.28 (0.01 to 0.56)* | 7.62 (2.46) | 7.97 (2.42) | −0.18 (−0.47 to 0.11) | 7.86 (2.41) | 8.17 (2.27) | −0.14 (−0.43 to 0.15) | |
OC | 7.77 (2.48) | 8.06 (2.38) | 8.11 (1.80) | 8.17 (1.76) | 0.21 (−0.08 to 0.50) | 7.57 (1.92) | 7.52 (1.93) | 0.31 (−0.02 to 0.64)*** | 7.34 (2.03) | 6.94 (2.15) | 0.75 (0.44 to 1.06)** | |
Gender | p = 0.247 | |||||||||||
Male | 7.14 (2.61) | 7.47 (2.46) | 7.26 (2.46) | 7.39 (2.39) | 0.24 (−0.05 to 0.52) | 6.92 (2.36) | 7.16 (2.39) | 0.01 (−0.30 to 0.31) | 6.94 (2.37) | 6.82 (2.46) | 0.45 (0.15 to 0.74)* | |
Female | 8.57 (2.02) | 8.70 (1.83) | 8.51 (1.88) | 8.38 (1.95) | 0.22 (−0.07 to 0.51) | 8.33 (1.90) | 8.39 (1.91) | 0.06 (−0.26 to 0.37) | 8.31 (1.91) | 8.44 (1.80) | 0.08 (−0.23 to 0.38) | |
Deprivation level | p = 0.571 | |||||||||||
Deprived | 7.63 (2.59) | 8.04 (2.30) | 7.84 (2.37) | 7.87 (2.29) | 0.35 (0.07 to 0.63)* | 7.46 (2.36) | 7.71 (2.41) | 0.08 (−0.22 to 0.38) | 7.57 (2.30) | 7.61 (2.36) | 0.38 (0.08 to 0.68)* | |
Not deprived | 8.08 (2.25) | 8.12 (2.22) | 7.93 (2.16) | 7.91 (2.18) | 0.09 (−0.20 to 0.39) | 7.79 (2.10) | 7.86 (2.07) | −0.02 (−0.34 to 0.30) | 7.67 (2.21) | 7.66 (2.24) | 0.13 (−0.18 to 0.44) | |
SDQ score at baseline | p = 0.045 | |||||||||||
Normal | 5.02 (5.20) | 4.29 (4.71) | 7.67 (6.43) | 8.23 (6.54) | 0.05 (−0.15 to 0.26) | 7.76 (6.90) | 7.26 (6.12) | −0.21 (−0.43 to 0.02)*** | 7.57 (6.28) | 7.75 (6.73) | −0.00 (−0.22 to 0.22) | |
Not normal | 14.86 (7.07) | 13.12 (6.65) | 11.46 (6.91) | 12.53 (7.01) | 0.26 (−0.20 to 0.71) | 11.06 (6.97) | 11.48 (7.43) | 0.53 (0.00 to 1.03)* | 12.17 (7.20) | 12.02 (7.26) | 0.51 (0.03 to 1.00)* |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 3.29 (0.80) | 3.32 (0.80) | 3.21 (0.84) | 3.21 (0.85) | 0.04 (−0.07 to 0.15) | 3.12 (0.89) | 3.01 (0.89) | 0.11 (−0.01 to 0.24)*** | 2.91 (0.89) | 2.87 (0.94) | 0.07 (−0.04 to 0.19) | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.190 | |||||||||||
Male | 3.37 (0.76) | 3.34 (0.77) | 3.39 (0.70) | 3.29 (0.80) | 0.08 (−0.07 to 0.23) | 3.38 (0.70) | 3.16 (0.86) | 0.17 (0.01 to 0.34)* | 3.27 (0.77) | 3.25 (0.82) | 0.01 (−0.13 to 0.15) | |
Female | 3.21 (0.84) | 3.31 (0.83) | 3.01 (0.93) | 3.12 (0.88) | 0.00 (−0.15 to 0.15) | 2.81 (0.98) | 2.86 (0.90) | 0.04 (−0.13 to 0.22) | 2.98 (0.94) | 3.03 (0.93) | 0.10 (−0.04 to 0.25) | |
Deprivation level | p = 0.115 | |||||||||||
Deprived | 3.33 (0.82) | 3.34 (0.80) | 3.20 (0.87) | 3.25 (0.84) | −0.05 (−0.20 to 0.10) | 3.11 (0.89) | 3.07 (0.88) | 0.03 (−0.14 to 0.20) | 3.12 (0.89) | 3.19 (0.87) | −0.07 (−0.21 to 0.08) | |
Not deprived | 3.25 (0.79) | 3.31 (0.80) | 3.21 (0.82) | 3.16 (0.85) | 0.14 (−0.02 to 0.29)*** | 3.13 (0.89) | 2.96 (0.91) | 0.20 (0.03 to 0.38)* | 3.13 (0.85) | 3.09 (0.89) | 0.18 (0.04 to 0.33)* |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 2.69 (1.04) | 2.74 (0.99) | 2.86 (0.99) | 2.84 (0.98) | 0.05 (−0.07 to 0.17) | 2.82 (1.05) | 2.80 (0.98) | 0.07 (−0.07 to 0.20) | 2.87 (0.94) | 2.73 (0.94) | 0.16 (0.03 to 0.28)* | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.545 | |||||||||||
Male | 2.53 (1.05) | 2.55 (0.98) | 2.68 (1.00) | 2.66 (1.01) | 0.03 (−0.13 to 0.20) | 2.72 (1.05) | 2.66 (1.01) | 0.06 (−0.13 to 0.25) | 2.70 (1.00) | 2.59 (0.98) | 0.14 (−0.02 to 0.30)*** | |
Female | 2.87 (0.99) | 2.94 (0.96) | 3.05 (0.95) | 3.03 (0.90) | 0.07 (−0.11 to 0.24) | 2.94 (1.03) | 2.96 (0.92) | 0.04 (−0.15 to 0.24) | 2.87 (0.96) | 2.94 (0.93) | 0.02 (−0.15 to 0.19) | |
Deprivation level | p = 0.789 | |||||||||||
Deprived | 2.63 (1.05) | 2.61 (1.01) | 2.80 (1.00) | 2.73 (0.99) | 0.04 (−0.13 to 0.21) | 2.73 (1.04) | 2.69 (0.99) | −0.01 (−0.19 to 0.18) | 2.73 (0.98) | 2.69 (1.00) | 0.05 (−0.11 to 0.21) | |
Not deprived | 2.77 (1.01) | 2.86 (0.96) | 2.92 (0.98) | 2.96 (0.95) | 0.06 (−0.12 to 0.23) | 2.96 (1.04) | 2.91 (0.95) | 0.12 (−0.08 to 0.32) | 2.85 (0.99) | 2.84 (0.93) | 0.11 (−0.06 to 0.28) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 2.87 (0.65) | 2.86 (0.64) | 3.01 (0.66) | 2.98 (0.68) | 0.02 (−0.06 to 0.10) | 3.15 (0.68) | 3.11 (0.68) | −0.01 (−0.10 to 0.08) | 3.17 (0.67) | 3.09 (0.67) | 0.05 (−0.04 to 0.13) | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.453 | |||||||||||
Male | 2.94 (0.66) | 2.97 (0.61) | 3.09 (0.65) | 3.08 (0.64) | 0.03 (−0.08 to 0.14) | 3.30 (0.60) | 3.25 (0.63) | 0.04 (−0.09 to 0.16) | 3.14 (0.63) | 3.08 (0.64) | 0.07 (−0.04 to 0.18) | |
Female | 2.79 (0.62) | 2.74 (0.66) | 2.91 (0.66) | 2.86 (0.71) | 0.00 (−0.11 to 0.12) | 2.97 (0.72) | 2.95 (0.70) | −0.07 (−0.20 to 0.07) | 2.88 (0.70) | 2.89 (0.65) | −0.04 (−0.15 to 0.07) | |
Deprivation level | p = 0.789 | |||||||||||
Deprived | 2.82 (0.67) | 2.81 (0.65) | 3.00 (0.63) | 2.98 (0.70) | 0.04 (−0.13 to 0.21) | 3.14 (0.68) | 3.11 (0.71) | −0.01 (−0.19 to 0.18) | 2.99 (0.69) | 3.02 (0.65) | 0.05 (−0.11 to 0.21) | |
Not deprived | 2.92 (0.61) | 2.91 (0.63) | 3.01 (0.69) | 2.98 (0.66) | 0.06 (−0.12 to 0.23) | 3.16 (0.67) | 3.10 (0.65) | 0.12 (−0.08 to 0.32) | 3.02 (0.68) | 2.95 (0.65) | 0.11 (−0.06 to 0.28) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 3.37 (0.58) | 3.36 (0.59) | 3.38 (0.56) | 3.40 (0.56) | −0.02 (−0.09 to 0.06) | 3.27 (0.59) | 3.26 (0.63) | 0.03 (−0.06 to 0.11) | 3.21 (0.62) | 3.11 (0.65) | 0.09 (0.01 to 0.17)* | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.812 | |||||||||||
Male | 3.25 (0.61) | 3.22 (0.661) | 3.27 (0.59) | 3.27 (0.62) | −0.03 (−0.14 to 0.07) | 3.12 (0.59) | 3.07 (0.67) | 0.03 (−0.08 to 0.15) | 3.15 (0.62) | 3.08 (0.68) | 0.07 (−0.03 to 0.16) | |
Female | 3.50 (0.51) | 3.52 (0.54) | 3.51 (0.50) | 3.54 (0.44) | −0.00 (−0.11 to 0.11) | 3.45 (0.54) | 3.46 (0.51) | 0.02 (−0.10 to 0.14) | 3.47 (0.50) | 3.48 (0.53) | 0.03 (−0.07 to 0.13) | |
Deprivation level | p = 0.416 | |||||||||||
Deprived | 3.34 (0.60) | 3.31 (0.64) | 3.33 (0.60) | 3.37 (0.59) | −0.05 (−0.16 to 0.05) | 3.22 (0.61) | 3.25 (0.64) | −0.04 (−0.16 to 0.08) | 3.29 (0.60) | 3.29 (0.66) | 0.02 (−0.08 to 0.12) | |
Not deprived | 3.41 (0.54) | 3.41 (0.55) | 3.44 (0.52) | 3.43 (0.52) | 0.02 (−0.09 to 0.12) | 3.35 (0.56) | 3.27 (0.61) | 0.10 (−0.02 to 0.23)*** | 3.33 (0.57) | 3.27 (0.62) | 0.07 (−0.03 to 0.17) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 3.08 (0.60) | 3.12 (0.59) | 3.16 (0.57) | 3.14 (0.59) | 0.07 (0.01 to 0.14)* | 3.08 (0.60) | 3.03 (0.65) | 0.12 (0.05 to 0.20)* | 3.08 (0.62) | 2.90 (0.66) | 0.21 (0.13 to 0.28)** | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.320 | |||||||||||
Male | 2.95 (0.61) | 2.99 (0.59) | 3.03 (0.58) | 3.00 (0.62) | 0.06 (−0.04 to 0.15) | 2.98 (0.59) | 2.87 (0.70) | 0.16 (0.05 to 0.26)* | 3.02 (0.59) | 2.91 (0.64) | 0.18 (0.09 to 0.27)** | |
Female | 3.22 (0.55) | 3.27 (0.56) | 3.32 (0.53) | 3.28 (0.51) | 0.08 (−0.02 to 0.18) | 3.20 (0.59) | 3.19 (0.55) | 0.06 (−0.05 to 0.17) | 3.21 (0.59) | 3.17 (0.58) | 0.11 (0.01 to 0.20)* | |
Deprivation level | p = 0.340 | |||||||||||
Deprived | 3.02 (0.62) | 3.08 (0.60) | 3.11 (0.58) | 3.06 (0.61) | 0.10 (0.01 to 0.19)* | 3.02 (0.62) | 2.92 (0.67) | 0.15 (0.05 to 0.26)* | 3.04 (0.60) | 3.01 (0.64) | 0.13 (0.04 to 0.22)* | |
Not deprived | 3.15 (0.56) | 3.16 (0.58) | 3.23 (0.56) | 3.22 (0.56) | 0.04 (−0.06 to 0.13) | 3.18 (0.56) | 3.12 (0.61) | 0.07 (−0.04 to 0.19) | 3.20 (0.58) | 3.06 (0.61) | 0.16 (0.07 to 0.25)* |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 3.01 (0.55) | 3.06 (0.54) | 3.17 (0.51) | 3.15 (0.51) | 0.05 (−0.02 to 0.12) | 3.13 (0.54) | 3.06 (0.54) | 0.12 (0.04 to 0.19)* | 3.06 (0.54) | 2.94 (0.59) | 0.15 (0.08 to 0.22)** | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.004 | |||||||||||
Male | 2.97 (0.58) | 2.99 (0.53) | 3.11 (0.50) | 3.11 (0.53) | 0.02 (−0.08 to 0.11) | 3.11 (0.54) | 2.99 (0.59) | 0.15 (0.04 to 0.25)* | 3.07 (0.52) | 2.93 (0.58) | 0.17 (0.08 to 0.26)** | |
Female | 3.07 (0.51) | 3.13 (0.55) | 3.23 (0.52) | 3.20 (0.48) | 0.09 (−0.01 to 0.18)*** | 3.15 (0.53) | 3.12 (0.49) | 0.08 (−0.04 to 0.19) | 3.08 (0.53) | 3.13 (0.53) | 0.01 (−0.08 to 0.10) | |
Deprivation level | p = 0.065 | |||||||||||
Deprived | 2.98 (0.58) | 2.99 (0.56) | 3.09 (0.54) | 3.11 (0.54) | −0.02 (−0.12 to 0.07) | 3.07 (0.55) | 3.01 (0.57) | 0.07 (−0.04 to 0.17) | 3.01 (0.54) | 3.03 (0.57) | 0.01 (−0.08 to 0.10) | |
Not deprived | 3.06 (0.50) | 3.12 (0.52) | 3.26 (0.46) | 3.20 (0.47) | 0.12 (0.03 to 0.22)* | 3.22 (0.50) | 3.10 (0.52) | 0.17 (0.05 to 0.28)* | 3.14 (0.50) | 3.03 (0.56) | 0.18 (0.08 to 0.27)** |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 3.37 (0.61) | 3.37 (0.62) | 3.23 (0.56) | 3.23 (0.53) | −0.01 (−0.09 to 0.07) | 3.28 (0.56) | 3.23 (0.58) | 0.02 (−0.07 to 0.11) | 3.27 (0.53) | 3.19 (0.56) | 0.07 (−0.02 to 0.15) | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.632 | |||||||||||
Male | 3.32 (0.63) | 3.34 (0.64) | 3.20 (0.55) | 3.21 (0.52) | −0.01 (−0.12 to 0.10) | 3.26 (0.55) | 3.19 (0.60) | 0.04 (−0.08 to 0.17) | 3.20 (0.52) | 3.14 (0.60) | 0.07 (−0.03 to 0.17) | |
Female | 3.42 (0.58) | 3.40 (0.59) | 3.26 (0.57) | 3.26 (0.55) | −0.01 (−0.13 to 0.10) | 3.29 (0.57) | 3.27 (0.55) | −0.01 (−0.14 to 0.12) | 3.24 (0.56) | 3.25 (0.53) | −0.01 (−0.12 to 0.09) | |
Deprivation level | p = 0.075*** | |||||||||||
Deprived | 3.34 (0.62) | 3.33 (0.66) | 3.17 (0.57) | 3.24 (0.53) | −0.08 (−0.19 to 0.03) | 3.22 (0.59) | 3.27 (0.55) | −0.06 (−0.18 to 0.07) | 3.22 (0.54) | 3.20 (0.56) | 0.03 (−0.08 to 0.13) | |
Not deprived | 3.40 (0.59) | 3.41 (0.57) | 3.29 (0.54) | 3.22 (0.54) | 0.07 (−0.05 to 0.18) | 3.35 (0.51) | 3.20 (0.60) | 0.10 (−0.03 to 0.23) | 3.22 (0.53) | 3.19 (0.58) | 0.03 (−0.08 to 0.14) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 3.32 (0.67) | 3.34 (0.69) | 3.27 (0.70) | 3.26 (0.68) | 0.04 (−0.06 to 0.13) | 2.94 (0.74) | 2.87 (0.75) | 0.08 (−0.03 to 0.19) | 2.80 (0.75) | 2.58 (0.81) | 0.23 (0.13 to 0.33)** | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.078 | |||||||||||
Male | 3.24 (0.69) | 3.27 (0.74) | 3.21 (0.73) | 3.24 (0.67) | 0.00 (−0.13 to 0.13) | 2.85 (0.79) | 2.87 (0.78) | −0.01 (−0.16 to 0.14) | 3.06 (0.80) | 2.93 (0.85) | 0.19 (0.07 to 0.32)* | |
Female | 3.40 (0.64) | 3.42 (0.62) | 3.34 (0.66) | 3.28 (0.69) | 0.07 (−0.07 to 0.21) | 3.04 (0.67) | 2.86 (0.72) | 0.17 (0.01 to 0.32)* | 3.18 (0.74) | 3.09 (0.82) | 0.11 (−0.02 to 0.24) | |
Deprivation level | p = 0.082 | |||||||||||
Deprived | 3.34 (0.69) | 3.34 (0.69) | 3.20 (0.76) | 3.26 (0.69) | −0.05 (−0.19 to 0.08) | 2.90 (0.75) | 2.93 (0.76) | −0.02 (−0.17 to 0.13) | 3.10 (0.77) | 3.02 (0.87) | 0.15 (0.02 to 0.27)* | |
Not deprived | 3.30 (0.65) | 3.35 (0.68) | 3.34 (0.62) | 3.26 (0.67) | 0.13 (−0.01 to 0.26)*** | 2.99 (0.73) | 2.81 (0.74) | 0.17 (0.01 to 0.33)* | 3.15 (0.78) | 3.00 (0.81) | 0.15 (0.02 to 0.28)* |
Secondary outcomes | T0 | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 3.47 (0.58) | 3.51 (0.53) | 3.54 (0.55) | 3.54 (0.54) | 0.04 (−0.04 to 0.12) | 3.53 (0.55) | 3.44 (0.61) | 0.08 (0.00 to 0.17)*** | 3.38 (0.62) | 3.30 (0.66) | 0.12 (0.04 to 0.19)* | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.770 | |||||||||||
Male | 3.44 (0.64) | 3.45 (0.57) | 3.54 (0.54) | 3.52 (0.51) | 0.02 (−0.08 to 0.12) | 3.53 (0.55) | 3.42 (0.62) | 0.09 (−0.03 to 0.20) | 3.46 (0.58) | 3.37 (0.64) | 0.11 (0.02 to 0.21)* | |
Female | 3.50 (0.51) | 3.57 (0.47) | 3.54 (0.57) | 3.56 (0.56) | 0.05 (−0.05 to 0.16) | 3.52 (0.56) | 3.46 (0.60) | 0.09 (−0.03 to 0.20) | 3.45 (0.55) | 3.47 (0.56) | 0.07 (−0.03 to 0.17) | |
Deprivation level | p = 0.706 | |||||||||||
Deprived | 3.42 (0.62) | 3.47 (0.57) | 3.51 (0.58) | 3.54 (0.56) | 0.00 (−0.10 to 0.10) | 3.51 (0.54) | 3.43 (0.63) | 0.09 (−0.03 to 0.20) | 3.44 (0.58) | 3.44 (0.60) | 0.06 (−0.04 to 0.16) | |
Not deprived | 3.52 (0.54) | 3.55 (0.47) | 3.58 (0.52) | 3.54 (0.52) | 0.06 (−0.04 to 0.17) | 3.55 (0.57) | 3.45 (0.59) | 0.08 (−0.04 to 0.19) | 3.48 (0.55) | 3.40 (0.61) | 0.12 (0.02 to 0.21)* |
Secondary outcomes | T0 | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 2.90 (0.94) | 2.91 (0.95) | 2.91 (0.85) | 2.87 (0.82) | 0.05 (−0.07 to 0.16) | 2.64 (0.88) | 2.70 (0.86) | −0.03 (−0.16 to 0.10) | 2.66 (0.85) | 2.50 (0.82) | 0.14 (0.02 to 0.26) | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.379 | |||||||||||
Male | 2.70 (0.97) | 2.67 (0.98) | 2.77 (0.88) | 2.74 (0.86) | −0.02 (−0.17 to 0.14) | 2.51 (0.88) | 2.53 (0.90) | −0.05 (−0.22 to 0.13) | 2.74 (0.91) | 2.54 (0.90) | 0.14 (−0.01 to 0.29)*** | |
Female | 3.11 (0.86) | 3.18 (0.85) | 3.07 (0.79) | 3.00 (0.76) | 0.12 (−0.04 to 0.28) | 2.79 (0.85) | 2.87 (0.77) | 0.00 (−0.18 to 0.19) | 3.00 (0.85) | 2.97 (0.90) | 0.09 (−0.07 to 0.24) | |
Deprivation level | p = 0.005 | |||||||||||
Deprived | 2.93 (0.99) | 2.86 (0.99) | 2.79 (0.86) | 2.84 (0.83) | −0.13 (−0.28 to 0.03) | 2.54 (0.85) | 2.64 (0.92) | −0.16 (−0.34 to 0.01)*** | 2.76 (0.92) | 2.75 (0.96) | −0.05 (−0.20 to 0.10) | |
Not deprived | 2.87 (0.89) | 2.97 (0.92) | 3.05 (0.83) | 2.90 (0.81) | 0.24 (0.08 to 0.40)* | 2.78 (0.91) | 2.75 (0.80) | 0.14 (−0.05 to 0.32) | 2.98 (0.84) | 2.77 (0.89) | 0.28 (0.13 to 0.43)** |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 3.49 (0.55) | 3.52 (0.56) | 3.47 (0.56) | 3.50 (0.52) | −0.01 (−0.09 to 0.07) | 3.25 (0.65) | 3.21 (0.65) | 0.07 (−0.02 to 0.16) | 3.15 (0.65) | 2.98 (0.72) | 0.19 (0.10 to 0.27)** | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.178 | |||||||||||
Male | 3.37 (0.60) | 3.39 (0.61) | 3.36 (0.60) | 3.44 (0.53) | −0.07 (−0.18 to 0.04) | 3.18 (0.65) | 3.15 (0.70) | 0.04 (−0.08 to 0.17) | 3.29 (0.62) | 3.19 (0.73) | 0.14 (0.04 to 0.24)* | |
Female | 3.61 (0.46) | 3.66 (0.46) | 3.59 (0.48) | 3.57 (0.49) | 0.06 (−0.06 to 0.17) | 3.30 (0.63) | 3.27 (0.59) | 0.08 (−0.04 to 0.21) | 3.45 (0.58) | 3.39 (0.66) | 0.11 (0.00 to 0.21)* | |
Deprivation level | p = 0.085 | |||||||||||
Deprived | 3.51 (0.58) | 3.50 (0.59) | 3.40 (0.60) | 3.48 (0.56) | −0.09 (−0.20 to 0.02) | 3.19 (0.68) | 3.21 (0.69) | −0.03 (−0.15 to 0.09) | 3.34 (0.61) | 3.30 (0.73) | 0.07 (−0.04 to 0.17) | |
Not deprived | 3.47 (0.52) | 3.53 (0.54) | 3.55 (0.50) | 3.52 (0.47) | 0.08 (−0.03 to 0.19) | 3.33 (0.59) | 3.21 (0.61) | 0.17 (0.04 to 0.30)* | 3.41 (0.60) | 3.28 (0.67) | 0.18 (0.08 to 0.28)* |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 1.75 (0.83) | 1.77 (0.81) | 1.55 (0.72) | 1.53 (0.68) | 0.04 (−0.05 to 0.13) | 1.46 (0.63) | 1.43 (0.60) | 0.08 (−0.02 to 0.18) | 1.43 (0.59) | 1.46 (0.64) | 0.01 (−0.08 to 0.12) | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.173 | |||||||||||
Male | 1.83 (0.87) | 1.79 (0.81) | 1.54 (0.70) | 1.54 (0.68) | −0.04 (−0.16 to 0.09) | 1.45 (0.64) | 1.46 (0.64) | −0.01 (−0.16 to 0.13) | 1.65 (0.76) | 1.66 (0.77) | −0.05 (−0.17 to 0.07) | |
Female | 1.67 (0.78) | 1.75 (0.82) | 1.57 (0.74) | 1.52 (0.69) | 0.13 (0.00 to 0.26)* | 1.47 (0.62) | 1.41 (0.56) | 0.17 (0.02 to 0.32)* | 1.54 (0.68) | 1.56 (0.68) | 0.11 (−0.02 to 0.23)*** | |
Deprivation level | p = 0.513 | |||||||||||
Deprived | 1.83 (0.87) | 1.79 (0.83) | 1.59 (0.74) | 1.56 (0.73) | −0.01 (−0.14 to 0.11) | 1.51 (0.67) | 1.44 (0.64) | 0.03 (−0.11 to 0.17) | 1.62 (0.75) | 1.61 (0.76) | −0.03 (−0.15 to 0.09) | |
Not deprived | 1.67 (0.78) | 1.76 (0.79) | 1.51 (0.68) | 1.50 (0.63) | 0.11 (−0.02 to 0.24) | 1.40 (0.56) | 1.43 (0.57) | 0.13 (−0.02 to 0.28)*** | 1.57 (0.70) | 1.61 (0.69) | 0.09 (−0.04 to 0.21) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 1.36 (0.48) | 1.38 (0.49) | 1.28 (0.39) | 1.28 (0.40) | 0.03 (−0.03 to 0.08) | 1.26 (0.37) | 1.24 (0.35) | 0.04 (−0.03 to 0.10) | 1.22 (0.35) | 1.34 (0.50) | −0.08 (−0.14 to −0.02)* | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.128 | |||||||||||
Male | 1.46 (0.55) | 1.47 (0.52) | 1.35 (0.43) | 1.38 (0.48) | −0.00 (−0.08 to 0.07) | 1.30 (0.42) | 1.30 (0.40) | 0.02 (−0.07 to 0.11) | 1.35 (0.45) | 1.44 (0.55) | −0.10 (−0.18 to 0.03)* | |
Female | 1.24 (0.37) | 1.29 (0.44) | 1.20 (0.33) | 1.19 (0.28) | 0.06 (−0.02 to 0.14) | 1.20 (0.30) | 1.18 (0.28) | 0.07 (−0.02 to 0.16) | 1.18 (0.31) | 1.20 (0.36) | 0.02 (−0.05 to 0.10) | |
Deprivation level | p = 0.175 | |||||||||||
Deprived | 1.39 (0.50) | 1.44 (0.53) | 1.32 (0.43) | 1.31 (0.42) | 0.07 (−0.00 to 0.15)*** | 1.30 (0.41) | 1.26 (0.38) | 0.10 (0.02 to 0.19)* | 1.29 (0.41) | 1.34 (0.52) | −0.02 (−0.10 to 0.05) | |
Not deprived | 1.32 (0.45) | 1.33 (0.45) | 1.23 (0.34) | 1.26 (0.38) | −0.02 (−0.10 to 0.06) | 1.19 (0.29) | 1.23 (0.32) | −0.02 (−0.11 to 0.07) | 1.24 (0.37) | 1.30 (0.44) | −0.06 (−0.13 to 0.02) |
Secondary outcomes | T0, mean (SD) | T1 | T2 | T3 | Interaction significancea | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Intervention | Control | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | Intervention, mean (SD) | Control, mean (SD) | Adjusted effect estimate R2 (95% Cl) | ||
All available data (intention to treat) | 2.02 (0.83) | 1.95 (0.81) | 1.77 (0.76) | 1.71 (0.68) | 0.02 (−0.08 to 0.12) | 1.86 (0.70) | 1.76 (0.70) | 0.02 (−0.09 to 0.13) | 1.86 (0.72) | 1.91 (0.76) | −0.11 (−0.22 to −0.00)* | |
Additional subgroup analysis | ||||||||||||
Gender | p = 0.437 | |||||||||||
Male | 2.12 (0.88) | 2.04 (0.82) | 1.90 (0.80) | 1.78 (0.72) | 0.05 (−0.08 to 0.19) | 1.98 (0.76) | 1.85 (0.77) | 0.04 (−0.12 to 0.19) | 1.97 (0.78) | 1.97 (0.79) | −0.10 (−0.23 to 0.03) | |
Female | 1.90 (0.76) | 1.85 (0.79) | 1.63 (0.68) | 1.62 (0.62) | −0.02 (−0.16 to 0.13) | 1.71 (0.59) | 1.68 (0.61) | −0.00 (−0.17 to 0.16) | 1.76 (0.67) | 1.72 (0.69) | −0.02 (−0.16 to 0.11) | |
Deprivation level | p = 0.209 | |||||||||||
Deprived | 2.11 (0.86) | 2.08 (0.85) | 1.82 (0.75) | 1.76 (0.72) | 0.07 (−0.07 to 0.21) | 1.88 (0.71) | 1.74 (0.65) | 0.11 (−0.04 to 0.27) | 1.95 (0.75) | 1.87 (0.79) | 0.04 (−0.09 to 0.17) | |
Not deprived | 1.91 (0.79) | 1.82 (0.75) | 1.72 (0.76) | 1.65 (0.62) | −0.02 (−0.16 to 0.12) | 1.82 (0.69) | 1.79 (0.75) | −0.06 (0.23 to 0.10) | 1.77 (0.71) | 1.82 (0.72) | −0.15 (−0.29 to −0.02)* |
Health-related behaviour | T2 | T3 | ||||
---|---|---|---|---|---|---|
Intervention arm (N = 341), n (%) | Control arm (N = 381), n (%) | p-valuea | Intervention arm (N = 443), n (%) | Control arm (N = 433), n (%) | p-valuea | |
Alcohol | ||||||
Missing, not included | 12 | 12 | 15 | 17 | ||
Never tried | 255 (74.8) | 266 (69.8) | 248 (57.9) | 216 (49.9) | ||
Tried | 81 (23.8) | 104 (27.3) | 0.224 | 149 (34.8) | 183 (42.3) | 0.095 |
Use occasionally | 5 (1.5) | 8 (2.1) | 26 (6.1) | 26 (6.0) | ||
Use regularly | 0 (0) | 3 (0.8) | 5 (1.2) | 8 (1.8) | ||
Tobacco | ||||||
Missing, not included | 14 | 20 | 17 | 20 | ||
Never tried | 325 (95.9) | 352 (94.4) | 389 (91.3) | 371 (86.3) | ||
Tried | 13 (308) | 12 (3.2) | 0.109 | 24 | 44 (10.2) | 0.083 |
Use occasionally | 1 (0.3) | 4 (1.1) | 5 (1.2) | 6 (1.4) | ||
Use regularly | 0 (0) | 5 (1.3) | 8 (1.9) | 9 (2.1) | ||
E-cigarettes | ||||||
Missing, not included | 13 | 18 | 14 | 22 | ||
Never tried | 318 (93.5) | 331 (88.3) | 372 (86.7) | 334 (78.0) | ||
Tried | 21 (6.2) | 37 (9.9) | 0.049 | 44 (10.3) | 72 (16.8) | 0.010 |
Use occasionally | 0 (0) | 2 (0.5) | 7 (1.6) | 11 (2.6) | ||
Use regularly | 1 (1.3) | 5 (1.3) | 6 (1.4) | 11 (2.6) | ||
Cannabis | ||||||
Missing, not included | 14 | 22 | 17 | 25 | ||
Never tried | 337 (99.4) | 361 (97.3) | 407 (95.5) | 402 (94.6) | ||
Tried | 1 (0.3) | 4 (1.1) | 0.190 | 11 (2.6) | 9 (2.1) | 0.580 |
Use occasionally | 0 (0) | 3 (0.8) | 2 (0.5) | 5 (1.2) | ||
Use regularly | 1 (0.3) | 3 (0.8) | 6 (1.4) | 9 (2.1) |
Health-related behaviour | Intervention (N = 263), n (%) | Control (N = 301), n (%) | p-valuea |
---|---|---|---|
Alcohol | |||
Missing, not included | 20 | 13 | |
Never tried | 57 (21.7) | 34 (11.3) | |
Tried | 105 (39.9) | 130 (43.2) | 0.01 |
Use occasionally | 70 (26.6) | 97 (32.2) | |
Use regularly | 31 (11.8) | 40 (13.3) | |
In previous 12 months, how often have you got drunk? | |||
Missing, not included | 24 | 14 | |
Never | 130 (50.2) | 124 (41.3) | |
Once or twice | 48 (18.5) | 49 (16.3) | 0.09 |
Three or four times | 25 (9.7) | 40 (13.3) | |
Once a month | 28 (10.8) | 50 (16.7) | |
Once a week | 28 (10.8) | 37 (12.3) | |
Tobacco | |||
Missing, not included | 14 | 20 | |
Never tried | 325 (95.9) | 352 (94.4) | |
Tried | 13 (308) | 12 (3.2) | 0.81 |
Use occasionally | 1 (0.3) | 4 (1.1) | |
Use regularly | 0 (0) | 5 (1.3) | |
E-cigarettes | |||
Missing, not included | 13 | 18 | |
Never tried | 318 (93.5) | 331 (88.3) | |
Tried | 21 (6.2) | 37 (9.9) | 0.64 |
Use occasionally | 0 (0) | 2 (0.5) | |
Use regularly | 1 (1.3) | 5 (1.3) | |
Cannabis | |||
Missing, not included | 14 | 22 | |
Never tried | 337 (99.4) | 361 (97.3) | |
Tried | 1 (0.3) | 4 (1.1) | 0.80 |
Use occasionally | 0 (0) | 3 (0.8) | |
Use regularly | 1 (0.3) | 3 (0.8) |
Staff outcomes | T0, mean (SD) | T1 | T2 | T3 | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Intervention (n = 313) | Control (n = 343) | Intervention, mean (SD) (n = 283) | Control, mean (SD) (n = 297) | Adjusted effect estimate (R2 95% CI) | Intervention, mean (SD) (n = 274) | Control, mean (SD) (n = 232) | Adjusted effect estimate (R2 95% CI) | Intervention, mean (SD) (n = 259) | Control, mean (SD) (n = 261) | Adjusted effect estimate (R2 95% CI) | |
Pupil behaviour | 3.39 (0.60) | 3.45 (0.51) | 3.45 (0.48) | 3.44 (0.48) | 0.01 (−0.09 to 0.11) | 3.48 (0.46) | 3.39 (0.50) | 0.07 (−0.04 to 0.18) | 3.43 (0.52) | 3.32 (0.53) | 0.09 (−0.02 to 0.21) |
Pupil confidence | 3.28 (0.57) | 3.26 (0.54) | 3.28 (0.46) | 3.27 (0.48) | 0.02 (−0.08 to 0.11) | 3.31 (0.49) | 3.27 (0.47) | 0.03 (−0.08 to 0.13) | 3.26 (0.43) | 3.15 (0.50) | 0.09 (−0.01 to 0.19) |
Pupil engagement | 3.37 (0.50) | 3.45 (0.41) | 3.40 (0.43) | 3.42 (0.40) | 0.06 (−0.02 to 0.14) | 3.43 (0.45) | 3.37 (0.44) | 0.13 (0.04 to 0.22) | 3.39 (0.40) | 3.31 (0.46) | 0.14 (0.05 to 0.23) |
Pupil–pupil relationships | 3.07 (0.63) | 3.11 (0.61) | 3.21 (0.50) | 3.20 (0.49) | 0.07 (−0.04 to 0.17) | 3.30 (0.49) | 3.24 (0.50) | 0.11 (−0.00 to 0.22) | 3.23 (0.47) | 3.18 (0.51) | 0.10 (−0.01 to 0.21) |
Staff–pupil relationships | 3.32 (0.54) | 3.37 (0.52) | 3.39 (0.46) | 3.38 (0.46) | 0.06 (−0.03 to 0.15) | 3.44 (0.45) | 3.36 (0.44) | 0.11 (0.01 to 0.21) | 3.41 (0.47) | 3.30 (0.48) | 0.14 (0.04 to 0.24) |
Staff–staff relationships | 3.34 (0.48) | 3.40 (0.47) | 3.34 (0.47) | 3.38 (0.45) | −0.00 (−0.08 to 0.09) | 3.38 (0.48) | 3.36 (0.49) | 0.07 (−0.02 to 0.17) | 3.38 (0.50) | 3.30 (0.51) | 0.13 (0.03 to 0.22) |
Perceptions of management | 2.97 (0.50) | 3.04 (0.49) | 3.04 (0.49) | 3.04 (0.49) | 0.07 (−0.20 to 0.15) | 3.04 (0.53) | 2.98 (0.56) | 0.13 (0.04 to 0.22) | 3.02 (0.53) | 2.97 (0.51) | 0.10 (0.01 to 0.19) |
Staff support | 3.14 (0.46) | 3.23 (0.41) | 3.14 (0.45) | 3.17 (0.43) | 0.07 (−0.01 to 0.15) | 3.19 (0.48) | 3.15 (0.49) | 0.14 (0.06 to 0.22) | 3.13 (0.49) | 3.10 (0.46) | 0.12 (0.04 to 0.20) |
School ethos | 3.28 (0.47) | 3.40 (0.43) | 3.26 (0.50) | 3.32 (0.44) | 0.06 (−0.02 to 0.14) | 3.30 (0.47) | 3.26 (0.49) | 0.16 (0.07 to 0.25) | 3.28 (0.50) | 3.22 (0.50) | 0.16 (0.08 to 0.25) |
School support for SEW | 3.47 (0.41) | 3.51 (0.39) | 3.47 (0.44) | 3.54 (0.41) | −0.01 (−0.09 to 0.07) | 3.52 (0.40) | 3.47 (0.42) | 0.10 (0.01 to 0.18) | 3.45 (0.46) | 3.46 (0.41) | 0.03 (−0.05 to 0.12) |
Valued member of staff | 3.29 (0.58) | 3.38 (0.57) | 3.31 (0.60) | 3.40 (0.56) | 0.01 (−0.10 to 0.11) | 3.36 (0.58) | 3.39 (0.60) | 0.07 (−0.04 to 0.19) | 3.35 (0.57) | 3.32 (0.62) | 0.11 (–0.01 to 0.23) |
Training opportunities | 3.07 (0.54) | 3.14 (0.57) | 3.14 (0.60) | 3.27 (0.59) | −0.02 (−0.13 to 0.08) | 3.19 (0.60) | 3.22 (0.55) | 0.06 (−0.05 to 0.18) | 3.13 (0.59) | 3.13 (0.60) | 0.76 (−0.04 to 0.19) |
Self-efficacy | 3.40 (0.40) | 3.43 (0.38) | 3.49 (0.41) | 3.54 (0.39) | −0.04 (−0.12 to 0.04) | 3.55 (0.39) | 3.50 (0.43) | 0.07 (−0.02 to 0.15) | 3.45 (0.42) | 3.51 (0.39) | −0.04 (−0.12 to 0.05) |
Appendix 4 Economic evaluation
School | From APs (i.e. resources purchased as a result of involvement in SEED) | From all-staff questionnaires: not necessarily new/resulting from involvement in the SEED RCT. Reports of resource guide resources used at F3 (2017) | |||
---|---|---|---|---|---|
Resource guide | Resource guide cost (£) | Other costed | No cost/existing resources | ||
IA1P | None | SULP; A Volcano in my Tummy | MySELF/Early Years FAIR – Council Resources | Bounce Back! | |
IA2P | None | Lessons for Living training | Financial Education pack | PATHS | |
IA3P | None | Middle management course at Glasgow University | Treat Me Well/FAIR – council resources | ||
IB1P | None | ABLe (educational psychology service) | Bounce Back! | ||
IB2P | None | ABLe (educational psychology service) | |||
IA4P | None | ||||
IB3P | None | Tree of Knowledge | CPD re PALS club by Support for Learning team | ||
IC1P | Creating Confident Kids | 213.00 | Circle Time (existing resource); Big Deal Little Deal | Creating Confident Kids | |
IC2P | Creating Confident Kids | 100.00 | Big Deal Little Deal; Rights Respecting School Award | Creating Confident Kids; Good Behaviour Game | |
Seasons for Growth | 280.00 | ||||
IA4P | BounceBack! | 149.97 | Circle Time; nurture groups run by senior family support worker; Seasons for Growth(staff already trained) | BounceBack!; Seasons for Growth | |
IA5P | None | ||||
IC4P | Creating Confident Kids | 100.00 | Thinking Hats (unclear if purchased) | Creating Confident Kids; Seasons for Growth | |
Restorative approaches business cards | 50.00 | ||||
IC5P | Creating Confident Kids | 100.00 | Circle Time/Bubble Time; Rights Respecting School Award; Big Deal Little Deal | Creating Confident Kids | |
Circle Time refresher training | No cost | ||||
Seasons for Growth | 405.00 | ||||
Pupil councils | No cost | ||||
IA6P | None | Rights Respecting School Award; antibullying workshops; sleep workshops | |||
IA7P | Nurture | No cost | MySELF Emotional Literacy Framework; Staying Stronger (resilience); staff CPD for nurture approaches – council resources/training | ||
IC6P | None | Beanie Badges reward system; improve outdoor learning area | ‘Thought boxes’ | ||
IA8P | BounceBack! ‘considered’ | 149.97 | |||
IB4P | None | Emotional literacy resources ‘considered’ |
Scenario | Element | Position in base-case analysis | Variation for the sensitivity analysisa |
---|---|---|---|
1 | Intervention cost | Average intervention cost across school |
|
2 | Missing data | The missing whole-cost questionnaire was assumed to be missing at random and not incorporated in the estimation. The evaluation is conducted with complete-case analysis |
|
3 | Discount rate | 1.5% for both costs and benefits | 3.5% for both costs and benefits |
4 | Perspective | Public sector perspective (i.e. cost includes NHS, social care and public services, and intervention costs) | Educational sector perspective (i.e. cost includes only the intervention cost) |
Resource use item (unit) | Unit cost (£) | Source |
---|---|---|
GP (per contact) | 38.00 | PSSRU 2017,99 p. 162. Per patient contact lasting 9.22 minutes, with qualifications |
Practice nurse (per contact) | 10.85 | PSSRU 2017,99 p. 160. Nursing average cost per hour, with qualifications: £42 |
Duration of contact per patient is 15.5 minutes (PSSRU 2015,144 p. 174, based on the 2006/07 UK general practice survey32,145) | ||
A&E/minor injuries unit visit that led to a hospital admission (per A&E visit) | 148 | NHS reference costs 2016/17100 highlights, analysis and introduction to the data. Table 2. Unit costs by point of delivery, 2014/15 and 2016/17 (£). Unit cost for A&E attendance (p. 5) |
A&E/minor injuries unit visit that did not lead to a hospital admission (per A&E visit) | 148 | NHS reference costs 2016/17100 highlights, analysis and introduction to the data. Table 2. Unit costs by point of delivery, 2014/15 and 2016/17 (£). Unit cost for A&E attendance (p. 5) |
Social worker (per hour) | 59 | PSSRU 2017,99 p. 174. Unit costs including qualifications. Duration of visit is not available. Assumption: 1 hour per visit |
Speech therapist (per visit) | 96 | NHS reference costs 2016/17100 community health service. A13A1 speech and language therapist, adult, one to one |
Occupational therapist (per visit) | 77 | NHS reference costs 2016/17100 community health service. A06A1 occupational therapist, adult, one to one |
EP (per visit) | 42.66 | PSSRU 2014.146 CAF as a step-up to social care (p. 156): £41. Used HCHS inflation factor to inflate to 2016/17 pricea |
School nurse (per visit) | 55 | NHS reference costs 2016/17,100 community health service. N05CGM school-based children’s health core services, group multiprofessional |
Physiotherapist (per visit) | 53 | NHS reference costs 2016/17,100 allied health professionals A08A1, physiotherapist, adult, one to one |
Psychiatrist (per attendance) | 217.36 | NHS reference costs 2016/17.100 Total outpatient attendances. Child and adolescent psychiatry |
CAMHS (per contact) | 221 | NHS reference costs 2016/17.100 Mental health. CAMHS community contact, average cost per community contacts |
Hospital stay (per day) | 398 | NHS reference costs 2016/17,100 regular day or night admissions. Weighted average over all paediatric services (from record PC63A until PX57C) |
Outpatient visit (per visit) | 189 | NHS reference costs 2016/17.100 WF01A non-admitted face-to-face attendance, follow-up 420 paediatrics |
Dentist (per visit) | 78.25 | PSSRU 2017.99 NHS dentist (pp. 165–6). Performer-only £127 per hour of patient contact. Dentist: providing performer £186 per hour of patient contact. Average: £156.5 per hour. Assumption: 30 minutes for each appointment |
Optician (per contact) | 40.36 | Goodman C et al.:70 £91 per hour of client contact, cost from PSSRU 2015,144 and assumed 25.8 minutes for each contact, resulting in £39.13 per contact.147 Inflated using HCHS inflation factorb |
Police attendance (per attendance) | 41.26 | New economy Manchester: unit cost database. ‘Crime’ tab – labour costs per hour.148,149 Police officer, sergeant and below, cost per hour £40, March 2015 price. Inflated using HCHS inflation factor to 2016/17 priceb |
Mileage (per mile) | 0.56 | Mileage reimbursement rates for work-related car travel: 56p per mile for the first 3500 miles, then 20p for each additional mile148,149 |
Resource items collected from the questionnaires and included in the total cost | Intervention arm (N = 680) | Control arm (N = 689) | ||||||
---|---|---|---|---|---|---|---|---|
T0–T1 | T1–T3 | T0–T1 | T1–T3 | |||||
Missing (n) | % | Missing (n) | % | Missing (n) | % | Missing (n) | % | |
GP | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Practice nurse | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
A&E/minor injuries unit visit that led to a hospital admission | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
A&E/minor injuries unit visit that did not lead to a hospital admission | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Social worker | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Speech therapist | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Occupational therapist | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
EP | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
School nurse | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Physiotherapist | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Psychiatrist | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Child and Adolescent Mental Health Services | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Dentist | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Optician | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Police attendance | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Hospital stay | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Outpatient visit | 532 | 78.24 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Total cost | 534 | 78.53 | 582 | 85.59 | 505 | 73.29 | 577 | 83.74 |
Time point | Intervention arm | Control arm | ||
---|---|---|---|---|
Missing (n) | % | Missing (n) | % | |
T0 | 132 | 19.41 | 122 | 17.71 |
T1 | 126 | 18.53 | 92 | 13.35 |
T2 | 307 | 45.15 | 278 | 40.35 |
T3 | 226 | 33.24 | 220 | 31.93 |
School | Year 1 | Year 3 | Year 4 | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Feedback reports (90 pages) | ‘All schools’ reports (54 pages) | Resource guides (85 pages) | Year 1: total cost (£) | Feedback reports (44 pages) | ‘All schools’ reports (32 pages) | Resource guides (85 pages) | Year 3: total cost (£) | Feedback reports (30 pages) | ‘All schools’ reports (33 pages) | Resource guides (85 pages) | Year 4: total cost (£) | ||||||||||
n | Costa (£) |
n | Costa (£) |
n | Costa (£) |
n | Costa (£) |
n | Costa (£) |
n | Costa (£) |
n | Costa (£) |
n | Costa (£) |
n | Costa (£) |
||||
IA1P | 15 | 40.50b | 3 | 4.86c | 2 | 5.10d | 50.46e | 16 | 21.12 | 3 | 2.88 | 2 | 5.10 | 29.10 | |||||||
IA2P | 21 | 56.70 | 4 | 6.48 | 2 | 5.10 | 68.28 | 15 | 19.8 | 3 | 2.88 | 2 | 5.10 | 27.78 | |||||||
IA3P | 11 | 29.70 | 2 | 3.24 | 2 | 5.10 | 38.04 | 8 | 10.56 | 2 | 1.92 | 2 | 5.10 | 17.58 | |||||||
IB1P | 22 | 59.40 | 4 | 6.48 | 2 | 5.10 | 70.98 | 15 | 19.80 | 3 | 2.88 | 2 | 5.10 | 27.78 | |||||||
IB2P | 20 | 54.00 | 5 | 8.10 | 2 | 5.10 | 67.20 | 20 | 26.40 | 4 | 3.84 | 2 | 5.10 | 35.34 | |||||||
IA4P | 6 | 16.20 | 2 | 3.24 | 2 | 5.10 | 24.54 | 6 | 7.92 | 2 | 1.92 | 2 | 5.10 | 14.94 | |||||||
IB3P | 19 | 51.30 | 4 | 6.48 | 2 | 5.10 | 62.88 | ||||||||||||||
IC1P | 20 | 54.00 | 6 | 9.72 | 2 | 5.10 | 68.82 | 21 | 27.72 | 4 | 3.84 | 2 | 5.10 | 36.66 | |||||||
IC2P | 20 | 54.00 | 4 | 6.48 | 2 | 5.10 | 65.58 | 22 | 29.04 | 4 | 3.84 | 2 | 5.10 | 37.98 | 16 | 21.60 | 5 | 4.95 | 2 | 5.10 | 31.65 |
IA4P | 20 | 54.00 | 6 | 9.72 | 2 | 5.10 | 68.82 | 22 | 29.04 | 4 | 3.84 | 2 | 5.10 | 37.98 | 16 | 19.80 | 4 | 3.96 | 2 | 5.10 | 28.86 |
IA5P | 7 | 18.90 | 2 | 3.24 | 2 | 5.10 | 27.24 | ||||||||||||||
IC4P | 17 | 45.90 | 4 | 6.48 | 2 | 5.10 | 57.48 | 14 | 18.48 | 3 | 2.88 | 2 | 5.10 | 26.46 | |||||||
IC5P | 15 | 40.50 | 3 | 4.86 | 2 | 5.10 | 50.46 | 9 | 11.88 | 2 | 1.92 | 2 | 5.10 | 18.90 | |||||||
IA6P | 18 | 48.60 | 4 | 6.48 | 2 | 5.10 | 60.18 | ||||||||||||||
IA7P | 20 | 54.00 | 5 | 8.10 | 2 | 5.10 | 67.20 | 2 | 2.64 | 1 | 0.96 | 2 | 5.10 | 8.70 | |||||||
IC6P | 20 | 54.00 | 5 | 8.10 | 2 | 5.10 | 67.20 | ||||||||||||||
IA8P | 20 | 54.00 | 6 | 9.72 | 3 | 7.65 | 71.37 | 17 | 22.44 | 3 | 2.88 | 2 | 5.10 | 30.42 | |||||||
IB4P | 18 | 48.60 | 4 | 6.48 | 2 | 5.10 | 60.18 | 16 | 21.12 | 3 | 2.88 | 2 | 5.10 | 29.10 | |||||||
Total cost | 1046.91 | 378.72 | 61.51 | ||||||||||||||||||
Cost per personf | 0.77f,g | 0.28 | 0.05 | ||||||||||||||||||
Discounted cost per personf,g,h | 0.77 i | 0.27 | 0.04 |
First feedback session cost at year 1
School | Mileage to school (round trip) | Number of travels | Total mileage cost (£)a | Staff travel time cost | Time cost for the first presentation | Time cost for the RD sessions | Total first-year feedback session staff time cost (£)b | Total first-year feedback session cost (staff and travel) (£)c | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
For all staff or teaching staff only (hours) | Additional session for support staff only | Total time cost for the first presentation (£) | For all staff or teaching staff session only (hours) | Additional session for support staff only | Total time cost for the RD sessions (£) | |||||||||||||||||
Speed assumed (mph) | Travel time for each round tripd | Total travel time cost (£)e | Number of hours | Number of SEED researchers | Total unit cost of staff per hour (£)f | Support staff only (when applicable) | Number of SEED researchers | Unit cost per hour (£) | Number of hours | Number of SEED researchers | Total unit cost (£) | Number of hours | Number of SEED researchers | Total unit cost (£) | ||||||||
IA1P | 65.2 | 2 | 73.02g | 40 | 1.63h | 305.80i | 1.5 | 2 | 122.33 | 183.50j | 1.5 | 1 | 65.28 | 97.92 | 281.42b | 660.24c | ||||||
IA2P | 53.4 | 3 | 89.71 | 30 | 1.78 | 348.60 | 1.5 | 1 | 65.28 | 1.5 | 1 | 65.28 | 195.84 | 2 | 1 | 65.28 | 130.56 | 326.40 | 764.71 | |||
IA3P | 93.0 | 1 | 52.08 | 40 | 2.33 | 325.97 | 3 | 2 | 140.20 | 420.60 | 0.00 | 420.60 | 798.65 | |||||||||
IB1P | 161.2 | 2 | 180.54 | 50 | 3.22 | 578.32 | 1 | 1 | 57.05 | 1.5 | 1 | 57.05 | 142.63 | 1.5 | 2 | 122.33 | 577.89 | 720.51 | 1479.38 | |||
IB2P | 163.4 | 2 | 183.01 | 50 | 3.27 | 586.21 | 1.5 | 1 | 57.05 | 85.58 | 2 | 2 | 122.33 | 244.66 | 330.24 | 1099.46 | ||||||
IA4P | 73.2 | 2 | 81.98 | 40 | 1.83 | 343.33 | 1.5 | 1 | 65.28 | 97.92 | 1.5 | 2 | 122.33 | 183.50 | 281.42 | 706.73 | ||||||
IB3P | 165.4 | 2 | 185.25 | 50 | 3.31 | 404.67 | 1.5 | 1 | 57.05 | 85.58 | 2 | 1 | 65.28 | 130.56 | 216.14 | 806.05 | ||||||
IC1P | 59.2 | 1 | 33.15 | 30 | 1.97 | 112.58 | 3 | 1 | 57.05 | 171.15 | 0.00 | 171.15 | 316.88 | |||||||||
IC2P | 38.2 | 2 | 42.78 | 30 | 1.27 | 238.89 | 2 | 2 | 122.33 | 244.66 | 1 | 1 | 65.28 | 1 | 1 | 65.28 | 130.56 | 375.22 | 656.89 | |||
IA4P | 48.4 | 1 | 27.10 | 30 | 1.61 | 226.19 | 3 | 2 | 140.20 | 420.60 | 0.00 | 420.60 | 673.89 | |||||||||
IA5P | 83.4 | 3 | 140.11 | 40 | 2.09 | 629.07 | 1.5 | 2 | 122.33 | 183.50 | 1.5 | 1 | 57.05 | 1 | 2 | 122.33 | 207.91 | 391.40 | 1160.58 | |||
IC4P | 78.2 | 1 | 43.79 | 40 | 1.96 | 239.16 | 3 | 2 | 122.33 | 366.99 | 0.00 | 366.99 | 649.94 | |||||||||
IC5P | 79.2 | 2 | 88.70 | 40 | 1.98 | 484.43 | 2 | 2 | 122.33 | 244.66 | 1.75 | 2 | 122.33 | 214.08 | 458.74 | 1031.87 | ||||||
IA6P | 24.4 | 2 | 27.33 | 30 | 0.81 | 106.19 | 1.5 | 1 | 65.28 | 97.92 | 2 | 1 | 65.28 | 130.56 | 228.48 | 362.00 | ||||||
IA7P | 27.0 | 2 | 30.24 | 30 | 0.90 | 236.28 | 1.5 | 2 | 140.20 | 210.30 | 1.5 | 2 | 122.33 | 183.50 | 393.80 | 660.31 | ||||||
IC6P | 92.2 | 1 | 51.63 | 40 | 2.31 | 281.97 | 3.5 | 2 | 122.33 | 428.16 | 0.00 | 428.15 | 761.76 | |||||||||
IA8P | 54.2 | 2 | 60.70 | 30 | 1.81 | 341.50 | 1 | 2 | 131.97 | 131.97 | 1.5 | 1 | 57.05 | 85.58 | 217.55 | 619.75 | ||||||
IB4P | 194.4 | 3 | 326.59 | 50 | 3.89 | 919.24 | 1.5 | 2 | 122.33 | 183.50 | 1 | 1 | 57.05 | 1 | 1 | 57.05 | 114.10 | 297.60 | 1543.43 | |||
Total | 1718 | 6708 | 6326 | 14752 | ||||||||||||||||||
Total per personk | 1.27 | 4.97 | 4.68 | 10.92 |
Second feedback session cost at year 3
School | Mileage to school (round trip) | Number of travels | Total mileage cost (£)a | Staff travel time cost | Time cost for the first presentation | Time cost for the RD sessions | Total first-year feedback session staff time cost (£) | Total first-year feedback session cost (staff and travel) (£) | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
For all staff or teaching staff only (hours) | Additional session for support staff only | Total time cost for the first presentation (£) | For all staff or teaching staff session only (hours) | Additional session for support staff only | Total time cost for the RD sessions (£) | |||||||||||||||||
Speed assumed (mph) | Travel time for each round tripb | Total travel time cost (£)c | Number of hours | Number of SEED researchers | Total unit cost of staff per hour (£)d | Support staff only (when applicable) | Number of SEED researchers | Unit cost per hour (£) | Number of hours | Number of SEED researchers | Total unit cost (£) | Number of hours | Number of SEED researchers | Total unit cost (£) | ||||||||
IA1P | 65.2 | 1 | 36.51 | 40 | 1.63 | 106.41 | 2 | 1 | 65.28 | 130.56 | 130.56 | 273.48 | ||||||||||
IA2P | 53.4 | 2 | 73.02 | 30 | 1.78 | 232.40 | 1.25 | 1 | 65.28 | 81.60 | 2.25 | 1 | 65.28 | 228.48 | 533.90 | |||||||
IA3P | 93 | 1 | 36.51 | 40 | 2.33 | 284.42 | 2.5 | 2 | 122.33 | 305.83 | 305.83 | 626.75 | ||||||||||
IB1P | 161.2 | 1 | 36.51 | 50 | 3.22 | 210.46 | 1.5 | 1 | 65.28 | 97.92 | 97.92 | 344.89 | ||||||||||
IB2P | 163.4 | 1 | 36.51 | 50 | 3.27 | 213.34 | 3 | 1 | 65.28 | 195.84 | 195.84 | 445.69 | ||||||||||
IA4P | 73.2 | 1 | 36.51 | 40 | 1.83 | 223.86 | 2 | 2 | 122.33 | 244.66 | 244.66 | 505.04 | ||||||||||
IB3P | 165.4 | 0 | 0.00 | 0 | 0 | 0.00 | 0 | NA | ||||||||||||||
IC1P | 59.2 | 1 | 36.51 | 30 | 1.97 | 241.40 | 1 | 2 | 122.33 | 122.33 | 122.33 | 400.24 | ||||||||||
IC2P | 38.2 | 1 | 36.51 | 30 | 1.27 | 83.12 | 3 | 1 | 65.28 | 195.84 | 195.84 | 315.48 | ||||||||||
IA4P | 48.4 | 2 | 73.02 | 30 | 1.61 | 592.08 | 1.25 | 2 | 122.33 | 1 | 2 | 122.33 | 275.24 | 1.25 | 2 | 122.33 | 428.16 | 1093.26 | ||||
IA5P | 83.4 | 0 | 0.00 | 0 | 0 | 0.00 | 0 | NA | ||||||||||||||
IC4P | 78.2 | 2 | 73.02 | 40 | 1.96 | 366.78 | 1.5 | 2 | 122.33 | 1 | 1 | 65.28 | 248.78 | 248.78 | 688.58 | |||||||
IC5P | 79.2 | 1 | 36.51 | 40 | 1.98 | 129.25 | 1.5 | 1 | 65.28 | 97.92 | 97.92 | 263.69 | ||||||||||
IA6P | 24.4 | 0 | 0.00 | 0 | 0 | 0.00 | 0 | NA | ||||||||||||||
IA7P | 27 | 1 | 36.51 | 30 | 0.90 | 58.75 | 1.5 | 1 | 65.28 | 97.92 | 97.92 | 193.18 | ||||||||||
IC6P | 92.2 | 0 | 0.00 | 0 | 0 | 0.00 | 0 | NA | ||||||||||||||
IA8P | 54.2 | 1 | 36.51 | 30 | 1.81 | 117.94 | 2 | 1 | 65.28 | 130.56 | 130.56 | 285.01 | ||||||||||
IB4P | 194.4 | 1 | 36.51 | 50 | 3.89 | 253.81 | 2 | 1 | 65.28 | 130.56 | 130.56 | 420.88 | ||||||||||
Total | 621.00 | 3114 | 2655 | 6390 | ||||||||||||||||||
Total per persone | 0.45 | 2.27 | 1.94 | 4.66 | ||||||||||||||||||
Discounted total per personf | 0.44 | 2.20 | 1.88 | 4.52 |
Third feedback session cost at year 4
School | Mileage to school (round trip) | Number of travels | Total mileage cost (£)a | Staff travel time cost | Time cost for the first presentation | Time cost for the RD sessions | Total first-year feedback session staff time cost (£) | Total first-year feedback session cost (staff and travel) (£) | ||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
For all staff or teaching staff only (hours) | Additional session for support staff only | Total time cost for the first presentation (£) | For all staff or teaching staff session only (hours) | Additional session for support staff only | Total time cost for the RD sessions (£) | |||||||||||||||||
Speed assumed (mph) | Travel time for each round tripb | Total travel time cost (£)c | Number of hours | Number of SEED researchers | Total unit cost of staff per hour (£)d | Support staff only (when applicable) | Number of SEED researchers | Unit cost per hour (£) | Number of hours | Number of SEED researchers | Total unit cost (£) | Number of hours | Number of SEED researchers | Total unit cost (£) | ||||||||
IA1P | 65.2 | NA | ||||||||||||||||||||
IA2P | 53.4 | NA | ||||||||||||||||||||
IA3P | 93 | NA | ||||||||||||||||||||
IB1P | 161.2 | NA | ||||||||||||||||||||
IB2P | 163.4 | NA | ||||||||||||||||||||
IA4P | 73.2 | NA | ||||||||||||||||||||
IB3P | 165.4 | NA | ||||||||||||||||||||
IC1P | 59.2 | NA | ||||||||||||||||||||
IC2P | 38.2 | 1 | 36.51 | 30 | 1.27 | 83.12 | 3 | 1 | 65.28 | 195.84 | 195.84 | 315.48 | ||||||||||
IA4P | 48.4 | 1 | 36.51 | 30 | 1.61 | 105.32 | 2 | 1 | 65.28 | 130.56 | 130.56 | 272.39 | ||||||||||
IA5P | 83.4 | NA | ||||||||||||||||||||
IC4P | 78.2 | NA | ||||||||||||||||||||
IC5P | 79.2 | NA | ||||||||||||||||||||
IA6P | 24.4 | NA | ||||||||||||||||||||
IA7P | 27 | NA | ||||||||||||||||||||
IC6P | 92.2 | NA | ||||||||||||||||||||
IA8P | 54.2 | NA | ||||||||||||||||||||
IB4P | 194.4 |