Project objectives

The overall aim is to conduct a feasibility study followed by a pilot cluster RCT of an intervention that provides mental health support for secondary school staff. Specifically, staff will be provided with:

1. one to one support for work related and more general problems via staff peer support teams;
2. training in supporting student mental health (which may also impact on their own mental health, through increased knowledge and confidence).

Staff peer supporters will be trained in Mental Health First Aid (MHFA). This is a course developed and successfully trialled in Australia, which is widely used in England, but has never been evaluated in an RCT. The training offered to the wider staff body will be youth MHFA, a specially adapted version designed for adults working with teenagers.

Key questions to be addressed include: How appropriate is MHFA for the English secondary school context and what is the best format for its delivery? Which school staff are the most appropriate members of peer support teams? What is the most acceptable/accessible way for staff to receive the support? How acceptable is the intervention to schools, how much fidelity of delivery can be expected, and what proportion of schools will be willing to participate in a full trial? To what extent does context (e.g. school socioeconomic catchment area) affect delivery and outcomes? How can the intervention be made sustainable? What should be the main outcome measure for a full RCT? What is the intraclass correlation coefficient (ICC) and standard deviation (SD) for the main outcome measure, and what sample size is required for a full RCT? What does the intervention and evaluation cost?

The findings will be used to develop a proposal for a full RCT if the following criteria are met, or if we can develop strategies to meet them:

1. able to recruit and maintain peer supporter teams at a ratio of 1 to every 15 staff members;
2. 50% schools are willing to be recruited and randomised in the trial;
3. a response rate of at least 75% in the staff and student questionnaires;
4. an average of at least one intervention a month delivered by each peer supporter.

Changes to project objectives

In the original proposal we said we would include 6-8 schools in the pilot - we only included 6, because the baseline data collection coincided with the follow up of the feasibility schools, so we wanted to ensure we were not stretched beyond capacity. We also decided to limit our study to schools in Bristol and North Somerset, not Bristol and Northamptonshire as originally proposed. This was because we used teams of data collectors local to Bristol.

Brief summary

Methods

Sample and allocation

Two schools initially took part in a feasibility study, in which the intervention was delivered and baseline and follow up measures taken. Six schools were then recruited to take part in a pilot cluster RCT, covering a range of socioeconomic catchment areas and academic results. Following the collection of baseline measures, the six schools were paired according to free school meal eligibility (FSM) as a proxy for socioeconomic catchment area, and the schools within each pair were randomly allocated to intervention or control arm with the use of Stata by a statistician blinded to the actual school identities.  This resulted in control and intervention arms containing 3 schools each.

The Intervention

There were two parts to the intervention: i) 8-10 staff peer supporters were selected via a nomination process among all staff, and received two days training in Mental Health First Aid (MHFA) by an accredited trainer, before setting up a confidential support service for colleagues ii) two days training in youth MHFA was delivered to up to 25 staff.

Main and secondary outcome data collection and analysis

Baseline measures were collected before the intervention via self-complete questionnaires from all staff during staff meetings or training time, and students in years 8 and 9 during lesson time. The main outcome measure was staff wellbeing, measured using the Warwick Edinburgh Wellbeing Scale (WEMWBS). Secondary outcomes were staff depression, measured by the 9-item Patient Health Questionnaire (PHQ-9), staff absence, staff presenteeism, measured using an item from the Work Productivity and Activity Impairment questionnaire (WPAI), student wellbeing, using the WEMWBS, and student mental health difficulties, using the Strengths and Difficulties Questionnaire (SDQ). These measures were repeated at a 12 month follow up.

For staff and student outcomes, linear or logistic regression models were performed as appropriate to examine the outcome at follow up by arm, adjusted for baseline outcome and for FSM, as the sample were originally stratified and then paired according to this variable. Models for student outcomes were also adjusted for school year, as missing data analyses revealed differences by arm. For staff outcomes, a sensitivity analysis was conducted including teachers only (i.e. excluding support staff). We also ran all analyses again with the two feasibility schools included. All analyses took account of clustering by school using robust standard errors. We assessed the impact of non-response and missing data on the outcomes using MICE (Multivariate imputation by Chained Equations) implemented using the ice routine in Stata.

Process data collection and analysis

Quantitative and qualitative process data were collected. Staff questionnaires measured:

1. mental health knowledge and attitudes, and confidence and frequency in helping others.
2. use of the peer support service and reasons for non-use in intervention schools only. Half termly logs were collected from each peer supporter recording number of colleagues supported, time spent supporting others and number of colleagues helped in addition to usual role over the previous two weeks.

Qualitative data were collected during observations of the MHFA training, interviews with key senior contacts at each school and focus groups with peer supporters, attendees of the youth MHFA training, and non-trained staff, to examine fidelity of delivery, views of the training and peer support service, how the intervention worked in the different contexts, and likely sustainability. All interviews and focus groups were audio-recorded and transcribed. The different groups of data were initially analysed separately using constant comparison techniques common to qualitative research. Transcripts were scrutinised for emergent themes, and relevant sections of text were coded and grouped together according to those themes. Where relevant, themes were then compared across the different groups of data for similarities and contrasts.

Data regarding costs of the intervention (training and peer supporter time spent providing help that was additional to usual) and costs of staff absence were collected.

Results (including findings in relation to the objectives)

Of the 32 schools in Bristol, North Somerset and South Gloucestershire invited to participate in the pilot study via a letter to the head teacher, 8 requested further information, of whom 7 arranged a meeting with us, and 6 took part. No recruited schools left the study. The pilot schools contained a total of 1,024 staff and 2,616 students in years 8 and 9.

At baseline, 623 (59.9%) of all staff (74.1% of teachers) and 2265 (86.6%) students completed the questionnaires. At follow up, 544 (54.7%) staff (69.2% teachers) and 2274 (87.0%) students completed the questionnaires. In total, 8 training observations, 6 interviews with senior staff contacts, 10 focus groups and 5 staff interviews were completed.

Qualitative findings

The training was delivered either as complete days with cover paid for, or broken down into a number of in-service training sessions, depending on what suited the schools. Generally, it was received positively by those who attended, with specific benefits cited as new knowledge and skills, reassurance about current practice, opportunities for discussion of difficult work situations, and awareness of one’s own mental health. There was a common view among participants that student mental health is deteriorating and youth MHFA was seen as very relevant in this context. Suggestions for improvement to the youth version included ensuring that tutors and non-specialist teachers attended as they were perceived to have the most to gain from the course, reducing the length to make it easier for more teachers to attend, and making the content less factual and more skills focused. The training from both courses was not systematically ‘cascaded down’ to colleagues, but in one school trainees used staff meetings and assemblies to introduce some of the material.

Peer supporters covering a range of roles and experience in the schools were recruited based on nominations from staff questionnaires. It was not felt to be appropriate for senior leaders to be peer supporters, but they were seen to have an important role in supporting and promoting the service, rather than treating it as a ‘tick box exercise’. All those who attended the training remained as peer supporters for the duration of the study. Peer support tended to be delivered immediately and informally, with staff approaching supporters as and when they needed to. This was seen as appropriate in the fast paced and reactive school context, where little time and space was available for pre-planned, formal conversations. There was a strong sense among participants that school staff are under a great deal of pressure, and that the peer support service had the potential to reduce some of this. Examples of how the service had provided help ranged from staff who had wanted to talk about a difficult lesson they had just experienced to individuals with more serious and enduring mental health difficulties who needed to access outside services. The existence of the service was also considered by some participants to have raised the profile of staff mental health, in a context where performance management had largely suppressed conversations around this issue. Peer supporters described benefiting themselves from feeling they were making a positive difference in the lives of those they helped and receiving training that helped them fulfil a support role that they held. However, the potential for peer supporters’ mental health to be negatively affected was also discussed, for example if they no longer had any time for themselves, or if they worried about whether they had done enough to help staff who had approached them. Non-trained staff were generally positive about the existence of the service, although barriers to its use were cited, including not knowing about it, preferring to gain support from people that they already had a strong relationship with inside or outside of work, and not wanting to be a burden on the peer supporters. It was also noted that the service needed time to become trusted and embedded in school life. A few participants talked about using the service themselves, because they did not want their staff team to know about their problems, or because they had not been in the school long enough to develop support networks. One service user described how she would not have been able to continue with the day if she had not been able to access support at her moment of need. Participants discussed the need for ongoing promotion of the service to ensure sustainability, including at the beginning of each academic year to inform new staff. There was no evidence that delivery of the training or establishment of the peer support service was more or less challenging, or that the intervention was considered more or less relevant depending on a school’s socioeconomic catchment area.

Quantitative findings – process evaluation

A mean ratio of 1 peer supporter per 20 staff members were able to be recruited all of whom completed the training. According to the peer supporter logs, the median number of people helped by each supporter during a two week period was 2 (range 0-10), with 0.8 (range 0-4) people helped in addition to what would have happened anyway. The median time per supporters spent helping others during a two week period was 1.13 hours (range 0-9). In total, 6.3% respondents in the follow up staff questionnaire reported using the peer support service, and 14.1% said they would if they needed to.

According to the follow up questionnaires, once baseline measures were controlled for, those who completed either the adult or youth MHFA course had greater confidence in helping a colleague (OR=6.34[3.18, 12.64], p<0.01) and helping a student (OR=4.85[0.78, 29.91], p=0.09), and greater odds of providing help once a month to distressed colleagues (OR=1.54[0.76, 3.11], p=0.23) and students (OR=2.10[1.57, 2.81], p<0.01) compared to all other staff.  Those who completed either course also had greater mental health knowledge (difference=1.84[0.13, 3.56], p=0.04), and less stigmatising attitudes towards depression (difference=1.87[0.93, 2.80], p<0.01) and anxiety (difference=2.01[0.42, 3.59], p= 0.02). Finally, those who completed the training had better wellbeing (49.8 versus 47.4, p=0.09) and lower depression (4.0 versus 5.4, p<0.01) compared to all other staff, once baseline scores were adjusted for.

Quantitative findings – main outcomes

The ICC for the main outcome measure (staff WEMWBS) was 0.01 (95%CIs 0, 0.03) at baseline, and 0.00 (0, 0.02) at follow up.

Once baseline scores and FSM were adjusted for, there was no evidence of a difference in staff wellbeing or depression in the intervention arm compared to the control arm (WEMWBS 47.7 versus 47.2, p=0.72; PHQ9 5.4 versus 5.3, p=0.60).  Likewise there was no evidence of differences in absence (Odds Ratio (OR) 1.15 [0.96, 1.38], p=0.56) or presenteeism (OR 0.96 [0.46, 1.89], p=0.86) in intervention vs. control schools. Restricting the analysis to teachers only did not change the findings. Results for the complete case and imputed datasets were similar.

Once baselines scores, FSM and school year were adjusted for, students in the intervention arm had higher wellbeing (48.1 versus 45.8, p=0.12) and lower difficulties (11.2 versus 13.2, p<0.01) compared to those in the control arm.

The findings were not substantially changed when the analysis was run for teachers only, nor when the feasibility schools’ data were included.

Conclusions

We successfully conducted a feasibility study followed by a pilot cluster RCT of an intervention in secondary schools that i) created a confidential peer support for staff ii) provided training for staff in youth MHFA.

Qualitative findings indicate that the adult and youth MHFA training was perceived as relevant and useful by school staff. However, a shorter youth MHFA course with more of a focus on skills would be even more suited to the schools context. The idea of a peer support service was also viewed positively by school staff, and a number of stories were shared by peer supporters and service users as to how it had helped individuals. However, for such a service to become widely used and to be sustainable, certain key features emerged as important: all the peer supporters to complete the MHFA training, enough peer supporters, from a range of roles, to ensure that all staff seeking help would be familiar with at least one of them, a clear confidentiality policy, potential users to choose which supporter to approach, ongoing promotion of the service and a senior staff member to ‘champion’ it. In a future trial, fidelity to these features would be necessary, while allowing the detail of how the service is used to be adaptable to suit the contexts of the different schools.

The study was not powered to detect an effect of the main outcome measure. The secondary outcomes student wellbeing and student mental health difficulties did show a positive effect in the intervention group compared to the control group. There were also positive signs that those who had completed the MHFA training had improved in terms of mental health knowledge and attitudes, in confidence and actual helping behaviour, and in their own wellbeing and depression scores, although this was a non-random sample. The pilot has provided us with the necessary information - ICCs and SDs for the main outcome measure at baseline and follow up, size of schools and response rate - to calculate sample size required for a full RCT. We were also able to gain the necessary data regarding cost of the intervention (trainer fees, trainer travel costs, costs of cover and costs of venue hire), costs of peer supporter’s time, and costs of staff absence (based on anonymised absence data with salary grade information attached).

The criteria for proceeding to a full RCT were not all met, but pilot findings enabled us to develop strategies to meet them in a future trial:

1. We recruited a mean ratio of 1 peer supporter to 20 staff members, rather than the 1 in 15 which was our target. In fact we would aim for a higher ratio than this in a full RCT, due to the finding that staff did not always know much about the service, or did not feel they knew the peer supporters very well. We would make the number of peer supporters to be trained a requirement contained within a research contract between the school and the study team and would run several joint training sessions across schools to enable a higher number to be trained within each school.
2. 50% of the schools originally written to were not recruited. However, in a full trial we would identify a relevant staff member to approach first via local Healthy Schools coordinators within public health teams, for example a pastoral lead rather than the head teacher. We believe this would improve recruitment rates.
3. Response rates were higher among teachers than support staff. Teachers had significantly poorer wellbeing and higher depression than support staff, we have therefore decided we would focus on teachers only in a full trial. Among the teachers, the response rate was lower than the target 75% at follow up. This was largely due to data collection meetings being cancelled, or individuals not being present. In a full trial we would build in more time and resources during follow up to return to schools on several occasions, thereby increasing response rates. We would also include the requirement to hold a data collection meeting in the research contract.
4. According to the peer supporter logs, more than one intervention a month was delivered on average. The amount of support delivered by each supporter varied a great deal; in a full trial we would work with peer support teams to try and ensure that requests for help are shared more equally.
5. We conclude that the evidence is sufficient to conduct a full RCT of the WISE intervention, incorporating the improvements outlined above. This would enable us to establish if the intervention has an effect on teacher mental health, and if these effects are sustainable over a longer time period. An application for funding such a trial has been submitted to the NIHR Public Health Programme funding board.

Plain English summary

Background

School staff, in particular teachers, are consistently shown to have increased risk of stress, depression and anxiety compared to the general working population. If left untreated, such mental health issues may lead to poor performance at work (presenteeism), sickness absence, and even ill health retirement. Further, school staff report feeling ill-equipped to support vulnerable students, due to a lack of support for their own wellbeing, and a lack of training in mental health.

Methods

The WISE pilot study was an evaluation of a secondary school intervention, in which:

1. a small group of staff were trained in Mental Health First Aid (MHFA), and went on to provide a confidential peer support service for colleagues to use at when stressed or distressed;
2. youth MHFA training was delivered to the wider staff body, to equip them to recognise signs of distress and provide initial support and help.

We tested out the intervention and the outcome measures in two schools in an initial feasibility study. Six further schools then took part in the pilot study; three received the intervention and three were a comparison group. We measured staff wellbeing, depression, absence and presenteeism, and student wellbeing and mental health difficulties (years 8 and 9) before the intervention began, and one year later. We also conducted observations of the training sessions, and interviews and focus groups with staff, some of whom had completed the training, some who had not.

Results

Staff who attended the training felt that in what they perceived to be a context of high levels of staff and student stress, the course was relevant and useful. The peer support service was seen as very helpful to a number of individuals, although it was suggested that greater promotion, clear support from senior leaders and a larger number of peer supporters would increase service use. There were no large differences between intervention and comparison groups for staff outcomes, but this may have been because this was a pilot study with a small sample. Students in the intervention schools had better wellbeing and lower mental health difficulties at follow up, compared to the comparison group.

Conclusions

The MHFA training and peer support service were well received by staff, and there were positive signs regarding the effect on student mental health. The intervention has the potential to improve school staff mental health, and student mental health via enhanced skills among staff. A larger study over a longer time period is needed to definitively establish the impact on staff and student mental health.

Dissemination

Academic dissemination 

Kidger J, Evans R, Tilling K, Hollingworth W, Campbell R, Ford T, Murphy S, Araya R, Morris R, Kadir B, Moure Fernandez A, Bell S, Harding S, Brockman R, Grey J, Gunnell D. Protocol for a cluster randomised controlled trial of an intervention to improve the mental health support and training available to secondary school teachers – the WISE (Wellbeing in Secondary Education) study. BMC Public Health (2016) 16:1089 DOI: 10.1186/s12889-016-3756-8 https://goo.gl/ZPQm2y 

Kidger J, Stone T,  Tilling K, Brockman R, Campbell R, Ford T, Hollingworth W, King M, Araya R, Gunnell D. (2016) A pilot cluster randomised controlled trial of a support and training intervention to improve the mental health of secondary school teachers and students – the WISE (Wellbeing in Secondary Education) study). BMC Public Health 16: 1060.

Kidger J, Brockman R, Tilling K, Campbell R, Ford T, Araya R, King M, Gunnell D. Teachers’ wellbeing and depressive symptoms, and associated risk factors: a large cross sectional study in English secondary schools. Journal of Affective Disorders 192 (2016) 76-82 doi.org/10.1016/j.jad.2015.11.054 https://goo.gl/377XYt

Kidger J. A cluster RCT of an intervention to improve the mental health support and training available to secondary school teachers – the WISE (Wellbeing in Secondary Education) project. South West Public Health Scientific Conference. Bristol, 16 Mar 2016.

Kidger J “Mental Health Support and Training for Secondary School Teachers: The WISE (Wellbeing in Secondary Education) pilot randomised controlled trial”. South West Public Health Scientific Conference, February 2015.

Kidger J, Evans R. "How can it be anonymous?  Every single one of those questionnaires had a number on it!" The challenges of school based research. NIHR Schools Event: Public health research in schools. London, 21 Oct 2014.               

A summary report will be available in paper form, and published on our project website:http://www.bristol.ac.uk/social-community-medicine/projects/wise/

Non-academic dissemination

An oral paper was delivered at the Manchester Festival of Public Health, University of Manchester, 2nd July 2015: “A pilot cluster randomised controlled trial to improve the mental health support and training provided to secondary school teachers (the WISE project)”.

A poster presentation was given at Public Health England’s Annual Conference, September 2015: “A pilot cluster randomised controlled trial to improve the mental health support and training provided to secondary school teachers (the WISE project)”.

A lay summary report will be shared with non-academic collaborators (Mental Health First Aid, Teacher Support Network, Bristol City Council Public Health practitioners) and will be made available on their websites.

Public involvement

We found it very useful to consult school staff widely during the development of our study – we contacted senior leaders at local schools, spoke to members of the National Teacher Research Panel, and appealed for teachers to pilot our materials through the Teacher Support Network. The feedback gained helped shape the intervention delivery and the way we conducted the evaluation to make this more acceptable to schools. We also found it very valuable to hold a workshop for representatives from our participating schools to attend at the end of the study, the aim of which was to gather feedback on their experiences, and to consider sustainability and future research priorities. The discussion between individuals that was facilitated proved more fruitful than asking teachers for opinions on a one to one basis. We also found having practitioners on the steering group – staff from Bristol City Council and the Teacher Support Network – useful for discussions around how to engage schools, how to interpret and disseminate findings, and future research.

Impact

The study is highlighted in Public Health England’s recent guidance “Promoting children and young people’s emotional health and wellbeing” (March 2015) in a chapter devoted to the importance of school staff’s development, health and wellbeing.

Findings regarding the acceptability and relevance of Mental Health First Aid (MHFA) for schools have been fed back to MHFA England, and as a direct result of this, MHFA England have developed a schools’ version of the training programme, which is more appropriate for this particular setting and is now available to all schools. This feedback has been very timely, as MHFA is currently gaining a lot of publicity in both local and national media for its use within the workplace and schools (e.g. http://www.thetimes.co.uk/tto/education/article4429671.ece,http://www.leicestermercury.co.uk/New-project-tackle-mental-health-issues/story-26523927-detail/story.html). As a result of our collaboration with MHFA, David Gunnell helped them to edit and update the section of their training manual covering self-harm and suicide.

This project was funded by the National Institute for Health Research School for Public Health Research (project number SPHR-BRI-PH1-WIS).

Department of Health Disclaimer

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR School for Public Health Research, NIHR, NHS or the Department of Health.