Public engagement to refine a whole-school intervention to promote adolescent mental health

Article history

The contractual start date for this research was in November 2021. This article began editorial review in June 2023 and was accepted for publication in June 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Public Health Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.

Copyright statement

Copyright © 2024 Bonell et al. This work was produced by Bonell et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2024 Bonell et al.

Overall design

We refined LT-MH in collaboration with Place2Be (a charity specialising in promoting young people’s mental health, which subsequently supported delivery of LT-MH in schools); the National Children’s Bureau (NCB) Young NCB, a group exclusively for children and young people; staff and students from one secondary school; and a group of school senior leadership team members.

Refinement was conducted from November 2021 to September 2022. As described above, the key elements of the intervention were determined at the stage of writing the funding proposal. These are described below. After this, we turn to the main focus of the paper, how we drew on evidence and PPIE to refine LT-MH.

Initial plans for intervention

Theory of change

The LT-MH intervention was underpinned by a theory of change based on that used in the LT intervention (Figure 1) and drawing on the theory of human functioning and school organisation. 25 We also drew on evidence of the effects of other whole-school mental health interventions. 10–13 Our intervention inputs (described below) were intended to enable implementation of an action group, SEL curriculum, and restorative practice. Together these would modify schools’ social environments to: improve relationships among and between students and staff; provide more student-centred provision; increase social support for mental well-being; and teach social and emotional skills. 33 These were theorised to increase student sense of school belonging and engagement, and increase social and emotional skills. Through this, students would develop: increased emotional security in school; more trusting, empathetic, forgiving and accepting relationships with students and staff; increased self-belief (including belief in the ability to manage and resolve difficulties when they arise); and increased ability to resolve conflict, avoid peer pressure and seek support for one’s own and others’ mental health needs. These changes were theorised to mediate intervention impacts on mental health: fewer emotional problems and less disruptive behaviour; improved well-being and quality of life; improved body image and self-esteem; reduced antisocial behaviour, self-harm, substance use and disordered eating; and reduced use of NHS crisis services.

FIGURE 1.

Logic model.

Intervention refinement

Refinement was informed by an existing framework for intervention development,43 occurring in phases. Firstly, the intervention theory of change, logic model, and overall approaches were reviewed to identify required changes. Secondly, the student needs survey, the manual guiding the action group, and needs assessment guide were refined. Thirdly, an evidence-based SEL curriculum was identified. For each phase, refinement occurred through a systematic process involving: reflection on the intervention plans in light of existing evidence;7,10–13,20,21,44,45 discussions with PPIE stakeholders; drafting of resources by the research team informed by PPIE, and research team expertise and experience; and refinement of resources informed by PPIE.

Patient and public involvement and engagement involved multiple activities to provide diverse perspectives. It was consultative in orientation given that the aim was to adapt an intervention with existing evidence of mental health benefits to maximise these benefits. We felt that a more thoroughgoing reappraisal of the intervention involving more participant-led PPIE was not appropriate given the importance of maintaining the existing theory of change and effective components. Because this was PPIE and not research, formal consent was not sought for participation, and sessions were not audio-recorded. Instead, the team checked that participants were content to be involved and took detailed notes of views and suggestions. Each meeting was attended by two researchers, one leading facilitation and one taking notes. We held two meetings with Young NCB members. This is a membership group for children and young people in England, providing opportunities to speak about and contribute to matters relevant to them. The first meeting involved presentation of the aims of the LT-MH intervention by the research team. The Young NCB group was then split into two breakout rooms to discuss what young people worry about, distinguishing between new and existing worries, guided by slides indicating these topics. One researcher joined each subgroup. Again, guided by slides, Young NCB members were then asked for their views on the feasibility and acceptability in schools of the following intervention components: the student needs survey; the manual; and the manual section offering guidance on using the needs assessment to identify evidence-based actions. The second meeting involved presentation of the draft components of the LT-MH intervention and how the intervention was being refined to address mental health. The draft of the menu of evidence-based actions was presented. The Young NCB members were split into two breakout groups to: discuss the pros and cons of each intervention component; identify any gaps; and identify the best three actions and their reasons for choosing these. The group was then asked to further discuss the feasibility and acceptability of workshops on how to handle stress and of MHFA to ascertain young people’s views on how these could be delivered.

We also conducted two meetings with a purposively sampled secondary school, one with students and one with staff. We aimed to involve a school with student free-school-meal eligibility (as a proxy for student need) above the national average of 27.1% and a government school inspectorate rating of good or excellent (as a proxy for the school having sufficient organisational capacity to participate). This school was recruited via our existing contacts. The student meeting involved five boys and girls across years 8–11 and the staff meeting comprised four teachers from the senior leadership team and with pastoral responsibilities. Schools were asked to involve a diversity of students and staff, reflecting diversity across the school. Both meetings involved presentation of the intervention theory of change and overall components. Guided by the research team with slides indicating points to discuss, students were asked: whether the components made sense as an overall programme; whether students would be interested in participating; what information schools would need about student needs to inform choice of actions; and what information schools would need in order to deliver actions. Staff were asked: how the intervention should be presented to maximise school take-up and student participation; whether the intervention activities looked appropriate and manageable; whether they made sense as an overall programme; whether schools would agree to involve students in decision-making; whether students would be interested to participate; what information schools would need to deliver specific actions; whether schools would value local evidence about need and what needs they would want evidence about; and what support they would want from a facilitator and whether this should be face to face or online. We used standard operating procedures to deal with any potential safeguarding concerns becoming apparent during consultation with students but none arose.

We held one meeting with senior leadership team members from two different schools. Five schools were initially contacted for this meeting based on previously indicated interest. However, only two were able to participate in the meeting due to COVID-19-related challenges. One senior leader member per school attended. This meeting involved presentations to the staff members attending about the draft manual and menu of evidence-based actions, as well as how the results of previous meetings had informed these. Guided by researchers, senior leadership team members were asked whether: the manual guidance and menu of actions looked appropriate and manageable; these provided the right information; and participants thought these would impact on student mental health. Senior leadership team members were also asked what comparators would be most useful when benchmarking mental health needs.

After PPIE, researchers separately discussed views and suggestions of PPIE participants from notes, to decide which suggestions could be incorporated into research plans. As far as possible, suggestions were taken onboard, except where they contradicted existing evidence or were unfeasible.

Consultation with Young National Children’s Bureau members

We first met with the Young NCB members in January 2022. This session involved approximately eight members. Participants reported various worries, such as body image and self-esteem. Participants mentioned other worries, including: social media and online reputation; lack of sense of community; lack of trust in government and other institutions; sexism and misogyny; the impact of the COVID-19 pandemic; the climate emergency; and education and exams pressure. In commenting on LT-MH components, participants suggested that the need survey would need to include mostly closed-response questions. Participants suggested that the manual guiding actions groups should be read by, and include actions for, young people to maximise students’ participation.

We held a second meeting with eight Young NCB members in April 2022. In terms of improving LGBTQ+ inclusion, young people agreed that this was a good idea because they felt that LGBTQ+ young people sometimes did not feel comfortable coming out to others. However, participants thought that such activities should also encompass other groups, such as disabled students. Participants suggested that support should not be restricted to one-off events, and should include activities such as ongoing lunchtime clubs and presentations by guest speakers with lived experiences. The group felt it important that students involved in such work be well supported.

The group discussed student-led clubs focused on mindfulness, meditation, dance, and martial arts for mental health. Participants felt that the success of these would depend on who led them (possibly a role for older students) and how well they were supervised. The group discussed lessons on mental health literacy. Several thought this was a good idea but suggested that there might not be enough time in the school day. The group also discussed lessons on online safety and well-being, as well as lessons on ethnicity and mental health, there being limited support for the latter. There was not much support for an expansion of tutor time.

The group discussed stress-management workshops. All participants supported this option. Participants commented that stress comes from a variety of sources, connected to exams and life in general. They felt that this option would need to explore all potential sources of stress. Participants suggested that it might include activities such as aromatherapy sessions.

The group discussed the potential for expanding school extracurricular activities, with some participants commenting that students have very different interests so diverse options would need to be available. The group considered MHFA. One participant pointed out that this option is not preventative in nature. There was, however, widespread support for this option as many of the young people felt that their peers currently would not be properly equipped to deal with a peer who was, for example, having a panic attack. The group suggested that there could be a role for older students delivering this. The group discussed yoga, with some young people viewing this as a helpful option. However, others pointed out that this would not be suitable for all young people, such as those with disabilities. Therefore, a meditation option might be helpful.

There was little support for additional support for students transitioning from primary schools as participants felt that there was already provision in this area. The group discussed outdoors lessons and the use of physical activity in lessons. Although there was some support for these in that they would make lessons more engaging, participants highlighted that the time required should be considered.

The group discussed themed mental health weeks. Participants felt that mental health should be a priority all of the time and not just in particular weeks or months. They reported that the volume of awareness weeks in schools meant that they were not perceived as effective by students. When discussing posters and other resources on mental health, young people expressed some interest but believed that the quality of these would need to be improved. Of the options identified above, participants nominated the following as priorities: improving LGBTQ+ inclusion; lessons on mental health literacy; workshop on how to handle stress (including exam stress); MHFA; and posters and other resources on mental health.

The group then further discussed workshops on how to handle stress and MHFA, to ascertain young people’s views on how these activities could be best delivered. Regarding workshops on how to handle stress, participants thought that two sessions per week of between 30- and 60-minute duration would be appropriate. The group recommended that sessions would need to be delivered outside normal lessons by an external professional. Regarding MHFA, the group recommended that this should be delivered yearly or twice yearly, and should be for a whole day. One participant believed that a minimum level of training should be available for everyone, with optional training for selected students. All agreed that training needed to be delivered by external specialists.

Consultation with school students and staff

The school originally selected to participate matched our criteria of rate of free-school-meal entitlement above 27.1% and outstanding Ofsted inspection but dropped out at the last minute. We replaced this school at short notice. The school recruited to participate had an outstanding government inspectorate rating but below-average rate (approximately 15%) of free-school-meal entitlement. The meetings with students and staff occurred in May 2022 and involved presentations by the team followed by discussion. A staff member from Place2Be also helped facilitate discussions.

The student meeting involved discussion with two boys and three girls from across years 8–11. In terms of needs assessment, the students indicated that they would be happy for the needs survey to include mental health scales. The students emphasised the importance of participant anonymity. Students were concerned that surveying only one year group would not identify all mental health needs across different age groups. Participants thought that comparing each schools’ prevalence of mental health needs with broader data would be useful.

In terms of the action group, students agreed that it would be valuable to have staff and students meeting together. Students thought that, if participation were on the basis of students coming forward to volunteer, this would make for less-diverse participation. There was a suggestion that students could be approached to participate and could initially observe the meeting without committing to participate, as way to encourage participation among a broader group. There was a concern that selecting students on the basis of their mental health could cause participants to feel stigmatised. Participants thought that the action groups should be as student-led as possible. They suggested that meetings should be monthly and comprise six staff, six students, and the external facilitator.

In terms of the menu of actions, students were supportive of these addressing: physical activity sessions; stress management; and lessons or other activities occurring outdoors. Physical activity was viewed as something that might help improve some students’ attitudes to learning. Students commented that some of the planned activities were things already done in schools, though not linked together as a coherent programme. Students thought that there was no need for activities to be labelled as explicitly focused on mental health promotion. Some participants described stress as a problem linked to studying, not just exams, and that mindfulness or walks might help. Yoga was viewed as something that would appeal to some students. Participants also proposed that doing more group activities in classes might help promote mental health, ensuring that all students participated. In terms of restorative practice, students were supportive of this but suggested it should only be done when all parties to a conflict consented. Finally, students suggested that ‘Learning Together for Mental Health’ was too long a name.

The meeting with teachers involved four staff from the senior leadership team and with pastoral responsibilities. In terms of needs assessment, the teachers said that they would expect students to answer questions differently at different ages. There was a concern that results from year-7 students would not be representative of other year groups. Teachers thought that a different year group, such as year 8 or 9, might be better. There was also concern about survey overload, given that the school already carried out surveys.

In terms of action groups, staff thought that schools would be willing to involve students in decision-making. While they said that participation should be voluntary, they cautioned that the group should not comprise only the most vocal. It was important that membership of the group was not stigmatising of those with mental health needs. Staff suggested that students from a range of year groups should be included to give insights from across the school and, if necessary, compensate for the narrowness of focus of the needs survey on year-7 students. Teachers commented that, in terms of staff participation, all schools should have a mental health lead, who should be included in the group.

In terms of choice of actions, staff liked the idea of presenting a menu of evidence-based actions with indications of the strength of evidence, size of effects and costs. Staff were also interested in knowing where there was little or no evidence for other actions. They liked the idea that guidance should advise that if there were a certain need apparent then this would suggest the value of implementing a particular activity. It should be clear whether an action aimed to target a particular group or address the broader ethos and culture of the school. Teachers thought that actions should address the whole school, such as school rules, policies and staff training. There were some concerns about the implications of actions for teachers’ time. Teachers identified bullying prevention as a key priority and said that they would like to see evidence for interventions to address this. In terms of the SEL curriculum, staff said their school would be unlikely to adopt a new curriculum because they had already developed a curriculum meeting their requirements.

Teachers said that they would value an external facilitator for the action group. They suggested it should be provided for at least 2 years. Participants thought that in-person facilitation would be preferable but thought that a hybrid approach could work with half of meetings in person and half online. Staff felt that the facilitator should act as a critical friend, motivating and supporting implementation, and providing expertise about how to maximise outcomes. The facilitator should set and monitor success criteria for each meeting.

Consultation with school senior leadership team members

This meeting occurred in June 2022 and involved presentations by our team to a staff member each from two schools about the draft contents of the manual and the menu of evidence-based actions, as well as how the results of previous meetings had informed these.

Participants liked the whole-school, universal approach, suggesting that targeted interventions could sometimes miss those students in most need. One participant commented that the intervention and manual provided the outline and direction of what was required. Participants commented that schools would then need to identify what was required of them to implement the intervention and determine who would be responsible for delivery and oversight. The group recommended a greater emphasis on engaging parents. One participant commented that the needs assessment was very useful in quantifying need, given schools sometimes had to rely on anecdotal information. Comparisons, especially with national data, would be useful. Participants commented that the resources could be presented during Ofsted inspections, showing data on student needs and how these had been addressed.

Commenting on the menu of evidence-based actions, both participants liked the presentation of the menu and the links to all resources. A participant liked the format, given that schools were familiar with that of the Educational Endowment Foundation menu of evidence-based interventions to promote attainment. Participants agreed that it was also useful to summarise costs. One participant asked whether it could be made clear whether the evidence base referred to the actions in general or the specific materials linked to. A participant also suggested that the menu should include a disclaimer indicating that the intervention team did not necessarily endorse every resource. One participant commented that, in relation to MHFA, it would be useful to have guidance materials. This participant reported that, although many staff already received training in this, they were often unsure how to implement it without written guidance. Participants also supported the inclusion of daily miles and mental health check-ins. These have been used more in primary schools but secondary schools could also implement them.

Refinement of the intervention theory of change, logic model and overall approaches

No further refinements were made based on our review of existing systematic reviews and PPIE to the intervention theory of change and overall approach. The evidence from systematic reviews had already informed our theory of change as described above. Most PPIE comments aligned with our existing plans but some suggested possible changes, some of which we made. We did not, as one student suggested, shorten the name of the intervention as the name was judged to represent a link with an existing effective intervention and signalled the new focus on mental health. In response to a comment from senior leadership team participants to increase the focus on parents, we included various suggestions in the manual on how to win support from and involve parents but did not amend the overall theory of change or components.

Modifications to needs survey and guidance for determining actions

We retained the needs survey despite one staff member commenting that it would overload schools. This was both because it was a core component of tailoring the intervention to the school and because other participants were positive about its usefulness. We also could not act on the suggestions that the survey should broaden its focus from only year-7 students. This was because the survey of year-7 students was a central part of the evaluation protocol. However, we amended the manual so that it recommended schools draw on other data, including from other year groups, as part of the needs assessment. In response to a comment from senior leadership team participants, the manual also encouraged schools to share needs assessment findings with Ofsted inspectors where appropriate. Other comments from PPIE aligned with our existing plans.

The needs questionnaire retained the following measures, which had been included in the LT questionnaire: the Strengths and Difficulties Questionnaire (SDQ);3 the Gatehouse Bullying Victimisation Scale;46 questions on smoking, alcohol consumption, cannabis and other illicit drug use;47 the Beyond Blue School Climate Questionnaire (BBSCQ);48 and the Feeling Safe at School measure. 24 The following items were added: the Eating Disorder Examination Questionnaire Short (EDE-QS) (which included questions on body image);49 the Generalised Anxiety Disorder-7 (GAD-7) Scale; the Short Mood and Feelings Questionnaire (Child Report – Depression);50 the Health Behaviour in School-aged Children (HBSC) measure of self-harm;51 and the DAPHNE II measure of cyberbulling. 52 Decisions about which measures to include were informed by our expertise in young people’s mental health and current evidence about children and young people’s mental needs. 53

In the needs assessment report, we included plans to report on the following measures overall and stratified by gender: SDQ as a measure of general mental health as well as its subscales; EDE-QS as a measure of eating disorders; GAD-7 as a measure of anxiety; SMFQ-C as a measure of depressive symptoms; the HBSC measure of self-harm; the GBS measure of bullying victimisation; the DAPHNE II measure of cyberbullying; our measures of smoking, alcohol use and drug use; the BBSCQ measure of school commitment and its subscales; and our Feeling Safe at School measure. Where possible, we would compare school prevalence to national comparative data. Where no national data were available, we would refer to prevalence across all LT-MH schools as the comparator. We did not report mental well-being as we felt that that this overall measure would not provide useful information for schools in determining what mental health actions to take.

Drafting the intervention manual

We drafted the manual informed by the previous LT manual, the recommendations emerging from PPIE and experience within the research team developing group-facilitation manuals. The systematic reviews were not informative because they did not contain the necessary detailed information about interventions. Guidance on implementing and facilitating the action group was elaborated and given a more prominent role in the manual, given the action group’s central role in implementation of the programme. Meeting objectives, agenda items and practical exercises and tools were added to guide action groups. As recommended by students and staff, we refined the manual so that this supported the recruitment of diverse students to the action group. In response to student comments, the manual was worded to encourage schools to include students both with and without mental health problems, but did not aim to encourage staff to target students with mental health problems for recruitment.

We did not act on the suggestions that action groups should meet monthly or be student-led. The former contradicted evidence from the LT trial that meetings should not occur too frequently. 24 While students were to be empowered to actively contribute to decision-making, we decided the groups should be chaired by senior staff to support implementation of decisions, informed by the LT trial. 24 In response to staff comments, we included meeting-specific objectives in the suggested agenda, included as part of the manual. We also moved to a mixture of face-to-face and virtual external facilitation as this had been normalised post the pandemic and would enhance efficiency and scalability, which was supported by staff views. Other PPIE comments aligned with our existing plans.

Drafting of the menu of evidence-based actions

The systematic review of reviews that informed the menu is described in a separate paper. The PPIE comments were very helpful in drafting the menu. The PPIE participants supported our plans to indicate the strength of evidence, size of effects and costs of recommended actions. Our menu distinguished between general approaches and specific guidance materials and interventions, and indicated the level at which actions were to be delivered, as suggested by senior leadership team participants. We did not act on a staff suggestion that the menu should also indicate which actions were not supported by evidence since this would not be possible to do comprehensively within a short document. Most of the mental health topics and actions suggested in PPIE did feature in our menu. However, some, such as aromatherapy and mental health check-ins, did not feature because there was no evidence that these were effective. We also did not address some of the more upstream determinants of mental health suggested by students, such as lack of trust in government, since there is a lack of evidenced interventions to address these. We did not act on suggestions from Young NCB members about what specific actions should look like, such as how many sessions stress workshops should involve, because we were guided instead by the evidence base regarding the details of implementation. We amended the manual to encourage schools to consider the implications of actions in terms of staff time and their ability to continue other activities.

Refinement of restorative practice

We made few changes to the restorative practice component informed by PPIE since the suggestions aligned with our existing plans. We changed some terminology to ensure this was suitable for secondary-school students. For example, ‘circle time’ was instead referred to as ‘class group activities that promote positive behaviour and peer-to-peer support’. Informed by PPIE, we decided that the training on restorative practice should be delivered via a mixture of face-to-face and virtual sessions to reduce costs and enhance scalability.

Identification of evidence-based social and emotional learning curriculum

We retained our plans for a SEL curriculum despite one staff member commenting that their school would be unlikely to use this since they had developed their own. We judged that other schools would welcome this component given its effectiveness. The team identified and assessed various curricula cited in the systematic reviews, and assessed these based on criteria of: evidence of effectiveness; quality of materials; cost; age focus; training availability; and number of lessons. The Healthy Minds curriculum was judged most appropriate on ground of: involving a manageable number of lessons; having strong evidence in promoting multiple health outcomes,54 being inexpensive according to Education Endowment Foundation criteria; being suitable for students in years 8 and 9; and being deliverable by teachers with training provided by experts. The Healthy Minds curriculum is informed by the Penn Resilience Program and other evidence-based interventions. 55

All schools would be provided with training to deliver the six-lesson ‘Bounce Forward Resilience Skills’ section of the SEL curriculum. Teachers would be provided with 1.3 days of online training to support delivery. Schools could, if they deemed it appropriate, also deliver other Healthy Minds lessons: ‘Social Media Investigated’ (six lessons for 10- to 12-year-olds and eight lessons for 12- to 14-year olds); ‘Resilience Revisited’ (six lessons designed for 10- to 12-year-olds); ‘From School to Life’ (nine lessons designed for 12- to 14-year-olds); and/or ‘Mental Illness Investigated’ (seven lessons designed for 13- to 14-year-olds). Training and materials were provided by the Bounce Forward charity.

Summary of key findings

The finalised LT-MH intervention is a novel whole-school mental health intervention in that, like some existing whole-school interventions17,20,21 but unlike previous mental health interventions,10–13 it uses an evidence-based menu to inform local decision-making. We refined and elaborated the initial plans for LT-MH to generate an intervention supported by full materials, training and external facilitation. In line with this, we retained key components and did not modify the theory of change. The reviews of existing evidence proved useful in informing the questions on mental health included in the needs assessment survey (as well as in developing the menu of evidence-based actions which are described in a separate, forthcoming paper). However, the reviews of existing evidence did not prove very useful in informing refinement of other aspects of the intervention. These aspects of refinement involved detailed questions about delivery in English schools and existing research was generally conducted in other countries and did not engage at this level of detail concerning delivery. However, existing evidence had earlier proved useful in developing the initial plans for LT-MH in terms of what broad approaches to promoting mental health in schools had previously been effective. 7,20,21

Patient and public involvement and engagement proved valuable in informing refinement of LT-MH in terms of ensuring its feasibility, acceptability, and inclusiveness. The PPIE participants were well placed to comment on some aspects of our planned intervention. Teachers and senior leadership team members were well placed to comment on the likely feasibility of LT-MH in their schools. For example, they offered useful suggestions about the need to consider the implications of schools choosing actions from the menu. Students had a good understanding of student mental health needs, exemplified by the close alignment between the mental health topics suggested by the students and those emerging from our review of research evidence.

However, we did not act on some recommendations from PPIE. Sometimes this occurred when recommendations ran counter to the evidence, for example when students suggested the value of aromatherapy for mental health, an intervention for which there is little evidence of effectiveness. 56 It was also sometimes necessary to prioritise the evidence base over PPIE suggestions when it came to the detail of particular evidence-based actions, such as our not acting on student recommendations of how many sessions stress workshops should involve. We also did not act on PPIE recommendations when these were practically unfeasible. Our proposal had been funded based on a clear initial plan for the intervention. We could amend some aspects of this but did not feel we had the scope to radically change the intervention. Therefore, we could not act on recommendations that proposed radical reformulations, such as those proposing dropping the needs assessment survey or broadening which student year groups were surveyed. Finally, we did not act on some comments when we judged that these took an overly conservative approach to defining what is and is not possible in a setting. This was illustrated in some staffs’ view that needs assessment surveys would overload schools when there was no evidence that this was the case from previous evaluation. 24

Limitations

There is a spectrum of participation for PPIE, ranging from consultation to fully participative co-production. 29 The processes we describe here are appropriate but not innovative, and nearer the consultation than the co-production end of this spectrum. This was appropriate given our aim to adapt an intervention with existing evidence of mental health effectiveness. The research team maintained control over final decisions. However, the refinement was done ‘with’ members of the public. With the exception of the Young NCB members, the engagements with PPIE stakeholders were one-offs, not sustained over time. Sustained engagement might have allowed a more participative approach, in which we used repeated meetings to present how we had reflected on earlier advice and which suggestions from participants we had made use of and which we had not. This could have allowed discussion with participants about whether they thought that these decisions were reasonable. Nonetheless, participant views did influence how we refined LT-MH. Despite the meetings with most stakeholders being one-offs, we still took a phased approach to PPIE with sessions occurring at different points during the process of refinement of the intervention. This enabled us not only to get stakeholders’ initial views but also to seek stakeholders’ opinions on the emerging plans. A consultation approach is appropriate where there is limited scope for adapting interventions. The numbers of participants involved in PPIE was in line with protocol and appropriate given these aims.

The PPIE work was affected by its occurring during the recovery period of the COVID-19 pandemic when schools were more stressed than normal. We had planned for the school involved in PPIE to be above average in student free-school-meals eligibility but the school initially recruited dropped out and its replacement had a lower-than-average rate of entitlement. This may have resulted in the students involved being less disadvantaged and educationally disengaged. Participation at the meeting for senior leadership team members was also lower than anticipated due to COVID-19-related challenges faced by schools. We did not record demographic characteristics for participants given that this was PPIE and not research.

Implications for policy and research

It is possible to refine interventions and elaborate them to provide full materials and support via processes drawing on evidence review and PPIE. 29,57 Patient and public involvement and engagement is extremely useful in addressing questions of feasibility, acceptability and equity. 29 In this regard, the views of those who would participate in an intervention as providers or participants are critically important. Patient and public involvement and engagement is also useful in assessing whether intervention components are likely to align with the felt needs of the population being involved in or targeted by an intervention.

Not all the suggestions from PPIE can or should be accepted and acted on. 32 It can be challenging to use PPIE to refine or co-produce interventions in parallel to drawing on existing evidence when the recommendations emerging from these diverge. 30 Neither providers nor affected populations will necessarily know what interventions are likely to be most effective in meeting population needs. It makes most sense to be guided by the evidence of what has proven effective while listening to PPIE views about how things might need to be refined to maximise the likelihood of success in a new context.

Patient and public involvement and engagement recommendations are challenging to incorporate when they do not align with existing plans for the intervention. 32 Those facilitating PPIE should be clear with stakeholders about which aspects of intervention design are negotiable and which are non-negotiable. It may be that, in some cases, funding proposals need to be less prescriptive so that there is more scope for intervention refinement informed by PPIE. Patient and public involvement and engagement recommendations can also sometimes be challenging to incorporate when there is a feeling that they might simply reflect participants thinking aloud or where stakeholders might be moving beyond their area of expertise or experience when offering comments. As with some previous analyses of public participation in health decision-making,58 we recommend that PPIE involves more open dialogues between researchers and PPIE stakeholders. Researchers and PPIE stakeholders should feel able to probe and challenge each other’s views, and examine what light evidence can shed on the questions being discussed. Patient and public involvement and engagement would also benefit from sustained engagement so that, where recommendations are not accepted, this can be reported back to participants and further discussed if necessary.

The approach to adaptation was not informed by recent guidance59 because it was planned prior to publication of this. Furthermore, the focus was slightly different: adapting an existing intervention to maximise its effects on mental health rather than adapting it to a new context. Nevertheless, the approach was consistent with this guidance in terms of: involving diverse participants; being clear about decision-making authority; ensuring appropriate consistency with the intended theory of change; considering contextual influences on feasibility and acceptability; and ensuring intervention materials are amended appropriately. Despite the consultative approach to PPIE and the challenge recruiting a disadvantaged school, PPIE was useful in informing the limited adaptation of LT-MH required. Future work will include a feasibility study to optimise the intervention and assess whether progression to a full trial is justified.

CRediT contribution statement

Chris Bonell (https://orcid.org/0000-0002-6253-6498): Conceptualisation, Formal analysis, Funding acquisition, Investigation, Methodology, Resources, Software, Supervision, Validation, Visualisation, Writing – original draft.

Steven Hope (https://orcid.org/0000-0003-4159-9518): Data curation, Formal analysis, Project administration, Writing – editing and reviewing.

Neisha Sundaram (https://orcid.org/0000-0003-4159-9518): Formal analysis, Writing – editing and reviewing.

Oliver Lloyd-Houldey (https://orcid.org/0000-0001-6081-3546): Data curation, Project administration, Writing – editing and reviewing.

Semina Michalopoulou (https://orcid.org/0000-0003-3198-0411): Project administration, Writing – editing and reviewing.

Stephen Scott (https://orcid.org/0000-0003-4680-6213): Conceptualisation, Funding acquisition, Investigation, Methodology, Writing – editing and reviewing.

Dasha Nicholls (https://orcid.org/0000-0001-7257-6605): Conceptualisation, Funding acquisition, Investigation, Methodology, Writing – editing and reviewing.

Russell Viner (https://orcid.org/0000-0003-3047-2247): Conceptualisation, Funding acquisition, Investigation, Methodology, Resources, Supervision, Validation, Visualisation, Writing – editing and reviewing.

Acknowledgements

We thank all those who participated in our PPIE.

Data-sharing statement

PPIE sessions were not recorded there is therefore no data that can be shared. All queries should be submitted to the corresponding author in the first instance.

Ethics statement

The research was conducted in accordance with the World Medical Association Declaration of Helsinki. Regulatory approval for the overall research project was given by the research ethics committees of University College London (reference 21179/001 on 30 May 2021) and the London School of Hygiene and Tropical Medicine (reference 27994 on 26 August 2022). Participation in the PPIE subcomponent of the overall study was deemed not to be research participation by these ethics committees and so formal written consent was not sought. However, participants were given verbal information about the work and freely agreed to participation. No PPIE sessions were recorded other than via written notes, which did not attribute any statements to individuals.

Information governance statement

All personal information was managed in line with the Data Protection Act (2018) and General Data Protection Regulation (EU GDPR) 2016/679. University College London was the data controller and data processor.

Disclosure of interests

Full disclosure of interests: Completed ICMJE forms for all authors, including all related interests, are available in the toolkit on the NIHR Journals Library report publication page at https://doi.org/10.3310/JWGT4863.

Primary conflicts of interest: Chris Bonell was a member of the Research Funding Board, NIHR Public Health Research Programme between 2013 and 2019. Fifteen per cent of Stephen Scott’s time is funded by NIHR for a RCT of personalised Programme for Children with conduct problems.

Department of Health and Social Care disclaimer

This publication presents independent research commissioned by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, MRC, NIHR Coordinating Centre, the Public Health Research programme or the Department of Health and Social Care.

This article was published based on current knowledge at the time and date of publication. NIHR is committed to being inclusive and will continually monitor best practice and guidance in relation to terminology and language to ensure that we remain relevant to our stakeholders.

Study registration

This study is registered as Current Controlled Trials ISRCTN15301591.

Funding

This article presents independent research funded by the National Institute for Health and Care Research (NIHR) Public Health Research programme as award number NIHR131594.

About this article

The contractual start date for this research was in November 2021. This article began editorial review in June 2023 and was accepted for publication in June 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The Public Health Research editors and publisher have tried to ensure the accuracy of the authors’ article and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.

Copyright

Copyright © 2024 Bonell et al. This work was produced by Bonell et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.