Emotional literacy programme in special schools for children with a learning disability in England: the ZF-SEND feasibility RCT

Article history

The research reported in this issue of the journal was funded by the PHR programme as award number NIHR129064. The contractual start date was in March 2020. The draft manuscript began editorial review in July 2023 and was accepted for publication in March 2024. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The PHR editors and production house have tried to ensure the accuracy of the authors’ manuscript and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this article.

Copyright statement

Copyright © 2024 Stenfert Kroese et al. This work was produced by Stenfert Kroese et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaptation in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2024 Stenfert Kroese et al.

Why is this research important?

A learning disability is characterised by an intelligence quotient below 70 and associated deficits in adaptive functioning, arising before the age of 18 years. It is estimated to affect 1.4–2% of the UK population. 1 Children with a learning disability are four to five times more likely to have a mental health disorder compared with other children and account for 14% of all children with mental health problems. 2 Their parents, especially mothers, are also more likely to report psychological problems. These health inequalities for children with a learning disability and their parents emerge early in the child’s life. 2

Social exclusion and poverty are more likely to be experienced by children and young people with a learning disability, along with other negative life experiences, such as health issues, abuse and bereavement, as well as having fewer friends than other children. These biological, psychological and environmental factors increase their risk of developing mental health difficulties. 2 At least half of children with a learning disability are victimised, rejected or mistreated by peers3 and, compared with their typically developing peers, 75% of children and young people with a learning disability have low social competence4 and low levels of emotional literacy and coping skills, which further increases their risk of developing mental health difficulties. 5 These early negative experiences have long-term consequences, as young people with mental health difficulties are more likely to have further negative life experiences and unequal life chances as they progress into adulthood. 6

Access to specialist mental health support poses challenges and it has been reported that fewer than 30% of children have access to such services. 7 Thus, children with a learning disability and their parents face significant health inequalities and problems gaining access to appropriate and timely services.

For the general population, there is compelling and consistent empirical evidence that social and emotional competencies can be taught and that these competencies lead to positive and significant improvements in mental health and well-being, behaviour and academic achievement. 6 Improved social and emotional literacy may therefore mitigate some of the impact from inequalities experiences by children with a learning disability. Given this evidence, interventions are needed that aim to protect and improve the mental health and resilience of children with a learning disability. Despite higher prevalence rates of mental health problems in children with a learning disability2 and research demonstrating a link between emotional literacy and mental health in adults and adolescents in the general population, there has been limited research that has examined the link between emotional literacy and mental health in children with a learning disability.

A recent systematic review of randomised controlled trials (RCTs)8 indicates that most reported interventions (including school-based interventions) designed to improve psychosocial–behavioural functioning of school-aged (5–18 years) children with a learning disability may be effective. However, the studies included in this review only report on intellectual functioning and adaptive skills (e.g. communication, social skills and educational/vocational functioning) and were not designed to measure impact on emotional literacy.

The findings of our early (uncontrolled) pilot work5 suggest that an adapted school-based intervention, a programme called Zippy’s Friends (ZF) adapted for children with special educational needs and disabilities (SEND), is acceptable to and valued by teachers, with some promise of improvements in mental well-being, social interactions and problem solving in children with a learning disability. It is therefore important to establish in a controlled and systematic manner whether a school-based emotional literacy intervention such as ZF can be effective in protecting and improving the mental health and resilience of children with a learning disability.

Before such research is conducted, a feasibility study is required to investigate whether ZF-SEND can be delivered successfully to small groups of children in special schools by teachers in a classroom setting, and whether it would be feasible to conduct a large-scale RCT of the effectiveness and cost-effectiveness of ZF-SEND.

Conceptualisation of emotional literacy

Emotional literacy has been defined as:

the ability to perceive accurately, appraise and express emotion, the ability to access and/or generate feelings when they facilitate thought, the ability to understand emotion and emotional knowledge; and the ability to regulate emotions to promote emotional and intellectual growth. 9

Bar-On’s10 model of emotional literacy is the most comprehensive and inclusive conceptualisation of this construct, including an array of emotional, personal and social abilities and skills that influence an individual’s ability to cope effectively with environmental demands and pressures. The key factors involved in this model include intrapersonal capacity (understanding, awareness and expression of one’s emotions), interpersonal skills (understanding, awareness and appreciation of others’ feelings), adaptability (altering one’s feeling and thoughts according to different situations and solving interpersonal problems), stress management (coping with stress and strong emotions) and motivational and general mood factors.

Evidence for the positive effects of improving emotional literacy on mental health

Emotional literacy skills have been shown to be associated with resilience to mental health problems. 11 When individuals have a broad repertoire of coping skills they are considered to have ‘coping flexibility’ and research12 has shown that having such flexibility is associated with positive short- as well as long-term outcomes. Studies on coping distinguish between strategies that focus on decreasing the negative feelings a person has after a difficult or stressful situation (‘emotion-focused coping’) and those which attempt to improve or change the situation (‘action-focused coping’). Emotional literacy is associated with (and ZF addresses) both these types of coping strategies.

Research findings suggest that high emotional literacy reduces stress, improves self-esteem and reduces rates of emotional difficulties later in life. A meta-analysis conducted in 2010,13 which includes adult and adolescent participants (from the general population) found evidence that higher emotional literacy is linked to better mental health. A more recent study14 also suggests that emotional literacy predicts mental health in adolescents (without a learning disability) and concludes that teaching emotional literacy is an effective preventative intervention, as emotional literacy was a significant predictor of psychological well-being and adjustment.

Zippy’s Friends, a school-based intervention designed to enhance emotional literacy

Schools have an important role to play in helping to identify mental health difficulties. Early detection and intervention are key so that children get the support they need, when they need it. 6 The UK Government’s Green Paper ‘Transforming Children and Young People’s Mental Health Provision’15 proposes a new joint working approach between schools and the NHS in England to help children and young people live fulfilling and happy lives. National guidance on the social and emotional well-being of children by the National Institute for Health and Care Excellence (NICE) recommends that priority should be given to those children most at risk of mental health problems. 6 However, the recent government ‘SEND and Alternative Provision Improvement Plan’16 fails to mention mental health or emotional literacy promotion as a priority area for children with SEND or a learning disability.

A systematic review17 concludes that schools that promote positive mental health and help children to cope with negative life experiences can create psychological resilience. This review shows school-based psychoeducational interventions to have positive effects on outcomes, including mental health, social, emotional and educational factors for families, children and communities, with the most effective interventions including skills teaching, liaison and education of teachers and parents, involvement in the community, continuity of interventions starting with young children, long-term whole-school approaches, adaptations to the curriculum and a focus on positive mental health. A recent systematic review of classroom-based mental health interventions for children in adverse environments18 also found evidence that such interventions can promote resilience to psychological problems. However, the authors stressed that risk of bias, especially due to confounding variables and deviation from intended intervention delivery, suggests that the findings of most of the 17 included studies should be interpreted with caution.

Zippy’s Friends for mainstream schools has been extensively evaluated in a number of studies in and outside the UK. 19–24 An early (2010) systematic review found support for the effectiveness of ZF for children in mainstream schools, improving coping skills and increasing emotional vocabulary and positive behaviours. 25 The review identified four controlled studies, conducted between 2000 and 2010. Subsequently, research published in 2010 on the effect of ZF on the emotional well-being of 523 primary school children in ‘disadvantaged’ schools in Ireland found a significant positive effect of ZF on emotional literacy, with significant increases in the intervention group’s scores for self-awareness, self-regulation, motivation, empathy and social skills. 20 In 2014, the same authors reported that the significant increase in emotional literacy in the intervention group was maintained at 12-month follow-up. 23 A large RCT with 7- to 8-year-old children (N = 1483) in Norway also found ZF to have a significant positive impact on coping and mental health outcomes. 22

The studies mentioned so far have all been conducted in mainstream schools. Other than the small pilot study carried out by ourselves5 (with no control condition and no recording of feasibility outcomes), to date we have found no studies reporting on trials of whole-class or school-based mental health interventions for children with a learning disability and/or for special schools. Thus, an evidence inequality exists and research on early school-based interventions designed to improve social/emotional functioning and mental health is needed urgently for children with a learning disability.

Rationale for the study

In brief, conducting a feasibility study of ZF-SEND is important because:

• children with a learning disability experience a range of inequalities, which puts them at greater risk of mental and physical health problems in adulthood;

• children and young people with a learning disability experience negative life events and adversity more frequently than their non-disabled peers;

• the construct of emotional literacy has been shown to be a distinct and moderating factor of how life stress affects mental health and well-being;

• there is evidence that teaching emotional literacy in primary schools is an effective early intervention to promote positive mental health and help children cope with negative life experiences, resulting in better mental health in later life;

• in mainstream schools, the ZF programme has been shown to be an effective way in which to improve emotional literacy, coping skills and mental health outcomes;

• emotional literacy is underemphasised in the SEND curriculum and mainstream emotional literacy programmes (except ZF-SEND) do not have SEND adaptations;

• NICE recommends that help should be given to those most at risk of mental health problems;

• lack of investment in mental health promotion in primary schools, particularly special schools, has significant costs for society;

• there is an identified need for SEND-adapted emotional literacy programmes in special schools.

Aims

The primary aim of the study was to evaluate whether it is feasible to conduct a large-scale RCT of the effectiveness and cost-effectiveness of ZF-SEND for children with a learning disability in special schools. We aimed to assess the acceptability and feasibility of: (1) participating in the trial; (2) data collection and (3) the ZF-SEND intervention, through a feasibility RCT which aimed to recruit and randomise 12 special schools to either deliver the ZF-SEND intervention over 1 academic year (6 schools) or continue with practice as usual (PAU; 6 schools) and to collect data from 96 pupils at baseline (pre randomisation) and 12 months post randomisation.

The study explored the following, in relation to each aim:

Participation in the trial:

• Feasibility of recruiting and retaining eligible schools and participants to the study and to identify the most effective recruitment pathways

• The acceptability of study processes, including randomisation, to schools, teachers and parents/carers.

Data collection:

• Feasibility and acceptability of the proposed outcome measures as methods to measure the effectiveness of the intervention and to conduct an embedded health economic evaluation within a large-scale RCT.

The ZF-SEND intervention:

• The feasibility of recruiting suitable schools and teachers to deliver the intervention

• Adherence to the intervention and fidelity of implementation

• Acceptability of the intervention to teachers, pupils and parents/carers.

Objectives

• To collect quantitative data using a range of standardised measures at baseline and 12 months post randomisation from teachers, pupils and parents/carers.

• To conduct qualitative interviews with pupils, teachers, senior leadership staff and parents/carers to explore their experiences of participating in the feasibility study (two pupil participants in each ZF-SEND school, one teacher in each school, a senior member of staff in each school and one or two parents/carers in each school).

• In the ZF-SEND arm, to assess intervention delivery, fidelity and adherence, and factors influencing implementation, mechanisms of impact and context using data from multiple sources, including teacher-completed session records, qualitative interviews and observations of two ZF-SEND lessons in each ZF-SEND school.

• In the ZF-SEND arm, to explore how children, parents and teachers experience the intervention through qualitative interviews with 10–12 pupil participants, 10–12 parents/carers, 5–6 teachers and 5–6 senior staff.

• To investigate the validity and reliability of the self-report measure of mental health (‘Me and My School’; MAMS) and its relationship with other (proxy report) measures of mental health and behaviour.

• To establish what constitutes education as PAU for emotional literacy in special schools for children with a learning disability through a survey of 20 special schools involved in the trial and external to the trial.

• To undertake a nested ‘study within a trial’ (SWAT) to explore the acceptability of two different study designs: one where PAU does not come with the offer of delayed access to ZF-SEND, and one where it does. The aim of this SWAT will be to explore the extent to which offer of a ‘waitlist’ comparator influences recruitment and retention of schools. The findings from this SWAT will be used to inform the design of a subsequent large-scale effectiveness study, if indicated.

• To review the feasibility study against the progression criteria and ascertain whether progression to a large-scale RCT is feasible.

Primary outcomes

The primary outcome is to determine the feasibility of conducting a future large-scale trial to establish the impact of ZF-SEND on mental health, behaviour/emotional/social functioning and quality of life and its cost-effectiveness (economic evaluation). To determine these outcomes, the following were assessed:

• Recruitment of schools, pupils and parents/carers: What are the most effective recruitment pathways to identify special schools? What recruitment rate for parents can be achieved? What are the characteristics of schools and families of children with a learning disability approached and recruited?

• Recruitment of schools and teachers: Can schools and teachers be recruited to run the ZF-SEND programme over 1 academic year? What factors influence schools’ willingness to take part in the research? Can sufficient teachers be recruited and trained?

• Acceptability of research design: Are schools and parents willing to be randomised within the context of a RCT? Do they prefer a design with delayed access (after follow-up data collection) to ZF-SEND in the PAU arm or will they accept PAU with no access to ZF-SEND? How does the offer of delayed access to ZF as part of PAU influence recruitment and retention of schools and pupils?

• Fidelity of implementation: Can teachers deliver ZF-SEND with a high degree of fidelity to the programme manual? What are the key barriers/facilitators for successful implementation of ZF-SEND and how does this vary across different school contexts? Does the implementation of ZF-SEND support the logic model?

• Adherence: What proportion of children with a learning disability in the intervention arm schools complete the ZF-SEND programme?

• Retention: What proportion of schools, children and parents/carers are retained in the research study up to the 12-month post-randomisation follow-up?

• Usual practice: What does PAU consist of for promoting mental health and well-being on a class-wide curriculum basis for children with a learning disability in special schools? How is PAU different from the programme content of ZF-SEND? Does the offer of delayed access to ZF-SEND as part of PAU alter what is offered as part of PAU in the year following randomisation?

• Estimation of parameters to inform a definitive sample size calculation: What are the estimated standard deviation (SD), intracluster correlation coefficient, average cluster size and coefficient of variation of cluster size for the strengths and difficulties questionnaire (SDQ) at 8–12 months post randomisation?

• Feasibility of outcome measures: Do children, teachers and parents complete the outcome measures for the study?

• Design and methods for health economic analysis: What is the feasibility of collecting resource use and health-related quality of life data for parents and the child participants? What sources of unit costs for potential resource consequences are appropriate, and how much primary costing research will be required for a later large-scale trial? What is the most appropriate approach for measuring and valuing child, family and school outcomes for incorporation into a subsequent trial-based economic evaluation?

• Evidence of harm: Is there evidence on the basis of the outcome measures that the ZF-SEND programme results in harm, in which case progression to a full trial would not be recommended.

Secondary outcomes

The SDQ27 total difficulties score as reported by teachers and parents/carers was anticipated as the primary outcome for a future trial. The SDQ total difficulties score includes 20 behavioural and emotional problems items (5 each for hyperactivity, conduct problems, emotional problems, peer problems). The SDQ is a mental health screening questionnaire used extensively in UK child mental health settings and in research. The SDQ has also been used in research with children with a learning disability in the UK and internationally,28–31 and maintains good psychometric properties with this population including associations with psychopathology scores from the Developmental Behaviour Checklist (a measure that has four times the number of items but was developed specifically for children with a learning disability and validated against clinician-rated psychopathology judgements). Other outcomes (likely to be secondary outcome measures in a large-scale RCT) were selected based on experience in research with children with a learning disability, brevity but with good psychometric properties and match to the key domains of the logic model (see Appendix 1). To inform the design of a future trial, secondary analyses compared the outcomes of each of the secondary measurements from baseline to follow-up between the study arms.

Patient and public involvement

The study had a robust programme of patient and public involvement (PPI) built in. The aim was to gain advice from PPI partners (including teachers and parents of children with a learning disability) in the design and delivery of the study. As the study was carried out during the COVID-19 pandemic, PPI advice was essential to help us understand the pressures on teachers and parents and then plan the study processes accordingly.

Partnership for Children

Zippy's Friends for Special Educational Needs and Disabilities is provided by Partnership for Children, a charitable organisation based in the UK that works in schools and internationally, and trains teachers to promote mental health in children. The ZF and ZF-SEND programmes were developed jointly by Partnership for Children and academics and educational resources specialists. ZF has been implemented around the world since 1998, currently in over 30 countries. To date, the programme has been offered to over 1.6 million children.

The Zippy’s Friends programme

Zippy’s Friends is a manual-based, classroom programme that aims to develop children’s emotional literacy through improving children’s repertoire of coping skills and their ability to adapt those coping skills to various situations. ZF consists of six modules: feelings, communication, making and breaking relationships, conflict resolution, dealing with change and loss and coping.

The ZF programme uses a problem-solving (as opposed to a rule-bound) approach and teaches children to develop different ways of dealing with social and emotional problems and to self-evaluate. The manual comprises session plans and activities. Each session begins with a story about a group of children to introduce a number of situations and concepts relevant to emotional literacy. These situations and concepts are then explained and consolidated by means of a range of exercises. By hearing how the children in the ZF stories cope with interpersonal problems and with their emotions, and by roleplays and other activities, children are taught to choose effective coping strategies and deal with real-life situations. There is evidence that the ZF programme has a number of beneficial outcomes relevant to mental health,19–24 quality of life and relationships, and that mainstream pupils generally like to participate in ZF activities. 20

In mainstream schools, teachers and teaching assistants deliver the programme during routine classroom time over a 24-week period with 45-minute weekly sessions (four sessions per module).

The Zippy’s Friends for Special Educational Needs and Disabilities programme

While the mainstream programme is designed for children aged 5–7 years, our pilot study5 indicated that ZF-SEND, which was adapted from the original ZF programme by the charity Partnership for Children (www.partnershipforchildren.org.uk), caters best for an older age range (9–11 years) with the proviso that teachers take a flexible approach and ensure that the programme is age appropriate, yet at the same time considering levels of cognitive and emotional functioning. The ZF-SEND programme closely aligns with the mainstream programme but has additional resources and supplements developed by Partnership for Children in consultation with teachers of children with SEND to cater for children with a wide range of special needs. It provides a selection of alternative activities (approximately five for each of the mainstream activities to cater for varying levels of learning disability) and the stories have been adapted at four different ability levels and using Widgit^® (Widgit Software Ltd, Warwick, UK) symbols. The activities include craft sessions, completion of worksheets, roleplays, discussion and use of metaphors. Completion of ZF-SEND takes longer than the mainstream programme owing to the increased complexity of running the programme with pupils with SEND and to allow for shorter sessions, repetition of sessions and a range of extra activities. Teachers deliver two 45-minute sessions a week to cater for this and to ensure adequate time for completion within 1 academic year. Teachers are asked to send ZF-SEND materials to parents/carers throughout the programme, to be used in the home to reinforce and generalise the principles of ZF-SEND.

In summary, the ZF-SEND programme is an adaptation of the original ZF. The main difference is that the ZF-SEND version has materials and exercises that cater for children with SEND. The content of each of the sessions remains the same as the original but the ZF-SEND programme takes longer to deliver due to shorter sessions, and the need for repetition and more activities to consolidate learning and memory.

Prior to running the ZF-SEND programme, teachers are required to attend a 1-day training course organised by Partnership for Children. Support and supervision via Partnership for Children is available to teachers throughout the six modules.

A logic model was developed (see Appendix 1) to describe the anticipated mechanisms and processes for the ZF-SEND intervention, how this translates into short- and long-term outcomes and the role of other personal, contextual or situational variables.

Programme fidelity and adherence

Trial design and setting

The design is a two-arm cluster feasibility trial of the ZF programme adapted for special schools/units (ZF-SEND), with clear progression criteria and incorporating a process evaluation.

Following recruitment, enrolment of pupils, and collection of baseline data, schools were randomised either to receive training in ZF-SEND and implement the programme plus PAU for 1 academic year or to provide PAU. Partnership for Children trained and supervised teachers delivering ZF-SEND. PAU for emotional literacy in special schools/units for children with a learning disability was established through a survey of special schools/units (including the three PAU schools/units) and a sample of teachers and members of senior leadership/teachers with management responsibilities were interviewed for this purpose.

Researchers blind to allocation assessed school-related well-being by interviewing pupil participants. Teacher- and parent-reported data were collected through self-completed questionnaires. Quantitative outcome data were collected at baseline (prior to randomisation) and at 8–12 months follow-up (8–9 months post randomisation). The statisticians remained blind to allocation prior to analysis. Online randomisation used minimisation with a random element, balanced by size of school. Figure 1 provides an overview of the study timeline.

FIGURE 1.

Overview of study timeline.

Training in ZF-SEND was provided to schools in the ZF-SEND arm after randomisation. As Partnership for Children are the only organisation providing the programme there was minimal risk of contamination between arms. Furthermore, the schools were widely spread geographically (Figure 3) and schools were not in contact with each other, further reducing any risk.

FIGURE 2.

CONSORT diagram of recruitment and data collection process.

Research ethics

Prior to opening the study to recruitment, an application was made to University of Birmingham Research Ethics Committee, who reviewed and approved the study (8 March 2023, application number: ERN_20-0191). The study was conducted in accordance with the University of Birmingham’s code of practice for research. In addition, as the study was undertaken during the COVID-19 pandemic, a risk assessment was undertaken in line with the University of Birmingham’s restarting research process.

Eligibility criteria

The study had three types of participants: pupil participants, parent/carer participants, teacher participants and senior teaching staff participants. In addition, heads of school had to consent to their school taking part in the study. Schools and participants were eligible for the study if they met all of the following inclusion criteria and none of the exclusion criteria applied.

Recruitment

Agreement and consent

Sample size

The target sample size was 12 schools (8 in England and 4 in Scotland) with 6 randomised to each arm of the trial. Based on the pilot study, it was estimated that there would be one class/group per school and an average of eight pupils per class (and therefore eight pupils per school). Recruiting 12 schools would therefore provide a pupil sample size of 96 in total (48 per arm). As this is a feasibility study, a formal, a priori power calculation was not conducted. Instead, this feasibility study aimed to provide estimates of key parameters for a future trial. If two-thirds (66.7%) of schools approached agreed to take part (i.e. 12 of 18 approached), the 95% confidence interval (CI) around the percentage was estimated within ± 21.8% (i.e. 44.9–88.5%). Assuming that 12-month follow-up data were obtained for 75% of children, randomising 96 allowed the 95% CI for retention, to be estimated within ± 8.7%.

Randomisation and masking

For the SWAT, a sampling frame of 50 potentially eligible schools was established. Schools were then allocated at random to information sheets describing a study where PAU either does or does not offer delayed access to ZF-SEND. This randomisation process was carried out by the lead statistician using random permuted blocks with varying block sizes (two and four). Schools were stratified by region (England/Scotland), whether the schools was part of a mainstream school or a standalone special school and school size (< 100 pupils, 100 + pupils).

Following recruitment, enrolment of pupils and baseline data collection, schools were similarly randomised to ZF-SEND or PAU using random permuted blocks. Block sizes were fixed at two, owing to the large number of strata and small overall sample size. Schools were similarly stratified by region, type of school and school size. Randomisation was carried out by a member of staff who was not involved in recruitment or data collection. A member of the research team then informed the schools of their allocation via e-mail.

In the majority of cases, baseline assessments were carried out prior to randomisation, and this was the aim. However, owing to time pressures, randomisation was carried out on some schools before all parent/carer and teacher data were returned. However, the risk of bias was deemed to be low in these circumstances as individual class teachers were not informed of their allocation until after they had completed the baseline assessments and completion of these assessments was not administered by a member of the research team, instead, teachers completed the assessments independently. The same applies for parent/carer assessments.

Pupil assessments were administered by a member of the research team, and these were all completed prior to randomisation. At follow-up, a new member of the research team, who was masked to allocation, administered the pupil assessments and a record was kept of any unmasking.

Withdrawal/changes to participation and loss to follow-up

Experimental intervention: Zippy’s Friends for Special Educational Needs and Disabilities

Those in classes allocated to the intervention arm received the ZF-SEND programme in the 2021–2 academic year. Class teachers were provided with training in ZF-SEND through a 2-hour remote training session, delivered by Partnership for Children in October–November 2021. The session was recorded so that those who were unable to attend could watch the recording. Four teachers from three schools attended the remote session. An additional two teachers from two schools watched the video. These plans were adapted due to the COVID-19 pandemic, as limiting non-essential in-person contact was still preferred by many. The aim was to provide training to at least two teachers from each ZF-SEND class; however, owing to time constraints and capacity of schools to release staff, this was only possible for one class. Follow-up interviews with class teachers in the ZF-SEND arm examined the perceived quality and value of the training. The trainer was also interviewed to explore their experience of delivering the training and the extent to which it was delivered as intended.

Zippy's Friends for Special Educational Needs and Disabilities classes started the intervention between 15 November 2021 and 15 January 2022. It was recommended that schools deliver the ZF-SEND programme at least twice weekly throughout the academic year. However, flexibility was offered in how schools arranged delivery. Schools were encouraged to spend as many lessons as required to work through a session plan as detailed in the manual. However, it was expected that classes would work through the session plans and modules in the specified order, using a variety of activities to explore the topic and provide opportunities for learning. Telephone supervision and support was offered by Partnership for Children on an ad hoc basis and teachers could contact Partnership for Children for advice as required.

Comparator intervention: practice as usual

Classes that were not allocated to receive ZF-SEND received PAU alone. Participants attended their usual classes as well as other services, outside school. No limitations or stipulations were placed on PAU group schools in terms of the emotional literacy initiatives they could implement in the school year. PAU may include any services (mainstream and specialised) provided to families and their children with a learning disability as a part of an education health and care plan in England or equivalent in Scotland. Any schools already delivering a manualised, lesson-based emotional literacy programmes were ineligible to participate. Because Partnership for Children is the only organisation in the UK offering training in the intervention, there was no possibility of contamination within the PAU schools during the study.

Information on PAU in emotional literacy among control schools was collected through three short interviews with one school staff per school: at the end of the winter term (6–15 December 2021) after the spring term (27 April–4 May 2022) and in the follow-up interviews (conducted 27 June–18 July 2022).

Data collection

Data were collected during recruitment, at baseline (pre-randomisation) and throughout the course of the 8- to 12-month follow-up. Baseline quantitative outcome measurement, collecting data from pupils, parents/carers and teachers, was conducted either at the end of the preceding school year (July 2021) or beginning of the new school year (September–October 2021). The same outcome measures were repeated at follow-up towards the end of the 2021–2 academic school year (June–July 2022). The process of follow-up outcome measurement started in June 2022 to allow adequate time for teachers to respond, making allowances for the last few weeks of the school year, which are often off timetable and include special activities and trips.

Measures completed by teachers and parents/carers were self-completed via online or paper questionnaire. Parents/carers were also offered the option to complete via a structured interview over the telephone. Pupil measures were conducted through structured interview by a member of the research team who remained masked to allocation. They were face to face or remote, during school time. Teachers or teaching assistants were available to support the pupil, if required; however, this support was limited to helping the pupil feel comfortable rather than being involved in conducting the assessment. Observational notes were made by the researcher to allow an evaluation of this element of data collection, including the feasibility of conducting remote assessments in this way.

To monitor delivery of the ZF-SEND programme, data were collected throughout the school year through a specially designed session checklist, completed by class teachers. In addition, observations of one ZF-SEND lesson in each ZF-SEND school were made around halfway through the school year.

Qualitative interviews were conducted during the summer term, from April 2022 until the end of the school year. Interviews with teachers from ZF-SEND schools were conducted from the start of the summer term to minimise the competing demands we placed on teachers in these schools as they were also involved in intervention delivery and follow-up outcome assessment. Teachers, members of senior management teams or school staff with managerial roles, parents/carers and pupils from ZF-SEND schools were interviewed about their experience of taking part in the research and, if relevant, the ZF-SEND programme. In addition, a series of interviews with schools involved in the trial as well as other special schools explored PAU in terms of emotional literacy initiatives implemented in schools.

Statistical analysis

Qualitative analysis

Analysis of the follow-up interview transcripts used framework analysis. Framework analysis is a flexible approach which suits the large and diverse data set generated in the present study. It combines a priori and emergent data-driven themes to guide the framework,42 which fits with the feasibility aims of this study. Data analysis was carried out in NVivo (QSR International, Warrington, UK), with collaboration between team members and leaving a clear record of the process from raw data to the analysis.

Framework analysis has five stages: identifying a framework; indexing; charting; mapping and interpretation. 43 In the initial familiarisation stage, we ‘immersed’ ourselves in the data. The data were then sorted into eight initial categories by a senior member of the research team (GU), providing a preliminary framework. These categories were based on the a priori aims of the study, so that specific topics relevant to the feasibility nature of the RCT, such as the implementation of the programme, were explored in detail. As the interviews were semistructured, we ensured all topics relevant to the aims of the study were explored. The process of familiarisation and developing the framework was a ‘to-and-fro process, in which the familiarisation and development of a framework interact with each other’;42 that is, as we kept readjusting our framework, we became more and more familiar with the data.

To familiarise themselves with the data set and refine the framework, two research assistants (EW and NDC) used the initial framework independently to code a subsample of six transcripts and noted any codes that might have been be missing or did not seem to be relevant to the data, thus refining the framework. The wider qualitative analysis team (including the two research assistants, BSK, GU and JS) then met to discuss the initial experience of using the framework and refine the codes and descriptors. Top-level codes were reorganised to improve consistency in analysis. All changes and updates to the framework were recorded.

The two research assistants both recoded one of the interviews using the updated framework. This allowed the team to check that the codes were being interpreted and used consistently. A further meeting was convened with the qualitative analysis team to make the final adjustments to the framework before moving into the ‘indexing’ phase of the analysis in which the framework was systematically applied to code the entire data set. 42 The data set was split in half randomly and the two research assistants used this final framework to code the data. The data were then merged into the same NVivo file once coding was complete.

The qualitative analysis team then met to discuss the coding of the whole data set. The three senior members of the team (BSK, GU and JS) were allocated a top-level theme each, to audit and check the coding before moving forward. The three densest codes were chosen (‘COVID-19-related issues’, ‘general experience of the research study’ and ‘intervention outcomes’). The team met again to discuss this interpretation of the nodes and the data in each. The two research assistants moved on to formulate code summaries. This formed the ‘charting’ stage of framework analysis,43 where the data collected for each ‘top-level’ code were summarised, including convergence or divergence in perspectives across the different stakeholder groups (teacher, parents/carers and pupils). This process led to concise summaries of the key findings for each code. These were then interpreted alongside the quantitative analysis in an interpretative exercise to inform the feasibility evaluation.

Economic evaluation

The health economic component of the study has two main elements: an estimation of the intervention costs of the programme, and an assessment of the feasibility of collecting data on use of health services and health-related quality of life of children with a learning disability for a future trial. The economic evaluation took a service perspective including costs to education, health and social care services.

To estimate the cost of the ZF-SEND intervention, we collected information on the costs of training, subsistence or travel costs for attending training sessions, the costs of materials such as booklets or access to specialist platforms and the cost of supervision/supervisors’ time.

For health-related quality of life, we assessed completion rates and item ‘missingness’ for both pupil completed and parent/carer report versions of generic paediatric quality of life measures CHU-9D44 and EQ-5D-Y-3L. 45 These measures allow for the calculation of QALYs and would be used as the primary outcome measure for cost-effectiveness analysis in a subsequent large-scale RCT. For the feasibility of collecting data on broader health service use, we modified and tested a version of CASUS, which had been used previously in trials involving school children. 46 Following completion by the parent/carer, we assessed the measure’s comprehensiveness in capturing all relevant services, specifically to identify items that were redundant or important services that had been omitted.

Evaluation of training in Zippy’s Friends for Special Educational Needs and Disabilities

A semistructured interview with the ZF-SEND trainer was conducted shortly after training was provided to teachers in the ZF-SEND group to explore adherence to the training model. Teachers who attended the training were also asked about the perceived quality and value of training in the follow-up interviews. In addition, a researcher (NDC) observed the training session and took notes on the content of the training, teacher response and engagement.

Combining the quantitative and qualitative data

The qualitative and quantitative results are presented for each of the three areas under investigation [feasibility and acceptability of (1) participation in the trial; (2) data collection; (3) the ZF-SEND programme] and then combined and discussed. The quantitative results describe what happened during the study and the qualitative data provide insight into the how and why. The results of the interpretive exercise are presented in discussion sections at the end of Chapter 6, pertaining to each of the three primary aims of the study.

Changes to the protocol

The following changes were made to the protocol during the study:

• Extension to baseline data collection period to allow baseline data to be collected at the beginning of the school year rather than completed by the end of preceding academic year. This resulted in the follow-up period being 8–12 months post randomisation rather than 12 months.

• Observation of one ZF-SEND lesson in each ZF-SEND school rather than two.

• The timing of the PAU interviews with control group schools was adapted to move the first interview to the end of the first term of the school year and include an additional interview around halfway through the academic year so that data were collected retrospectively at the end of each of the three school terms.

• Adjustment of data collection procedures to include remote observations of ZF-SEND sessions and remote pupil assessments.

Aims

There is evidence that PPI can have positive impact on research, enhancing the quality of research and ensuring its appropriateness and relevance46 and that it may have a more positive impact when service users are involved throughout the study and when involved as partners in the research team. We therefore ensured that PPI was integral to: (1) the initial stages of the research (identifying and prioritising relevant topics for the research agenda, providing constructive criticism of the research protocol and commenting on whether research was relevant or appropriate to users); (2) undertaking the research [assessing the appropriateness, wording and timing of research instruments (e.g. questionnaires, interview schedules)] and helping to adapt the language and information for the various participant groups. Moreover, assisting with recruitment to the study and improving response rates, (3) writing up the findings (identifying potential knowledge gaps and grounding the final report in user experiences); (4) dissemination and implementation (ensuring that dissemination and implementation take place in a poignant and user-friendly manner and reach the relevant audiences).

Where PPI is described and reported in this report is indicated by page numbers in the Guidance for Reporting Involvement of Patients and the Public 2, short version checklist (Table 2). 47

Methods of involvement

The study management group met monthly throughout the project. Members of the group includes all investigators, all employed project staff and two representatives of the PPI advisory group (nasen and BILD representing children and their parents/carers) to discuss study progression and key management issues. Nasen and BILD were co-applicants of the research study.

The NASEN is a charitable membership organisation that exists to support and champion those working with, and for, children and young people with SEND and learning differences. Rooted in research and evidence-informed best practice, their information, training and resources ensure that a wide range of stakeholders are provided with the most up-to-date knowledge and support required to become effective, inclusive practitioners. Nasen is an influential UK-wide organisation that actively contributes to the SEND debate and effects change to improve provision within early years, schools, post-16 and wider settings and, ultimately, outcomes for children and young people with SEND.

The BILD is a national organisation that aims to develop the skills and culture necessary to understand the needs of children and adults with a learning disability and improve their quality of life. They work with universities, government departments, local authorities, NHS trusts, service providers and mainstream organisations to champion rights for people with a learning disability and improve practice. BILD has over 40 years’ experience of supporting people and families with a learning disability and is concerned with workforce development and providing accessible resources and knowledge. Part of BILD’s role is to be a bridge between the academic community and the direct workforce. BILD is internationally known for the quality of their resources and publications and is in an excellent position to facilitate the dissemination of good practice.

Throughout this project, both nasen and BILD have been enthusiastic about the research and keen to continue to work with us, considering ZF as an excellent way to address the health inequalities of children with a learning disability. Their knowledge and expertise informed us on how best to communicate and work with children with a learning disability, their parents, and their teachers and senior staff. Both organisations were uniquely placed to guide the recruitment, research protocol, training of research staff and more recently they have advised on dissemination of research findings as they understand and value the purpose, aims and application of the project, and as third-sector organisations have the well-being of children with a learning disability and their families at the heart of their mission statements.

In addition to PPI representation during the monthly Senior Management Group meetings (held between March 2020 and December 2022), the chief investigator, study manager and research associates met with the PPI partners on a regular basis (at least quarterly) during the course of the study, more frequently at the start to provide input on all materials and procedures. The PPI partners advised and supported the research team in adapting materials, advising on information sheets, consent forms and other study materials, co-producing dissemination outputs (including BILD currently producing an accessible video for the child participants, thanking them and summarising our research findings), acting as ambassadors for the research project, and creating communication pathways with SEND educational organisations and other stakeholder groups for children with a learning disability and their families. The partners offered strategic advice on recruiting and engaging organisations and participants, and to this day contribute to the interpretation of the study’s findings and their dissemination (to be continued over 2024–5).

In addition, nasen coordinated external focus groups to advise the researchers throughout the study. Nasen held 6-monthly meetings with teaching staff in special schools (three focus group meetings held) and (separately) with parents of children with a learning disability (three focus group meetings held) in order to gauge opinions from Experts by Experience, not directly involved in the study, on a number of topics including acceptability of research procedures and how these may impact on the pupil participants, the role of the teachers and the support they may need to deliver ZF-SEND, and the balance between burden imposed on the participants and schools on the one hand and the advantages gained by them on the other. Feedback from these groups was regularly fed into the study management group agenda so members were kept informed, and then to discuss and evaluate the project’s aims and protocol in the light of the views and opinions of Experts by Experience. Teaching staff and parents/carers were paid for their participation in the focus groups.

The research proposal and initial pilot study were prepared in collaboration with the franchise holders of the ZF programmes, Partnership for Children, an organisation that works in schools and trains teachers to promote mental health in children including children with SEND. This brings them in close contact with the setting in which this research was carried out and as providers of evidence-based interventions, they are on the cutting edge of mental health initiatives for children, including children with a learning disability and/or SEND. Their input, in addition to the PPI provided by nasen and BILD, has enabled us to be aware of the needs of these children, and those of their parents and teachers. Partnership for Children will also be involved in future dissemination activities.

All PPI partners received regular feedback about how their advice has been taken forward to ensure they could see how their contributions shaped the project. Where their suggestions were not possible to incorporate, feedback was provided as to the reasons. Both Partnership for Children, BILD and nasen commented on drafts of the original research proposal, which significantly improved and shaped up our recruitment plans and design of the intervention protocol. They also gave significant input into the final report and are currently supporting us to disseminate our findings to the relevant audiences. They have all been enthusiastic and valued contributors throughout this project and they are keen to continue to work with us.

A study steering committee was established and met biannually during the study (between June 2020 and December 2022). It comprised an independent chair with expertise in SEND research and in trials in the SEND field and other independent members: two clinicians who are expert in the learning disability field, a statistician and a parent/carer of a child with a learning disability. The chief investigator and study manager attended the study steering committee as observers and provided information about the progress of the study when requested by the chair. The committee reviewed conduct of the study, provided overall supervision for the study and advice through its independent chair.

COVID-19-related issues: National Association for Special Educational Needs patient and public involvement groups

The PPI groups gave advice on the impact of the COVID-19 pandemic in schools and at home. The parents provided descriptions of how COVID-19 had impacted the children. All parents described that during ‘lockdown’ periods, their children missed the structure of school and the opportunities for social engagement. They also found home schooling to be a huge challenge. Additionally, parents believed that children benefitted from school during the pandemic when they were allowed to go in. This was due to factors such as smaller classes leading to increased attention, shorter days, more child-led work and less of a negative impact on sensory needs. Some parents also described that they were working full time from home, so schoolwork was unfortunately placed as less of a priority.

The teaching professionals reported that the COVID-19 pandemic had meant that there were more disruptions in the timetable and that time had to be spent reintegrating pupils into the school. They described that there had been an exceptional high workload in the schools due to the pandemic. They observed that this had had a negative impact on staff’s mental health and wellbeing due to relocating staff to areas most depleted, exhaustion and serious adverse events such as a pupil dying due to the COVID-19 virus. To cope with staff absences, the curriculum had been reduced to only literacy and numeracy and a focus on meeting pupils’ immediate needs. They also noted that the impact on staff absence due to the pandemic meant that a whole year’s supply budget had already been used up by the end of the autumn term.

Action taken (outcomes of patient and public involvement)

The PPI partners:

• Provided the researchers with contact details of schools in England that were willing to be approached by the researchers during the recruitment phase.

• Determined/approved wording, layout, font and illustrations of all communication with participants (children, parents/carers, senior teaching staff and teachers). At a later stage in the research, the nasen parents’ focus group discussions expressed some reservations about some of the wording already used in the pupil questionnaires. These concerns will be addressed in the full trial proposal.

• Determined/approved the payment (high street vouchers) to adult participants and rewards (stickers) for the child participants. However, later feedback from the nasen focus groups was that £10 vouchers were not really an incentive for teachers and that they were more likely to be motivated by an interest in the research topic and a desire to contribute to worthwhile research that would be of benefit for their pupils. This feedback will influence the full trial proposal; we intend to focus more on public relations during the recruitment phase (e.g. with nasen’s support, we could provide on-line or in-person events for teachers and parents to advertise the research to potential participants).

• Offered feedback that may inform any potential future trial proposal. Whereas the nasen professionals’ focus group discussions indicated an enthusiasm for taking part in a research trial, such as ZF-SEND, the parents’ focus group appeared to be less convinced. The latter group raised the fear that their children may be used as a ‘guinea pigs’. In future, we propose that the teachers recruited to the project will receive, in addition to their ZF-SEND training, materials for a ‘parents’ information module’, which can be delivered (by teachers and researchers co-presenting) to parents to inform them of the value and the practicalities of being involved in research.

• Provided the researchers with information about the impact of the COVID-19 pandemic on pupil/teacher absenteeism, the workload/stress levels of special school managers and teachers, and on the well-being being and mental health of the children. Specifically, the professionals’ nasen focus group indicated that: (1) high levels of staff absence had had a negative impact on the delivery of the curriculum as a whole (including the ZF-SEND programme); (2) staff absence had also resulted in staff exhaustion; (3) the pupil deaths due to the COVID-19 virus had an impact on staff morale and causing their anxiety levels to reach levels ‘not seen before’; (4) the number of e-mails teachers received and thus their workload increased dramatically during the COVID-19 pandemic. With bubbles and restricted access to people, all communication became online, with e-mails dominating the communication routes. This meant that staff missed vital e-mails due to overload. This information has been considered when (post hoc) discussing the outcome measures used and the pre/post difference scores and has informed the results, feasibility and acceptability of outcome measurement of this report.

• Contributed important information to the debate on when to start recruitment in the academic year, when to collect data and the time period during which to deliver the ZF-SEND intervention. The nasen professionals’ focus group discussions indicate that the optimum time for recruitment is well before the Easter holidays and no later than the early part of the summer term as this is when the teaching staff are planning the curriculum and their teaching priorities for the next academic year.

• When the NIHR call came out in July 2021 to consider expanding existing research projects to include geographical areas that were under-researched, our PPI partners (nasen and BILD) both advised against this due to time constraints – most schools were breaking up that week or the next and it would not be viable to recruit. Nasen would normally be able to stay in touch with senior staff of special schools. However, these networks were not functioning as normal due to the COVID-19 pandemic. It was therefore decided not to apply for this expansion.

• BILD are currently producing, with input from Experts by Experience, a video for the child participants to thank them and to summarise (in an accessible manner) the main findings of the study.

School and participant characteristics

Feasibility and acceptability of participation in the trial

Feasibility and acceptability of outcome measurement

Feasibility and acceptability of the Zippy’s Friends for Special Educational Needs and Disabilities intervention

Practice as usual

Analysis of secondary outcome data

See Appendix 9 for a summary of the scores for the secondary outcomes at baseline and follow-up.

Regression/ANCOVA analyses were carried out on all the outcome measures, including subscale scores, to estimate effect sizes across outcomes and ascertain if there were any signals of differences between the ZF-SEND and PAU groups at follow-up. Table 6 shows the results of these analyses, together with an interpretation of the results. The results indicate that the participants allocated to the ZF-SEND group tended to have better outcomes in terms of behavioural problems and prosocial behaviour and also for the teacher-rated NCBR, apart from adaptive social behaviour. For those allocated to the intervention group, there was a difference in NCBR self-isolated/ritualistic behaviour and irritable behaviour compared with the control group.

Parent-rated scores were limited by low sample sizes and wide CIs.

Review of progression criteria

Avery et al.’s traffic light system26 was used to prespecify progression criteria against which feasibility outcomes were evaluated. Achieving ‘green’ criteria would suggest progression to a large-scale trial is warranted without any amendments, achieving ‘amber’ would suggest progression is only warranted with an amendment to the study design and/or processes, while only achieving ‘red’ would indicate that progression is not warranted. See Table 7 for each criterion and performance against it. A narrative summary of the results is presented following the table and then the results are further discussed in the discussion section in Chapter 7.

Owing to the difficulties recruiting in Scotland (because recruitment started near the end of the school year in Scotland as outlined in the Feasibility and acceptability of participation in the trial), we calculated the ‘recruitment of schools’ metric based on schools in England only. This achieved an amber result. The COVID-19 pandemic is likely to have contributed to the recruitment rate, as well as the study timeline, and the research team have identified a range of modifications to the study that could improve this. All eight schools randomised remained in the study and participated in follow-up data collection achieving a green result. Similarly, the parents/carers of all 54 pupils who were invited to take part provided their consent. Only one pupil was subsequently withdrawn by a teacher.

For the ZF-SEND programme, 48% of sessions that were rated were delivered with fidelity, according to the definition described in Chapter 4. These data were derived from the session checklists that were an incomplete data set so the results should be interpreted with caution. The observations of ZF-SEND lessons suggest that ZF-SEND sessions were delivered with adequate fidelity. Therefore the ‘red’ result on this criterion is more suggestive of the need to improve data collection on delivery of the ZF-SEND programme rather than fidelity.

Pupil engagement in ZF-SEND sessions was rated on the session checklists across pupils, rather than at the individual level so we were unable to calculate the ‘pupil engagement’ metric as intended. Instead, we calculated the proportion of sessions that were rated in which ‘at least some’ of the pupils engaged in at least 50% of the core activities of that session. We have rated this metric as amber and the observational data support this. However, there are issues in assessing pupil engagement in the programme using teacher report. This requires pupils to display behavioural indicators of engagement; however, the ZF-SEND programme does not require pupils to play an active role in the programme – much can be learnt from passive observation. Interpretation of this metric therefore requires caution and is confounded by the aforementioned issues in collecting the session checklist data.

At follow-up, 62% of teacher-rated SDQ measures were returned, achieving an amber result. We have already identified ways to improve response rate, discussed in Chapter 6. We have based this metric on a teacher-rated measure, as this is likely to be the primary outcome for a large-scale trial. The parent/carer response rate was lower, with 38% of parents/carers completing the questionnaire at follow-up. Engagement with parents/carers has been identified as an area for improvement for a large-scale trial and this is discussed in more detail in Chapter 7.

Discussion

This feasibility, cluster RCT aimed to evaluate whether it was possible to conduct a large-scale RCT of the ZF-SEND programme for children with a learning disability in special schools. A total of 8 schools and 53 pupil participants were recruited to the study and followed up over the 2021–2 academic year. Emotional literacy, behaviour, health and service use were measured at baseline (before the start of the intervention) and at follow-up (8–12 months post randomisation), collecting data from pupils, parents/carers and teachers.

The eight schools were randomised to provide PAU or the ZF-SEND programme. Three schools were randomised to the PAU group and five to the ZF-SEND group. Teachers in the ZF-SEND group were trained in ZF-SEND through a 2-hour online training session (either joining remotely or watching the recorded session) and provided with supervision throughout the study period from Partnership for Children. Interview and observational data were collected to evaluate the training provided.

Towards the end of the academic year (from May 2022), follow-up interviews were conducted with eight pupils from four of the ZF-SEND schools, four parents/carers (two from PAU and two from ZF-SEND schools), seven class teachers or those responsible for delivering the ZF-SEND programme (three from PAU and four from ZF-SEND schools) and four members of senior leadership or teachers with management/oversight roles (two from PAU and two from ZF-SEND schools). To assess ZF-SEND delivery, teachers in the ZF-SEND group completed session checklists and had one ZF-SEND lesson observed by the researchers. To gather information on PAU in the control group, members of staff from the control groups were interviewed three times, towards the end of each academic term about the emotional literacy initiatives in their school. Additionally, staff from other special schools (not involved in the study) were also interviewed to gain insight into PAU with respect to emotionally literacy in special schools.

The quantitative and qualitative data were brought together to assess the feasibility and acceptability of: (1) participation in the trial; (2) data collection; and (3) the ZF-SEND intervention. The predetermined progression criteria were reviewed to further inform whether progression to a large-scale trial was warranted.

The study started in April 2021; 50 schools were invited to take part in the trial, 39 in England and 11 in Scotland. Eight schools were recruited, all from England. This is likely due to the start of the study being delayed due to the COVID-19 pandemic combined with schools in Scotland closing for the summer break earlier than ones in England. The recruitment rate of schools in England was therefore 20.5%, which is an amber result according to our predefined progression criteria (see Chapter 6). The UK was just emerging form a national lockdown when the study started and schools were under immense pressure to resettle pupils and implement a host of COVID-19-related procedures. We are therefore confident that the recruitment rate for schools could be improved through modifications to the study design. A future study should open for recruitment earlier in the preceding academic year, for example in January, to allow adequate time for initial discussions, selection of classes and baseline data collection before the end of the academic year. This would facilitate other aspects of the study, as described later. Furthermore, schools in England were identified through nasen and this proved to be a useful route for recruitment that could be extended to Scotland, Wales and Northern Ireland, as they are a UK-wide charity.

All randomised schools remained in the study showing high levels of retention and achieving a green result according to our predefined progression criteria. Moreover, the qualitative interview data and 100% (of those invited to take part) rate of pupil recruitment, indicate that the trial was acceptable to pupils, parents/carers and teachers. Any issues with recruitment of schools may therefore be due to factors external to the study rather than reflecting schools’ willingness to participate. Some teachers had concerns about the lack of clarity on randomisation and therefore the risk of being allocated to the control group. Communication with schools and teachers could therefore be strengthened to make this clearer. However, the SWAT indicated that the offer of the ZF-SEND programme at the end of the study, if allocated to the control group, did not result in increased recruitment rates so the RCT design itself may not deter schools from participating and access to the ZF-SEND intervention was not the only motivating factor for schools. Indeed, the qualitative data suggest that schools are supportive of this kind of research, acknowledge the importance of emotional literacy and are invested in improving access to evidence-based interventions for their pupils.

We anticipated recruiting eight pupils per school; however, a mean of 6.7 pupils were recruited per school. We also aimed to recruit 96 pupils across 12 schools: 54 pupils were recruited. Pupil eligibility criteria stipulated an age range of 9–11 years, or those in years 5–6. A minority of pupils who were recruited fell outside of this range and there was nothing to indicate that they were less suitable for the research. Future research could therefore broaden the age-related eligibility criteria to include 8- to 12-year-olds. This would facilitate delivery in special schools in which classes are not exclusively age-based, instead, classes may be based on abilities and may cross over from primary to secondary. This may allow more pupils to be recruited per school and therefore improve the feasibility of meeting sample size targets.

A key aspect of the study design was to collect data directly from pupils rather than rely solely on proxy report. This is an important step for research involving children with a learning disability, which rarely gives voice to those participants. High response rates were achieved to the researcher-administered pupil assessment and pupils generally enjoyed the process, suggesting that pupil assessment, both in-person and remotely were both acceptable and feasible. To achieve this, one of the eligibility criteria stipulated that pupils needed the cognitive and communication skills to agree to the pupil assessment. This may have excluded some pupils from the trial who may have otherwise been eligible. To improve access to the trial and recruitment rates of pupils per school, future research could seek to collect data from pupils where possible, but not use this as a criterion for eligibility. For example, a simple screening instrument with appropriate cut-off scores could determine which pupils to involve in pupil-rated measures. This would provide a more robust and systematic measure for suitability for pupil assessment.

The sample of pupil participants in this trial had a range of conditions, primarily ASD, followed by moderate learning disabilities. The study was designed to evaluate ZF-SEND for children with a learning disability; however, there was no formal assessment of this, so it is possible that some participants did not have a learning disability. Future research could use a measure of a learning disability or adaptive behaviour with thresholds to reflect presence/absence/severity of a learning disability. A further threshold could then be used to ascertain whether the pupil could participate in the aforementioned pupil assessments. Future research could then include both those with and without a learning disability (given that the ZF-SEND programme is usually provided to whole classes in special schools) and then undertake subgroup analyses on those with/out a learning disability. Similarly, only pupils with the capacity and communication skills to engage in the intervention were recruited, based on a subjective assessment by the teacher. A more formal evaluation would ensure that sampling was consistent across schools and would also allow analysis according to severity of a learning disability, which would ascertain which pupils might benefit more from the ZF-SEND programme.

Acceptable return rates were achieved for the teacher-rated measures with SDQ data available at follow-up on 62% of pupils included in the trial. This achieved an amber result against our progression criteria. The qualitative data indicate that teachers, parents and pupils found the process of completing outcome measures acceptable. However, it is important to note that the COVID-19 pandemic and subsequent national restrictions and changing circumstances for children and their parents/carers may have impacted in unanticipated ways on return rates. A number of studies48 and a recent scoping review49 suggest that pre-existing inequalities for children with SEND and their families became greater during the pandemic and affected parents/carers’ stress levels and mental health. These negative consequences for parents/carers as a result of the COVID-19 pandemic are likely to have had an adverse impact on their motivation and capacity to complete and return research questionnaires. Nevertheless, future research should seek to maximise response rates, especially of parents/carers.

As this was a feasibility study, with little previous research to inform the selection of outcome measures, a wide range of measures were included in the teacher and parent/carer questionnaires resulting in a rather long and, at times, repetitive process of data collection. Based on the results of this study, future research could reduce the number of measures and therefore shorten the length taken to collect data, which may improve the return rates at both baseline and follow-up.

Schools reported a lack of engagement from parents/carers. This was also a likely factor in the response rate of parents/carers to the questionnaires (38% of questionnaires containing all the parent-rated measures were returned at follow-up). We used opt out parental consent for pupil participants. Therefore, all communication with parents/carers was through schools and the research team did not have access to parent/carer contact details (except for those agreeing to be contacted for interview). Future research could use a combination of both opt-out (to pupil’s participation in the trial) and opt-in consent (to parents/carers completing the measures), which would allow the research team access to parent/carer contact details by requesting contact details on consent forms. Further engagement activities could include webinars and events to inform parents/carers about the study, its aims and how they will be involved. Starting the study earlier in the preceding academic year would provide adequate time for this.

In the schools allocated to deliver ZF-SEND, feasibility and acceptability of the intervention were assessed in a number of ways: through the session checklists, observation of ZF-SEND lessons, in the follow-up interviews and through evaluation of the training provided. We measured engagement, adherence and fidelity to the intervention through data collected on the teacher-completed session checklists so that quantitative measures of each could be provided and then assessed against the predetermined progression criteria. Pupil engagement for each session was assessed against three core elements of the session plan. We determined that at least some of the pupils present for the session should be engaged in at least 50% of the core elements of that session. High levels of engagement were recorded (94% of sessions rated). However, we acknowledge the challenges of rating pupil engagement in a programme like ZF-SEND, especially children with a learning disability who may not always show stereotypical markers of participation and engagement. For example, a pupil may choose to passively observe a session rather than participate in a roleplay but may still acquire the key learning from the roleplay. This is further complicated by the incomplete session checklists and reliance on teacher report. However, the observational and interview data provide evidence that pupils are actively engaged in ZF-SEND and found the lessons enjoyable and interesting.

We calculated adherence based on attendance of pupils and schools’ progress through the programme. Measurement of attendance was hampered by the incomplete session checklists; however, all pupils enrolled in the study in the four schools that started the programme, attended at least one ZF-SEND session. Rates of attendance varied and the COVID-19 pandemic was a factor in this. Progress through the programme also varied and only one school completed the programme within the academic year. However, schools started the programme later than anticipated as baseline data collection and therefore randomisation was pushed into the first half term of the year. Therefore, schools could not start the programme until after the autumn half term. This coincided with an increase in COVID-19 rates and high levels of pupil and teacher absence, so some schools did not state the programme until the beginning of 2022. Bringing forward recruitment, baseline data collection randomisation and a first instalment of training in ZF-SEND to before the end of the preceding academic year would allow schools allocated to ZF-SEND to start the programme immediately after settling into the new academic year. Based on schools’ progress through the programme in this study, once they got started, this would give adequate time to complete the programme in 1 academic year. Having more time for training would also allow us to provide more training sessions and train more teachers and teaching assistants in each school who could, in the event of class teacher absence, provide cover and therefore continue the programme. A further benefit of this is that we would achieve a full 12-month follow-up and baseline and follow-up measures would be taken at the same point in the school year. Furthermore, randomising schools early in the summer term would allow those in the ZF-SEND arm time to build the programme into their lesson planning from September.

As mentioned, the session checklists were partially completed. We know that some schools did not return checklists for all the sessions they provided. Similarly, analysis of the data indicates that teachers did not fully record the information requested making interpretation of the data difficult. This is especially relevant to the measure of fidelity: only 48% of sessions were rated with fidelity to the programme. The main issue was lack of recording of two different activities for each session. However, observational and interview data suggest that the programme was delivered with good fidelity, so this score is likely to reflect an issue with reporting rather than delivery. Future research should therefore improve data collection through the session checklists. We have liaised with Partnership for Children who oversee the ZF-SEND intervention and they have agreed to embed the checklists into the programme resources, as access to the programme is moved online. Each checklist will need to be completed before teachers access the next module. The checklists will therefore become part of the intervention, but will also serve research purposes, rather than as an additional task for teachers. Online completion and submission of the checklists will also allow us to monitor completion and chase up on missing data.

Furthermore, Partnership for Children has agreed to modify the programme of training. As mentioned earlier, we will provide a first, introductory training session before the end of the preceding academic year. This would equip teachers with the knowledge of the programme and materials to begin delivery after the summer break. This will incorporate the 2-hour online session provided as part of this study. A second training session will be scheduled for early in the academic year, after giving time for teachers to start the programme and deliver a few sessions. In this second training session, implementation issues will be discussed, and teachers will be asked and reminded about completion of the session checklists. The home activities and involvement of parents will also be discussed to ensure teachers are providing parents with the home activities, which was reported to be lacking in the present study.

Data from pupils, parents/carers and teachers suggest that the ZF-SEND programme is acceptable and feasible and addresses issues that are important for children with a learning disability. It is a relatively low-cost intervention, especially with the move to online resources and training. However, the costs of teachers’ time to deliver the programme should also be included in future costing models. The intervention can be adapted and delivered flexibly to meet the needs of pupils and teachers especially appreciated the comprehensive resources that, unlike other programmes, do not require any further adaptation for children with a learning disability. We are therefore confident that, with the modifications to the study design as previously discussed, the study, data collection and ZF-SEND programme are acceptable and feasible to stakeholders.

Practice as usual in the control group and in a wider sample of special schools was assessed. A wide range of emotional literacy initiatives were used in special schools, ranging from ‘feelings check ins’ during the day, visual displays relating to emotions, and the Zones of Regulation and behavioural support plans. However, no schools were using a manualised, modular classroom-based programme specifically designed for children in special schools.

Patient and public involvement

Public involvement was a key aspect of the study throughout. We sought advice from teachers and parents during study set up, data collection and analysis. Their advice helped shape the study procedures and our understanding of contextual issues, especially in identifying issues relating to the COVID-19 pandemic and their impact on the study. A representative from nasen, who sat on the study management group, was the key contact for the PPI groups, taking matter to the groups and then feeding back at study management group meetings. This process worked well and should be retained in future research.

Equality, diversity and inclusion

Impact and learning

This feasibility study has highlighted a number of modifications that could be made to enhance the design of a large-scale RCT such as starting recruitment earlier in the preceding academic year, broadening the inclusion criteria for pupils, improving engagement and communication with parents/carers, improving the way that ZF-SEND session are rated, and reducing the number of behavioural outcome measures for parents/carers and teachers. A large-scale trial would provide much-needed evidence on effective emotional literacy interventions for children with a learning disability. This study has also generated impact for the participants in the study. The teacher reported a range of positive impacts on the children who took part in the ZF-SEND programme. Schools allocated to the PAU group were also provided with the programme at the end of the study so 12 schools have been given access to the programme as part of the study. In the interviews, teachers commented that they would continue to use the programme in their schools, potentially giving access to well over the 53 pupils involved in this study.

Another aspect of key learning from this study is the collection of data from pupils using both in-person and remote techniques. Both modes were acceptable and feasible and the MAMS has good face validity. Further work is required to validate this measure for children with a learning disability, as the study did not replicate the factor structure of the MAMS; however, we are keen to highlight the importance of self-rating in research for children with a learning disability.

Implications for decision makers

This feasibility study does not warrant us to identify implications for practice or local service delivery. Once a full trial has been conducted, we will be in a position to set out implications for decision makers in the context of the evidence. The current study was not designed to produce such evidence. We are therefore keen to prepare a proposal for a large-scale RCT which can do this, as research that can produce evidence to support clear and achievable action points is more likely to encourage uptake.

Recommendations for future research

As this is a feasibility RCT, the primary recommendation for future research, based on the outcome of this study is to conduct a large-scale effectiveness trial of ZF-SEND for children with a learning disability in special schools. Currently, there are no evidence-based, emotional literacy interventions for children with a learning disability, despite emotional literacy being a proven moderator of a range of negative health and social conditions and children with a learning disability being at higher risk of such negative health and social conditions. Providing evidence of effectiveness of an intervention would enable special schools to make a case to purchase and provide the programme within their school and give their pupils the opportunity to develop their emotional literacy. In addition, research on how to improve parent/carer engagement in both school-based interventions and research is warranted.

CRediT contribution statement

Biza Stenfert Kroese (https://orcid.org/0000-0002-3027-7954): Conceptualisation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Supervision, Validation, Writing – original draft, Writing – editing and reviewing.

Gemma Unwin (https://orcid.org/0000-0002-5205-3150): Conceptualisation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualisation, Writing – original draft, Writing – editing and reviewing.

Richard Hastings (https://orcid.org/0000-0002-0495-8270): Conceptualisation, Funding acquisition, Methodology, Resources, Writing – editing and reviewing.

Andrew Jahoda (https://orcid.org/0000-0002-3985-6098): Conceptualisation, Funding acquisition, Methodology.

Rachel McNamara (https://orcid.org/0000-0002-7280-1611): Conceptualisation, Data curation, Funding acquisition, Methodology, Supervision, Writing – editing and reviewing.

David Gillespie (https://orcid.org/0000-0002-6934-2928): Conceptualisation, Data curation, Formal analysis, Funding acquisition, Methodology, Resources, Software, Supervision, Validation, Visualisation, Writing – editing and reviewing.

Jeremy Segrott (https://orcid.org/0000-0001-6215-0870): Conceptualisation, Formal analysis, Methodology, Resources, Supervision, Validation, Writing – editing and reviewing.

Kate Ingarfield (https://orcid.org/0000-0002-6910-0267): Data curation, Formal analysis, Writing – editing and reviewing.

Myrsini Gianatsi (https://orcid.org/0000-0003-3664-4302): Data curation, Formal analysis, Writing – editing and reviewing.

Elizabeth Randell (https://orcid.org/0000-0002-1606-3175): Conceptualisation, Data curation, Funding acquisition, Methodology, Writing – editing and reviewing.

Zoe Mather (https://orcid.org/0009-0009-8213-7855): Methodology, Writing – editing and reviewing.

Barbara Barrett (https://orcid.org/0000-0002-0270-6256): Conceptualisation, Formal analysis, Funding acquisition, Methodology, Resources, Supervision.

Poushali Ganguli (https://doi.org/0000-0002-1764-5273): Conceptualisation, Formal analysis, Methodology, Resources.

John Rose (https://orcid.org/0000-0003-2672-9767): Conceptualisation, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision.

Mariam Sahle (https://orcid.org/0009-0006-2533-2222): Investigation, Project administration, Resources, Visualisation.

Emily Warren (https://orcid.org/0009-0007-3242-0535): Formal analysis, Investigation, Project administration.

Nathan Da Cruz (https://orcid.org/0009-0006-6106-6240): Formal analysis, Investigation.

Claire Wilson: Conceptualisation, Methodology. Sarah Leitch: Conceptualisation, Funding acquisition.

Acknowledgements

This study would not have been possible without the hard work and dedication of the teachers who took part. The team expresses their sincere gratitude to these teachers who showed impressive commitment, despite challenges during the study. Our thanks also go to members of senior leadership in the schools for agreeing to take part in research at a time of such uncertainty. Similarly, to the parents and carers who completed questionnaires and took part in interviews – we thank you for your time and energy in making this study a success. Finally, to the pupils who took part, thank you for speaking with us, we are proud to have included your voice in this study.

Data-sharing statement

All data requests should be submitted to the corresponding author for consideration. Access to anonymised data may be granted following review.

Ethics statement

Ethical approval was granted by the University of Birmingham (UK) Ethics Committee on 8 March 2021 (ERN_20-0191).

Information governance statement

University of Birmingham is committed to handling all personal information in line with the UK Data Protection Act (2018) and the General Data Protection Regulation (EU GDPR) 2016/679. Under the Data Protection legislation, University of Birmingham is the Data Controller, and you can find out more about how we handle personal data, including how to exercise your individual rights and the contact details for our Data Protection Officer here (https://www.birmingham.ac.uk/university/leadership/governance/policies-regs/data-protection.aspx). The Data Processor was the Centre for Trials Research at Cardiff University.

Disclosure of interests

Full disclosure of interests: Completed ICMJE forms for all authors, including all related interests, are available in the toolkit on the NIHR Journals Library report publication page at https://doi.org/10.3310/JTJY8001.

Primary conflicts of interest: The following authors held the following NIHR Committee memberships: David Gillespie: HTA Associate Board 2020; HTA Commissioning Committee 2021–5, John Rose: HTA MPOH Panel 2015–18; HTA Prioritisation Committee 2015–19.

Disclaimers

This article presents independent research funded by the National Institute for Health and Care Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the PHR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the PHR programme or the Department of Health and Social Care.

Factors affecting delivery of Zippy’s Friends for Special Educational Needs and Disabilities programme

Seven teachers or senior leads mentioned how COVID-19 has impacted the delivery of ZF-SEND. The staff split in their opinion, with two teachers (schools 1 and 11) describing that the pandemic has had minimal impact due to online communication methods and staffing issues only causing disruption for no more than a week. However, the other schools (schools 5, 10 and 9) did stress that COVID-19 had had a profound impact on how ZF-SEND could be delivered. These schools described how staff absences caused large disruptions, preventing the programme being delivered on a regular basis. The teacher from school 9 also mentioned that children often missed sessions of ZF-SEND because of being absent due to COVID-19.

Factors affecting delivery of other newly introduced programmes

Four teachers or senior leads talked about COVID-19 potentially impacting the delivery of new programmes or initiatives. All of them stated that COVID-19 has not had a serious impact on any new initiatives that year. However, three of these interviewees (schools 2, 1 and 4) explained that one of the reasons for this was because no new initiatives were being put in place, due to the schools using this year to adapt to changes necessary due to the pandemic. One teacher also mentioned that their school had to cope with the post-pandemic impacts on some of the children including the worsening of emotional dysregulation.

Impact of COVID-19 on schools

Nine teachers or senior leads discussed the impact that COVID-19 has had on schools in the past year. Staffing issues were commonly mentioned as the largest issue, but many teachers and senior leads believed that they were able to cope because there were adaptations in place to deal with difficulties that could arise due to COVID-19. Most staff however also agreed that COVID-19 had caused a significant change in schools compared to pre-COVID-19 years, with measures such as social restrictions (in assemblies or meals) and bubbles still in practice. Three teachers (teachers 5 and 9, school 4) also suggested that the impact of COVID-19 was as significant, if not more significant, in that academic year compared with previous years since the onset of the pandemic. For example, the teacher from school 4 believed that children and families coped well during the pandemic but the biggest issues are being seen now due to children struggling to adapt to returning to the school and community.

Some of the teachers mentioned that COVID-19 had a worse impact on special schools because of the extra social, health, emotional and psychological needs which children with SEND experience. Furthermore, it was described that children with SEND are less able to deal with change or loss (senior lead, school 10) and that they are less able to learn from bubble or remote learning (teacher, school 5). Staff from schools 1 and 10 also reported that the disruption caused by COVID-19 has meant that there has been less focus on non-core subjects such as art or history. Schools instead have prioritised what they perceive to be essential teaching of mathematics and English.

Impact of COVID-19 on the research experience

Most teachers who talked about COVID-19’s impact on the research reported this to be minor, especially in the schools in the PAU group. The school 1 senior lead and school 6 teacher mentioned the use of technology such as video calls and the remote observation as useful tools in enabling the school to participate in the research. However, COVID-19 was mentioned to have a negative impact on the ability to participate in the research by the school 9 teacher; she mentioned that because of staff and pupil absences caused by COVID-19, the data collection had been delayed.

Pupil absence

Nine staff members described how COVID-19 has impacted pupil attendance. All reported that COVID-19 had significantly reduced pupil attendance that year compared to the years before the pandemic. The school 1 senior lead mentioned that it was the lowest attendance they had ever recorded. The school 10 senior lead on the other hand believed that while large student absences had caused partial closures of the school, the absences had not caused as many disruptions as in the previous (pandemic) year. Teachers from schools 2 and 6 noted that when pupil absences occurred, programmes such as ZF-SEND would continue as normal and that students who were absent would miss sessions.

Staff absence

All staff members stated that COVID-19 had caused high rates of staff absence and that there had been significantly more absences compared to previous years. The school 4 senior lead and school 2 teacher reported that while staff absences due to COVID-19 had caused issues, their schools had been able to cope relatively well because staff absences had not been long term, and they were able to still cover the curriculum. However, the rest of the schools found it difficult to cope with staffing issues because of the unpredictability of their occurrence and staff contracting the virus multiple times. Three schools (schools 10, 11 and 6) also described that staff absences due to COVID-19 disrupted the delivery of programmes such as ZF-SEND because other teachers could not fill in because they had not been trained in the programme. This resulted in the programmes stopping and starting on multiple occasions.