Interventions to improve mental health and well-being in care-experienced children and young people aged less than 25: the CHIMES systematic review

CHIMES collaboration partnership: establishing review need and relevance

The review was a formal collaboration between Cardiff University, University of Bangor, University of Exeter and the Fostering Network in Wales. Our collaboration initially addressed stage one of the TRANSFER framework,72 which is to establish the need for a review. Meetings between the review team and the Fostering Network in Wales (2018–9) identified a paucity of evidence-based interventions supporting the mental health and well-being of care-experienced children and young people in Wales, despite an evident need. The application to the National Institute for Health and Care Research (NIHR) Public Health Research (PHR) funding panel for the CHIMES review was a product of these initial meetings.

Project advisory group

The review was overseen by a project advisory group, which comprised two academics, two policy and practice professionals and two foster carers. The advisory group convened at four key time points during the study:

• On completion of the initial mapping of the research evidence to confirm the parameters of the review (month 6).

• On completion of the process evaluation synthesis to reflect on the findings and undertake an initial exploration of how they might support the interpretation of the outcome evaluation synthesis (month 12).

• To consider the content and structure of final stakeholder consultations (month 22).

• To reflect on the review and explore opportunities for future intervention research (month 28).

Stakeholder consultations

We conducted stakeholder consultations in two phases. The first phase, undertaken within the first 12 months of the review, was to refine the review parameters after the evidence-mapping stage. The first phase included identifying key context factors in the UK social care system that should be prioritised in the conduct of the review, which is prescribed by stages 2 and 3 of the TRANSFER model. 72 The second phase, undertaken in the last 6 months of the review, was to interpret and reflect upon the review findings (WP5).

A summary of the three consultations undertaken during phase one is presented in Table 1. The first consultation was conducted with CASCADE Voices, a young people’s advisory research group comprising care-experienced individuals up to the age of 25 years. As the group was facilitated by a third-party organisation, we did not have specific details on the age or care history of participating members. Key discussion points confirmed that the synthesis needed to focus on the priority outcomes of well-being and suicide-related outcomes. It further identified key context factors in the UK that should be attended to and foregrounded as part of the process evaluation synthesis, notably around system identities and resources.

A further two consultations were planned at this time but, as a consequence of the COVID-19 pandemic, these consultations were delayed until January–February 2021. One consultation was hosted with the Fostering Network in Wales Young Person Forum, which is a group of care-experienced young people who provide advice and guidance to the charity on their programmes of work. A second consultation was conducted with the All-Wales Fostering Team Managers Forum, which is also facilitated by the Fostering Network in Wales. The forum comprises a range of local authority and independent foster-care providers working in Wales. While these consultations were hosted as the review was in progress, they did help to confirm the scope and focus. They also extended and refined the key context factors to be explored.

It should be noted that, in our consultation with the Young Person Forum, young people queried why the review was considering the transportability of interventions across contexts, questioning why interventions were not being developed to meet the specific needs of care-experienced children and young people in the UK.

Convergent synthesis design

The review adopted a convergent synthesis design. 73,74 This approach entailed data from method-specific WPs being extracted, analysed and synthesised in a complementary manner, before being harmonised and integrated into an overarching review-level synthesis. RQs were designed to be complementary and contingent, where achieving a comprehensive answer to one question was dependent on the answers to other questions. Furthermore, when conducting the review, we (1) had the same members of the research team work on both the synthesis of qualitative and quantitative data; (2) screened all study types simultaneously and by the same members of the research team, storing data in EPPI-Reviewer version 4 (EPPI Centre, Social Science Research Unit, UCL Institute of Education, London, UK) to ensure ease of sharing of study data across syntheses and (3) used method specific appraisal tools that have been combined in previous reviews due to providing epistemological flexibility or consonance.

Eligibility criteria

The inclusion parameters for the review are reported in accordance with the PICOS (population, intervention, comparison, outcomes and study) framework.

Information sources

We identified study reports from five information sources: (1) electronic bibliographic databases; (2) websites; (3) expert recommendations; (4) screening of relevant systematic reviews and (5) citation tracking of included study reports.

Search strategy

For bibliographic databases, we developed and tested a search strategy in OVID MEDLINE (see Appendix 2). It was adapted to the functionality of each bibliographic database. Search terms were clustered around the areas of: children; social care; mental health; well-being; study design. Where appropriate, subject headings were included in the search strategy. Searches were limited in date from 1990 to coincide with the ratification of the United Nations Convention on the Rights of the Child,75 which prescribes comprehensive social and healthcare provision for children internationally and started the proliferation of intervention in this area. No further limits were used within the search.

For websites, the search strategy depended on the functionality of the platform. Search terms focused on children and young people in care, mental health and well-being. These terms were searched through the website search function or Google advanced search, and in the absence of this functionality website pages and publication lists were screened for relevant study reports.

Selection process

We uploaded retrieved study reports to the EPPI-Centre’s specialist online review software EPPI-Reviewer version 4.0 for storage and management. The software stores the bibliographic details of each study report, including the abstract. For citations that progress to full text screening, the software enables the upload of related electronic documents.

We conducted screening of retrieved study reports in three stages. First, retrievals from electronic bibliographic databases and websites were screened to identify clearly irrelevant retrievals by checking the record title (e.g. animal testing of pharmacological treatment). To note, while the search strategy was designed for specificity and sensitivity, it did retrieve a large evidence base on older people’s social care. This stage was conducted by one member of the review team. Retrievals identified as clearly irrelevant were checked by a second reviewer. Where there was a conflict in decisions, the study report was marked as clearly relevant and progressed to the next stage of screening. Study reports identified through the other additional information sources were not assessed for relevance.

Second, we screened the title and abstracts of retrievals from almost all information sources independently and in duplicate. Where there was a conflict on exclusion, the study report progressed to the next stage of screening. At this stage, there was a 5% rate of conflict in decision-making. Expert recommendations were not screened at this stage as responses provided study reports specific to the review remit and generally needed consideration at full text in the first instance.

Third, we screened the full text of study reports from all information sources independently and in duplicate. Where there was a conflict, a decision was made through recourse to a third member of the research team. At this stage, there was a 13% rate of conflict, reflecting some of the complexities in deciding if the population (e.g. care-experienced or children in need) or outcome (e.g. self-esteem) was eligible for the review.

An inclusion criteria proforma guided the selection process (see Report Supplementary Material 1). The review protocol specified that the proforma would be tested and calibrated by two reviewers screening the title and abstracts of the same 50 references. Owing to the size and brevity of the available literature, we increased this to 117 references, which was more than 1% of the retrievals from the electronic bibliographic database and website searches. Three members of the review team test screened the sample, with each retrieval being screened by two reviewers. There was a 10% conflict rate. Discussion among the review team indicated that the inclusion criteria were clear, but that we needed to ensure familiarity with the agreed criteria. For example, this included the countries (e.g. higher income countries) that were eligible for the review. The inclusion criteria proforma was regularly reviewed, with any clarifications reported in an update to the review protocol.

Economic evaluation searches and study identification

While we conducted the aforementioned searches and selection, we progressed with searches for economic evaluations by unpicking a recent relevant review of economic evaluations of children and young people’s social care interventions conducted by authors of the CHIMES review. 76

The 20 study reports included in the economic evaluation review were assessed against the CHIMES review’s inclusion criteria. We screened titles and abstracts independently and in duplicate. Eighteen progressed to screening at full text. None were assessed to be eligible for the present review. Reasons for exclusion were: wrong population (n = 14); wrong outcome (n = 3); evaluation report could no longer be accessed (n = 1).

The original review had run searches until 2018. Using the review’s search strategy, we reran searches from 2018 to May 2020. This retrieved 3411 additional citations. Following de-duplication, we screened 1636 retrievals at title and abstract, with 42 progressing to full text screening. No study reports were identified as eligible. Reasons for exclusion were: wrong publication type (n = 4); wrong outcome (n = 3); wrong population (n = 33). Two study reports could not be accessed. Economic evaluations were also searched for in the main CHIMES review searches.

Evidence map, relationship between study designs and method-level syntheses

Following the identification of eligible study reports, we constructed an evidence map. From here, we assessed which study reports would be included in method specific syntheses.

To be included in the description of programme theories, interventions had to have an associated outcome evaluation.

For process evaluations, we constructed a classification which identified evaluations as either ‘conceptually and/or empirically thin’ or ‘conceptually and/or empirically rich’. Thin process evaluations had to have an associated outcome evaluation to be included. They often formed part of a mixed-method study report  anddid not have a dedicated description of method. They also lacked transferrable data or interpretations that could help to understand the context of intervention implementation and acceptability more broadly. Rich process evaluations were included as stand-alone study reports in the process synthesis, regardless of whether there was a linked included outcome evaluation, as they provided potentially generalisable contextual insight into how interventions might interact with complex system characteristics.

To support classification of thin and rich process evaluations, we drew upon an existing review’s classification system for the sampling of qualitative research to develop an assessment tool77,78 (Table 2).

Owing to the scale of the review, we made an a priori decision that we would only include interventions targeting the intrapersonal and interpersonal level in the outcome synthesis if they were evaluated with a RCT study design. For interventions operating at the organisational, community and policy level (which were identified as priority areas but are typically less amenable to RCT study designs), we included all eligible evaluation study designs (e.g. RCT and non-RCT) in the outcome synthesis.

Economic evaluations did not have to have an associated outcome evaluation.

Data extraction and data items

Quality assessment

Data synthesis

Mapping of evidence, intervention characteristics and theories

We used scoping review methods and systematic mapping guidance to support the mapping of the evidence base86–88 (Figure 1). Following the coding of study reports, we constructed numerical and narrative summaries of evidence clusters and gaps, accompanied by descriptive tables and infographics. For details on intervention characteristics, a narrative summary described the interventions in detail, with an accompanying table presenting a summary of extractable data according to the TIDieR framework. For the subset of interventions reporting on intervention theory, these were narratively summarised according to the socioecological domains in which the theories operated and accompanied by a summary table.

FIGURE 1.

Results-based convergent synthesis design.

Assessment of certainty

For the assessment of the certainty of the evidence base, we used the Grading of Recommendations Assessment, Development and Evaluation (GRADE)94,95 and GRADE Confidence in the Evidence from Reviews of Qualitative research (GRADE-CERQual)96 tools. This supported the convergent synthesis design and maps to stage 6 of the TRANSFER framework. 72

We applied the GRADE tool to evidence of effectiveness from randomised and non-RCTs. 95,97 Certainty was assessed for short- and long-term outcome subdomains relating to subjective well-being, mental, behavioural and neurodevelopmental disorders and suicide-related outcomes. This was done for the RCTs that assessed intrapersonal, interpersonal, organisational, community and policy-level interventions. It was conducted for non-randomised evaluations of organisational, community and policy-level interventions.

For both randomised and non-randomised studies, we conducted an assessment to decide if an individual study was biased or unbiased for each outcome, which was largely derived from the quality appraisals. RCTs had a baseline rate of high certainty and non-randomised studies a low certainty rating. As per GRADE guidance, the certainty assessment per outcome was then determined by prespecified criteria. Certainty was rated down according to RoB, imprecision, inconsistency, indirectness and publication bias. Certainty was rated up for large magnitude of effect, dose–response gradient and residual confounded would decrease the magnitude of effect (where this is an effect). The certainty of the evidence was assessed according to very low, low, moderate and high. Where there were serious concerns about the evidence, it was downgraded by + 1 points, and where there were very serious concerns, it was downgraded by + 2 points. Where there were reasons to upgrade the evidence, it was upgraded by + 1 point if there was sufficient reasoning and by + 2 points if there was strong reasoning.

We used the GRADE-CERQual96 tool to assess the certainty of evidence from rich process evaluation studies, with six statements being generated. Each statement was assessed across four components: methodological limitations; coherence; adequacy; and relevance. Each evidence statement was rated as high in the first instance and was rated down if there were concerns about each component. From here, an overall CERQual assessment of confidence in the evidence was made, with an accompanying explanation. Confidence in the evidence was rated as high, moderate, low or very low.

Phase 1: intervention identification

We asked stakeholders to assess the potential candidate evidence-based intervention theories and types identified from the review against the following progression criteria: (1) acceptability; (2) potential effectiveness and (3) feasibility (particularly feasibility of implementing an intervention in the UK context). To support this process, consultations considered the key context factors identified from earlier consultations and the process evaluation synthesis.

Phase 2: intervention development and adaptation

The next stage was to identify an intervention to take forward for future development, adaptation and evaluation. Stakeholders in this phase of consultation, combined with the discussion from earlier consultations, generally felt that the interventions identified in the evidence base were not exactly relevant to the UK context. As such, much of the emphasis of discussion was on the potential for de novo intervention development. To support this, we considered any additional context factors that would need to be taken account of, preferable intervention theories and associated components, and priority outcomes. Following the consultations, we cross-referenced this discussion with the review evidence base to check if there were interventions that may not necessarily have established evidence of effectiveness, but enacted the preferred theories, used suggested components and targeted priority outcomes. This was considered as part of the 2 × 2 integrative matrices on priority intervention types, theories and outcomes.

Phase 3: intervention evaluation

The final phase was considered by the research team and focused on recommendations for future research, considering potential intervention development and evaluation in accordance with Medical Research Council’s guidance. 98

Types of evidence

All 124 reports, linked to 64 interventions, were eligible as part of the evidence map. We have included these in the present chapter (RQ1). Study reports presented theory, process, outcome and economic evaluations. We classified study types by evaluation design to support understanding of whether current evaluation practice in this area is conducted in accordance with methodological guidance on intervention development and evaluation, which recommends the integration of these four evaluation types;98,99 24 reports provided an explanation of interventions’ programme theory (RQ1);25,27,28,30,100–119 50 process evaluations, both conceptually rich and thin, provided data on context, implementation and acceptability (RQ3; RQ4). 27,34,108,114,118,120–164 There were 86 outcome evaluations, using a RCT or non-randomised study design (RQ2). 26,27,29,31,33–35,100,103,108,109,111,113–118,122,128,130,131,134,135,140,142,143,147,148,151,157–160,162,163,165–212 There was one partial economic evaluation (RQ2). 213 The study reports according to each evidence type are presented in Report Supplementary Material 6.

Rates of report

The 124 study reports were published between 1990 and 2022. Only two reports were published between 1990 and 2000. 105,170 Reports were published by subsequent years as follows: 1 in 2001;178 3 in 2002;106,109,148 3 in 2003;30,126,202 1 in 2004;188 5 in 2006;101,171,174,203,205 2 in 2007;108,112 3 in 2008;117,127,192 6 in 2009;28,107,110,169,184,206 6 in 2010;26,131,168,172,197,211 4 in 2011;136,146,160,183 7 in 2012;100,102,114,122,134,159,201 8 in 2013;35,103,111,125,133,142,153,187 7 in 2014;31,120,124,129,141,158,176 5 in 2015;25,27,135,164,182 12 in 2016;104,144,150,151,163,177,180,181,189,191,194,213 11 in 2017;34,113,121,149,152,173,175,186,190,198,204 10 in 2018;33,128,132,143,145,166,185,193,199,200 8 in 2019;29,116,123,140,147,161,167,207 11 in 2020;157 and 8 in 2021115,130,137,139,155,162,165,210 and 1 in 2022. 157 Figure 4 reports the number of study reports published by year according to the review RQ and evidence type. Study reports are double counted where they report evidence according to more than one RQ. There were no significant increases in reporting on programme theory. In contrast, there was a growing number of intervention evaluations using RCT and non-randomised evaluation designs and an expansion in the use of process evaluation.

FIGURE 4.

Cumulative rate of report and evidence type.

Geographical location

We specified that study reports had to be located in higher income countries as the review was primarily concerned with intervention transportability to the UK context. In total, the study reports were from 12 countries, with one report being conducted across both the USA and UK. 102 A significant majority of reports were from the USA (n = 77). 25,26,28–30,33–35,101–108,110–112,115–117,119,123–125,127–129,131,132,135,137,141,142,145–147,152,156,158–161,164–167,169–172,174,177,180–184,186–192,194,197,199–202,205–207,210–212 The remainder were from UK (n = 22);27,31,102,109,118,120–122,126,130,133,134,136,139,140,149–151,154,155,178,179,213 the Netherlands (n = 6);143,163,173,175,193,203 Belgium (n = 3);113,114,185 Australia (n = 3);153,162,209 Portugal (n = 3);144,204,208 Canada (n = 2);148,176 Ireland (n = 2);138,196 Israel (n = 2);100,168 Germany (n = 1);157 Spain (n = 1)198 and Sweden (n = 1). 195

Types of interventions

We classified types of interventions according to the socioecological domain or domains in which they operated (see Report Supplementary Material 6). The range of interventions are described in Figure 5. As indicated in Chapter 2, this was undertaken to respond to the review’s focus on the contextual contingency of intervention effects; we operated on the assumption that interventions working at different socioecological levels may interact differentially with the system depending on the area in which they are implemented. Classification of socioecological domain was informed by the theoretical basis of interventions, where this information was available. However, as the majority of interventions did not present a clear theory, we also considered information on the population and setting that was targeted by intervention activities (e.g. a skills curriculum directly engaging children and young people).

FIGURE 5.

Overview of intervention type by socioecological domain.

Nine interventions, with nine study reports, operated at the intrapersonal level, primarily targeting children and young people;120,135,142,143,151,168,200,202,204 15 interventions, with 24 study reports, targeted both the intrapersonal and interpersonal domain. 25,26,100,105,111,112,115,116,118,129,131,147,154,155,158,159,169,176,188,195,198,203,206,208 These interventions largely combined skill and knowledge development for children and young people, with curricula and coaching to support the carer–child relationship. 105 One intervention targeted the intrapersonal, organisational and community domains. 170

A total of 26 interventions, with 47 study reports, targeted the interpersonal domain, primarily focused on the relationship between care-experienced young people and their peers, carers and parents or other significant adults. 27,33,103,108,109,113,114,123,128,130,133,134,138,140,145,146,148,156,157,160,162,163,165–167,174,175,178–181,183,185,186,192,194,196,204,205,207,209,210,212 Two interventions, with 2 study reports, operated at the interpersonal and organisation level;132,139 5 interventions, with 32 study reports, targeted the interpersonal, organisational and community domains. 28–31,101,102,105–107,110,117,121,122,124,125,127,136,137,141,149,150,152,153,161,164,172,173,182,189,201,213

One intervention, with two study reports, targeted the organisational domain, focusing on organisational culture. 119,191 Four interventions, with four study reports, targeted community mental health and well-being provision. 34,35,126,187 One intervention, with four study reports, operated at the policy level, and focused on the comparison of different placement types that might be prioritised and funded. 190,193,197,199

Intrapersonal intervention characteristics

The nine interventions classified as operating primarily at the intrapersonal level tended to focus on developing the skills, knowledge and resilience of children and young people. Cognitive and affective bibliotherapy comprises eight sessions with young people in residential care, exploring written texts as a departure point for discussing emotions. 168 Cognitively based compassion training is a 6-week, foster care-based cognitive training programme that delivers twice weekly sessions to teach competency in loving kindness, empathy and compassion. 142 One intervention provides additional therapeutic support, namely individual and rehabilitative strategies, to children and young people in intensive TFC. 200 The Sanctuary Model (Sanctuary Institute, New York, NY, USA) delivers 12 weekly psychoeducational group curricula to children and young people in residential care, with these groups also supporting trauma recovery. 202 Staff provide ongoing technical assistance and consultation, in addition to twice daily community meetings in the placement to teach young people awareness about the importance of proximal relationships.

Two cognitive–behavioural therapy (CBT) informed interventions were delivered through online and virtual modalities. One computer game intervention for young people in residential care entails playing the computer game The Sims: Life Stories (or ‘electronic dolls house’), to identify and model emotions, with parallel emotion regulation skill coaching by a social worker. 120 Meanwhile, Dojo Biofeedback is a game that teaches young people in residential care CBT-based relaxation techniques through a series of tutorials and mini-games. 143

Two interventions emphasised mindfulness practice. One is a 20-week Kundalini yoga programme delivered in residential settings, which addresses posture, breathing and mindfulness techniques. 151 The second is a mindfulness curriculum for children and young people in foster and kinship care, which includes guest speakers, arts and crafts activities, yoga instruction, playing music and open time to socialise. 135 Finally, Opportunities Box is a 6-week programme focusing on promoting career ability, adaptability and decision-making for institutionalised youth with a history of care. 204

Fifteen interventions operated at both the intrapersonal and interpersonal level, often combining individual development with group-based curricula and relationship-based components. Acceptance and commitment therapy for children in residential care comprises group-based psychoeducational curricula including experiential exercises, role plays and illustrations to develop psychological flexibility. 195 One intervention for children and young people in care tests a range of treatments that includes behavioural management by online care workers, psychodynamic treatment, structured boundaries and relationships, and adventurous learning that models self-supportive, adaptive behaviours. 203 Derived from MTFC, early-intervention foster care intends to provide parenting coaching and group support to foster carers to encourage placement permanency. 105 This is combined with behavioural specialist services and weekly therapy playgroup sessions for children.

Two interventions explicitly drew on a range of creative and leisure activities to support children and young people’s skills development and relationships. HealthRHYTHMS Drumming Protocol is a weekly group session for young people to non-verbally express themselves through music, before progressing to verbal and written forms of communication. 169 Wave by Wave is a 3-hour weekly psychoeducational programme for young people in foster care, which includes the physical activity of surfing to reduce the potential stigma of receiving mental health support. 208

Two interventions had a focus on animal-based therapy. Equine-facilitated psychotherapy (EEP) provides 7 months of weekly sessions with a horse for young people in residential care, with the aim of building a therapeutic alliance, developing adaptability and providing a healing experience. 100 Animal-assisted psychotherapy for children in foster care includes individual and small-group sessions with animals during an overnight visit to a farm. 198

Two interventions in this domain also included parents and carers alongside young people. One Medicaid-funded intensive outpatient service provides in home support for 1–2 years, offering individualised family-focused and child-centred treatment face to face, in addition to specialist support to relevant adults. 206 Trauma-focused CBT (TF-CBT) provides cognitive development support through child, carer and conjoint child-caregiver sessions. The intervention also includes an engagement component, with contact between the family and delivery agent to target perceptual barriers to participation. 129,158

Two interventions focused on mentoring. Fostering Healthy Futures (FHF) provides group-based curricula and mentoring for children and young people in out of home care. 25,26,111,112,115,116,147 The weekly sessions, over 30 weeks, follow a manualised curriculum addressing cognitive and behavioural skills, while individualised mentoring is provided by graduate social work students to model prosocial relationships. Take Charge provides mentor coaching to young people with additional needs in self-determination, combined with mentoring group meetings with intervention peers and foster care alumni. 159 With a focus on the relationship with the clinician, Life Story work targets children in foster care with a family history of methamphetamine-use and centralises building a trusting relationship before the child and clinician work together over 7 months to construct a culturally sensitive narrative of personal experience that challenges information and perceptions of high-risk behaviours. 131

Two interventions operating across the intrapersonal and interpersonal domains had a primary focus on educational outcomes. Kids in Transition to School (Oregon Social Learning Center, Eugene, OR, USA) is a school-based therapeutic playgroup, aimed at practising social skills and self-regulation to support school readiness. 188 Teach Your Children Well includes individualised tutoring for children, focused on reading and mathematical competency. 176

One intervention had a focus on drug and alcohol use. Supporting Looked-After Children in Decreasing Drugs and Alcohol (SOLID) provides six sessions of two interventions: first motivational enhancement therapy delivering client-centred counselling with a problem feedback component to reflect on the impact of drug and alcohol use and behavioural and cognitive strategies to build social networks to support positive behaviour change. 118,154,155

One intervention targeted the intrapersonal, interpersonal and organisational domain. Fostering Individualized Assistance Program (FIAP) provides a family specialist to children in foster care, acting as a family-centred clinical care manager and home-based counsellor. 170 They offer strength-based assessment, life domain planning, clinical case management and follow-along supports and services, providing a tailored set of services to meet the child’s needs.

Interpersonal intervention characteristics

Twenty-six interventions operated at the interpersonal level, with the vast majority focusing on training in parental management skills for foster and kinship carers. One social worker delivered intervention, derived from Save the Children’s content, provides 3 days of training to improve foster carers’ communication and attachment to the child. 178 Another intervention similarly offers 3 days of in-person group training to foster carers, although delivered by a clinical psychologist, with the aim to improve behavioural management. 109

Attachment and biobehavioural catch-up (ABC) is a home-based parenting programme for foster carers, with child–carer interactions videotaped to tailor content to their specific attachment and parenting needs. 184 Child Adult Relationship is a 6-hour, trauma-informed parenting programme to support the relationship with children and their foster carers. 33 Family Minds is a group-based online curriculum to increase carers reflective functioning and mentalisation skills. 165 One foster carer training provided at home visits by specialist foster carers to deliver a curriculum to improve parenting practices, behavioural management and carer stress, with monthly group sessions providing peer support. 113,114 Fostering Changes is a 12-week group-based training programme for foster carers that combines with a support group to improve parenting practices and communication styles. 179 Fostering Connections is a 6-week trauma-informed psychoeducational programme for carers to help understand and apply trauma-informed parenting strategies, with training including experiential exercises, videos, role play, discussion and home exercises. 138,196

The Herts and Mind study evaluated mentalisation-based therapy, a 12-week programme for foster carers, combing psychoeducation on attachment and mentalising among children with histories of maltreatment, consultation with professionals and direct relational work with families. 140 Non-violent resistance training is a 10-week foster carer training programme focused on parental submission and power struggles in the caring relationship, with telephone support sessions and accompanying learning materials. 185 Solution-focused parenting groups comprises a six-session group programme for foster carers of children with behavioural problems to improve parenting competence, with assignment activities to enact parenting goals. 148 A version of Triple P, adapted to foster carers, provides five weekly group parenting sessions to parents to teach parental management, combined with two telephone consultations. 157

Parent–child interaction therapy (PCIT) is a two-stage parent management training (PMT) programme for foster carers in outpatient settings: child-directed interaction to cultivate parent–child bonding; and parent-directed interaction to enhance parents’ behaviour management skills. 123,177,205,212 This is combined with a one-to-one home visit programme to reinforce and model key skills taught during sessions and is accompanied with a monthly support group for intervention families.

Several interventions had an explicit focus on kinship care or a range of placement types that went beyond foster care. Child-directed interaction training provides eight group-based coaching sessions to kinship carers to enhance the attachment relationship by improving parenting skills. 180 Connect-KP provides a nine-session group-based trauma and attachment-based parenting support for carers, while exploring the particular challenges of kinship care. 162 Incredible Years provides a trauma-informed group parenting programme to biological, foster, kinship and residential carers to support positive parenting and engaging in child serving systems. 108,128,130,133,144,174 PMT Treatment Oregon Model (PMTO) is a variation of MTFC. They tend to encompass a 1- to 2-weekly group parenting programme, home practice assignments and home visit supervision in behavioural management by trained PMTO facilitators. 146,160,163,166,167,175,210

Three interventions focused on relationships with biological families and reunification. One intervention, kContact, provides caseworker support to biological parents during contacts with their child, with telephone sessions encouraging parents to plan for, reflect upon and review the goals of contact. 209 Pathways Home aims to prevent reunification failure, providing a parenting curriculum and booster sessions to biological parents to ensure a safe and nurturing environment. 103 Similarly, Promoting First Relationships is a home visiting programme for biological parents to support reunification, through 10 brief manual-based sessions to increase parenting confidence and competence. 181

One intervention focused on improving carer–child relationships, in addition to peer relationships, to prevent internalising and externalising problem behaviours among girls. The intervention comprises six sessions of professionally led groups, one each for girls and foster carers during the school summer holidays, and weekly training and support over the subsequent academic year. 183 One intervention provides therapeutic mentoring by a clinician to young people in foster care, where they engage in a range of preplanned, mutually agreed activities. 211 Two interventions emphasised relationships with other adults and peers. Youth-initiated mentoring (YIM) matches young people in foster care with trained mentors to establish a one-to-one community-based relationship. 145 One intervention delivers a peer mentoring programme for girls at risk of pregnancy, where trained mentors support one mentee for a year to engage in a range of activities and contacts, through a range of means (e.g. face-to-face meetings, e-mail, telephone conversations and texts). 27

Interventions further worked to extend children and young people’s support networks. Family Finding entails professional, independent workers searching for, discovering and engaging actual and fictive kin for children and young people in foster care, with these individuals providing resources and various supports. 156,186,194

One interpersonal intervention increased resources available to foster carers through a private foster care agency, with carers having more financial resources to support the child, access to case manager support and educational support. 192

Two interventions targeted the interpersonal and organisational domains. Head, Heart, Hands aims to introduce social pedagogy to foster carers and service staff, through a series of taster sessions, courses and support groups, to create relationships with children that foreground social connections, co-creation and meaning making. 139 Intensive Permanence Services (IPS) is a 24-month programme that engages young people’s family members and other supportive family members to support them on the path to permanency. 132

Five interventions targeted the interpersonal, organisational and community level. Three of these interventions, MTFC, keeping foster parents trained and supported (KEEP), and TFC are derived from the same approach and principles. MTFC centralises placing young people in specialist, supervised foster homes with expertise in positive behavioural management. 30,31,101,102,107,122,124,136,173 Regular support is provided by programme supervisors with small caseloads, and the care placement is integrated into a range of system services. System-wide delivery models, such as train the trainer, have been developed to improve implementation.

Keeping foster carers and kinship carers supported draws upon components of MTFC for delivery with ‘regular’ foster and kinship carers. 28,29,102,125,127,152,182,189 Training can include participation in weekly training and support meetings, and twice weekly calls with a MTFC paraprofessional. As with MTFC, additional components have been added to optimise implementation and integrate the intervention into the wider social care system.

Treatment Foster Care (TFC) is a parenting programme that is integrated into a suite of wraparound services. 104,106,110,137,141,161 Together Facing the Challenge (TFTC), a close variant of TFC, is a weekly parenting programme for foster carers in effective behaviour management, which is embedded in a wider context of social care system activities. TFC for older youth is another adaptation that includes a clear psychiatric component, life skills development, future planning for education and employment and involvement of youth voice. In addition to foster carer training, it includes a range of wraparound services: psychiatric nurse support, family consultation, life coaching and skills coaching. 164

Evolve Behaviour Support Services provides therapeutic mental health, disability, behaviour support and educational services to disabled children and young people in out of home care. 153 Through co-ordinated stakeholders’ service delivery, it offers positive behaviour support through child-focused therapy, carer education and training and environmental strategies.

Glasgow Infant and Family Team (GIFT) and London Infant and Family Team (LIFT) are versions of the New Orleans Model. Children who have been abused and placed into foster care are referred to the service and, through a series of interviews, observations and questionnaires, working with families for approximately 3 months, the multidisciplinary team is able to make an informed decision about the best placement outcome for the child. 149,150,215

Organisational, community and policy intervention characteristics

One intervention operated at the organisational level, shifting organisational culture to have a more therapeutic, trauma-informed approach. Children and Residential Experiences (CARE) provides consultation, training and technical assistance to residential placement settings to create a more therapeutic environment through improvement to their policies and practices. 119,191

Four interventions targeted the community level, emphasising choice, availability and access to community mental health services. Head Start is a large publicly financed childhood education and care-programme, providing comprehensive wraparound community services to support disadvantaged children and families, including young people in care. 35 One study focused on optimising CAMHS for children and young people in care. It includes a model to support the interface between healthcare and social care professionals, and a single referral to services to improve access and effective engagement with young people through joint forums, partnership working and professional training. 126 One intervention improves choice and availability of different community outpatient mental health services, that includes drug and alcohol clinics, community health centres, crisis centres and private professional treatment. 187 Trauma systems therapy (TST) is a system wide-trauma informed model of practice to improve decision-making for the treatment pathway of children and young people in care, and enhance service co-ordination. 34

There was one intervention at the policy level, which was care placement type (e.g. foster, residential and kinship care). 190,193,197,199 In this instance, local and regional government legislation and policy increases the availability of different types of care and evaluation compares mental health related outcomes across placement type.

Quality appraisal

The quality appraisal assessment of theory study reports is presented in Report Supplementary Material 8. This was primarily to assess the strength of the theoretical basis of interventions, which could support decision-making on the development, adaptation and evaluation of theoretically informed interventions moving forward. To note, we assessed each report on a case-by-case basis, so study reports linked to the same intervention evaluation could be assessed differently across different quality domains.

We identified two key limitations with study reports. First was a lack of involvement of children, young people, carers and professionals in the development of the theoretical basis for interventions. Only two interventions reported explicit involvement of care-experienced individuals in theory development. 112,118 One stated the process for development but stakeholder involvement was not clearly specified. 27

Second, there was a lack of clarity regarding the theoretical constructs drawn upon. Some reports provided an extensive description of a range of principles and literatures, including ‘formal’ established theories but did not always integrate them into a coherent programme theory. Commonly, there was a lack clear linkage between the theory and the intervention components and outcomes. Reports tended to describe theories in the background section, while the description of the intervention focused more on activities and delivery strategies. There was also limited use of logic models, being included in seven study reports. 25,26,106,107,113,118,119

Description of intervention theories

We present intervention theories according to the socioecological domains used to classify intervention types.

Primary outcome domains

Targeted intervention outcomes are presented in Report Supplementary Material 9. There were three primary outcome domains of the review: (1) subjective well-being; (2) mental, behavioural and neurodevelopmental disorders and (3) suicide-related outcomes. These domains and their subdomains were specified a priori.

The majority of outcome measurements were linked to child mental, behavioural and neurodevelopmental disorders, which had 13 associated subdomains. The most frequently measured of these subdomains were: total social, emotional and behavioural problems, which was targeted by 48 interventions with 74 associated study reports;25–29,31,35,103,105,106,109,112–114,117–119,122,127,129–131,133,134,140,142,144,146–150,153–156,158,159,162–167,170–173,175,177,178–186,189,193,195–197,199–203,205,207–210,212,213 social–emotional functioning difficulties, which was targeted by 17 interventions with 33 study reports;25,31,34,100,106–108,110,111,115,120,122,128,134,152,157,166–170,174,188,190,198,199,202,205,206,211–213 internalising problem behaviours, which was targeted by 22 interventions with 32 study reports;103,108,111–113,115,123,128,131,133,135,140,160,163,170,173,175,177,180,181,183–187,193,194,197–199,203,207 and externalising problem behaviours, which was targeted by 26 interventions with 38 study reports. 103,108,112–114,123,128,129,131,133,135,140,143,158,160,163,169,170,173–175,177,180,181,183–188,193,194,197–199,203,207,212

Subjective well-being and suicide-related outcomes were less frequently assessed. Only 11 interventions targeted domains of well-being related to quality of life;25,26,111,115,118,147,152,154,155,208 subjective well-being;118,132,145,151,154,155,176,197 and life satisfaction. 100,204 Suicide-related outcomes were targeted by four interventions, including suicidal ideation,123 self-harm95,150 and suicide attempt. 59

Additional outcome domains

In total, there were eight domains (see Report Supplementary Material 10). Six of these related to child-level outcomes, one was related to parent-level outcomes and one was related to organisational- and community-level outcomes. These domains, and their associated subdomains, were inductively classified from the study reports. In this instance, reports were coded using the concepts cited by the author(s) and then combined into categories of domains and subdomains (e.g. child offending and delinquency were combined into a category of antisocial behaviour, delinquency and offending).

For child-level outcomes, the most frequently assessed outcome domains was additional health outcomes, with the majority within this domain linked to coping, self-esteem and related constructs. 26,27,100,111,116,119,135,137,142,152,156,158,159,169,172,178,186,195,198,202,204,208 Meanwhile, social care outcomes were commonly assessed, with 14 intervention and 19 study reports, targeting placement stability, changes and breakdown. 28,34,105,113,118,154–156,159,162,164,170,179,185,186,189,190,194,207 A range of subdomains linked to carer- and parent-level outcomes were assessed, with the more frequently targeted one being carer/parent mental health and well-being. 107–109,113,123,128,130,133,151,162,163,165,173,175,179–182,184,185,189,205,209,213 A total of 7 interventions, with 10 study reports, targeted organisational environment and functioning. 102,119,124,125,130,132,136,191,202,213

Synthesis of randomised controlled trial outcome evaluations

A total of 38 interventions, with 52 associated RCTs, were included in the synthesis (see Report Supplementary Material 11): 23 interventions, with 32 evaluations, were from the USA; 6 interventions, with 8 study reports, were from the UK; 3 interventions, with 4 study reports, were from the Netherlands; 2 interventions, with 2 study reports, were from Australia; 2 interventions, with 2 study reports, were from Belgium; 1 intervention, with 1 study report, was from Canada; 1 intervention, with 1 study report, was from Germany; 1 intervention, with 1 study report, was from Israel; 1 intervention, with 1 study report, was from Portugal.

Interventions primarily operated across the intrapersonal and interpersonal domains. Six interventions, with six associated study reports, targeted the intrapersonal domain; 22 interventions, with 32 associated evaluations, targeted the interpersonal domain, and 9 interventions, with 13 associated study reports, targeted both domains. There was one RCT evaluation of an intervention targeting the interpersonal, organisational and community domains. FIAP delivers intensive, personalised support services across agencies. 170 Only one RCT evaluation assessed a policy-level intervention, Head Start. 35

The comparator group for all evaluations was usual practice or standard care. There were three exceptions. Evaluation of TFTC, an extension of TFC, compared the extended intervention with the original TFC approach. 172 Evaluation of TF-CBT compared an optimised version of the intervention, including additional engagement activities, with TF-CBT alone. 158 Meanwhile, evaluation of Promoting First Relationships compared the intervention group with those receiving an active control of early educational support. 181

Three RCT evaluations reported more than one experimental group. Evaluation of bibliotherapy compared both cognitive bibliotherapy and affective bibliotherapy with usual care. 168 Evaluation of the HealthyRHYTHYMS protocol compared a standard or extended version of the protocol with usual care. 169 Evaluation of PCIT compared both a standard and extended version with usual care. 177

The duration of interventions ranged from 3 weeks178 to 18 months. 170 Nine interventions, with nine study reports, were delivered for 3–6 weeks. 33,142,157,165,169,172,178,180,183 Most interventions lasted a minimum of 2 months, with 13 interventions, and 22 associated study reports, lasting for 5 months or more. 26,27,31,35,103,116,122,131,147,159,163,166,167,170,173,175,176,186,209,210,218

We classified follow-up times for outcome assessment from baseline according to 0–6 months (shorter term) and more than 6 months (longer term) categories. Longer-term follow-up outcome data were available for 13 interventions, with 17 study reports. 26,27,31,113,122,131,147,159,163,166,170,172,176,178,179,186

Synthesis of quasi-experimental design outcome evaluations

Three interventions, with three associated study reports, were included in the synthesis34,187,191 (see Report Supplementary Material 12). All interventions were evaluated in the USA. One intervention targeted the organisational domain. CARE aims to improve ethos and practice across residential care agencies, and was evaluated using an interrupted time series design. 191 Two interventions targeted the community domain. One focuses on increasing the availability of community health services to young people, and was evaluated through routine administrative data;187 TST is system-wide effort to implement trauma-informed care and was evaluated across a large, private welfare system in the USA. 34 Intervention duration and time to follow-up times were typically longer than the RCT evaluations.

Synthesis of economic evaluations

Only partial economic evaluation was eligible for the review. 213 This was included on the advice of the project advisory group, who observed the paucity of economic evaluations in children’s social care. The evaluation examined the potential costs and consequences of implementing the New Orleans intervention in Glasgow (GIFT), comparing estimated costs of the new with the costs of the current foster care system (service as usual). Usual care was reported to cost approximately £30,586 per child for the first year and £24,924 per year thereafter. In comparison, they estimated that GIFT would cost approximately £55,345 per child for the first year, falling to £44,896 thereafter. However, based on reviews of the intervention delivered elsewhere and the consultation with expert stakeholders, evaluation estimated that implementation of GIFT would reduce the probability of children re-entering care from a 44% rate of return in the existing system to 3%. It would also reduce the average length of stay from 30 months to 20.5 months. As a result, the economic modelling estimated that the average cost per child would be lower with GIFT (£88,653) compared with the existing system (£95,473).

Randomised controlled trial evaluations: quality appraisal

We appraised the 52 RCT evaluation with the RoB2 tool (see Report Supplementary Material 13). 219 The tool assesses potential bias across five domains: (1) bias arising from randomisation processes; (2) bias due to deviations from the intended intervention; (3) bias due to missing outcome data; (4) bias in the measurement of outcomes and (5) bias in the selection of reported results.

Quasi-experimental design evaluation: quality appraisal

We appraised the three QED evaluation with the ROBINS-I tool. 220 The tool assesses bias according to seven domains: confounding; selection of participants in the study; classification of interventions; deviations from intended interventions; missing data; measurement of outcomes; and selection of the reported results. Agreed assessments are reported in Table 4.

We assessed the CARE intervention as low risk for each domain for the RoB from measurement, as the single informants (carers) were not blinded to allocation. 187 Insufficient data were provided to assess risk from missing data. We assessed TST34 as having risks due to the use of a non-standardised tool to assess outcomes, difficulties in defining dosage or exposure to the intervention, changes made to the evaluation programme and a reliance on proxy fidelity measures. We assessed the impact of access to outpatient mental health services as having low RoB across most domains, with the exception of bias from outcome measurement due to a reliance on agency staff to define ‘behavioural incidents’. 191

Subjective well-being

In accordance with the evidence map, we classified subjective well-being according to three subdomains: quality of life, subjective well-being and life satisfaction. Two interventions, with three RCT evaluations, assessed quality of life with child-reported measures at short and long-term follow-up (Table 5). 26,147,218

Fostering Healthy Futures reported significantly better quality of life at post intervention (5 months post baseline) compared with the control group (d = 0.42, 95% CI 0.12 to 0.71). 26 No group differences existed at 6-month follow-up in the same cohort, and there were no significant differences observed in long-term quality of life measures in a later extension of the same study. 147 No group differences in quality of life were observed in the Wave-by-Wave intervention. 218 One pilot RCT measured quality of life using the EuroQol-5 Dimensions tool but the study did not present any data on group differences.

No RCT evaluations assessed subjective well-being or life satisfaction. There were insufficient data available to conduct meta-analysis for this outcome domain.

Mental, behavioural and neurodevelopmental disorders

In accordance with the evidence map, we classified mental, behavioural and neurodevelopmental disorders according to eight subdomains: total social, emotional and behavioural problems; social–emotional functioning; internalising behaviour problems; externalising behaviour problems; anxiety and depression; stress, PTSD and trauma; attachment style and disorder; attention and hyperactivity disorder.

Total social, emotional and behavioural problems

Some 16 interventions, with 22 RCT evaluations, reported outcomes relating to total social, emotional and behavioural problems (see Report Supplementary Material 14).

Shorter-term outcomes

Of the interventions, 6, with 10 RCT evaluations, did not demonstrate overall effectiveness for reduced short-term problem behaviours,103,128,163,166,167,175,181,185,210,221 while 1 evaluation did not present enough evidence to assess group differences. 171

Two evaluations of the KEEP intervention showed mixed results. In one US-based study, children aged between 6 and 11 years had significantly fewer carer-reported problem behaviours than control peers at 6-month follow-up (d = −0.26, 95% CI −0.43 to 0.09). 117 However, in a larger 2015 US study of the same intervention, there were no significant differences between intervention and control group children for carer-reported problem behaviours at 4-month follow-up (d = −0.01, 95% CI −0.22 to 0.21). 182

Evaluation of mentalisation-based therapy found that children and young people self-reported significant reductions in their problem behaviours at 12-week (d = −0.67, 95% CI −1.12 to −0.22) and 24-week (d = −0.76, 95% CI −1.21 to −0.31) follow-up. However, carer-reported problem behaviours demonstrated the opposite trend, with carers reporting that children in the intervention group showed more problem behaviours than the control group at 12 weeks (d = 0.31, 95% CI −0.09 to 0.71) and significantly more at 24 weeks (d = 0.34, 95% CI 0.04 to 0.84). 140

Evaluation of the brief and extended version of PCIT on children’s problem behaviour at 8- and 14-week follow-up, compared with a control group. 177 Carer-reported problem behaviours were significantly lower in the brief intervention group than control at 8-week follow-up (d = −0.64, 95% CI −1.16 to −0.12), but the difference was not significant at 14 weeks (d = −0.23, 95% CI −0.77 to 0.23). Children receiving the extended version of the intervention also had lower carer-reported problem behaviours at both follow-up times, but differences were not significant.

Young adolescents who received the ‘affective’ arm of the cognitive and affective bibliotherapy intervention showed lower total problem behaviours (as rated by counsellors) at 3-month follow-up than a control group (d = −1.02, 95% CI −1.60 to −0.44). No short-term improvements to problem behaviours were observed for participants in a parallel ‘cognitive’ arm of the trial.

Finally, evaluation of TFTC reported significantly improved short-term outcomes for problem behaviours among adolescents as compared with an active control group receiving the standard TFC. The intervention group had lower scores on the Parent Daily Report [β = 0.30, standard error (SE) = 0.11, p < 0.01] at 6-month follow-up.

Longer-term outcomes

Four interventions, with five RCT evaluations, did not demonstrate effectiveness for reduced problem behaviours at longer-term follow-up. 31,131,172,173,186 Two evaluations measured the effect of PMTO on longer-term child problem behaviours, with mixed results. In one US-based study, carer-reported problem behaviours were significantly lower in the intervention group compared to control at 12-month follow-up (d = −0.31, 95% CI −0.55 to −0.06). 166 However, implementation of the same intervention in the Netherlands showed no beneficial effect on carer-reported problem behaviours at 12 months (d = 0.00, 95% CI −0.35 to 0.35). 163 Teacher reported problem behaviours were higher in the intervention group at the same follow-up, though this did not reach significance (d = 0.37, 95% CI −0.12 to 0.86).

Young children whose carers who received Communication and Attachment training self-reported fewer total problem behaviours at 9-month follow-up compared to controls (d = −0.28, 95% CI −0.61 to 0.04), though the effect did not quite reach significance. However, teachers independently evaluating the same cohort reported the reverse pattern, with more problem behaviours in the intervention than the control group (d = 0.27, 95% CI −0.06 to 0.60). Again, this did not reach statistical significance. 178

Finally, young adolescents receiving the TFTC intervention showed significantly fewer problem behaviours on the parent daily report at 12-month follow-up than an active control group receiving the TFC intervention (β = −0.38, SE = 0.14, p < 0.05).

Meta-analysis of shorter- and longer-term follow-up

A total of 13 interventions, with 15 RCT evaluations, were eligible for inclusion in the meta-analysis of total problem behaviours. Seven RCT evaluations were ineligible: three were moderation analyses or reused analysis from other included evaluations,175,210 two evaluations used an active control group,172 and two did not provide sufficient information to extract effect sizes. 173 We undertook separate meta-analyses to investigate the effect of interventions on total problem behaviours measured at shorter-term (0–6 months of follow-up) and longer-term (> 6 months of follow-up) follow-up. Collectively, interventions significantly reduced problem behaviours in the short term (d = −0.15, 95% CI −0.28 to −0.02; Figure 7). The short-term meta-analysis included 20 effect sizes from 11 evaluations, with substantial between-studies heterogeneity (I² = 62%). However, there was no evidence that interventions reduced total problem behaviours at longer-term follow-up (d = −0.07, 95% CI −0.38 to 0.25; Figure 8). For the longer-term follow-up meta-analysis, we included 12 effect sizes from 6 evaluations and observed moderate heterogeneity (I² = 56%).

FIGURE 7.

Forest plot for meta-analysis of total problems (short term).

FIGURE 8.

Forest plot for meta-analysis of total problems (long term).

Total social–emotional functioning and/or impaired functioning

Of the interventions, 18, with 24 RCT evaluations and 1 QED evaluation, reported outcomes relating to social, emotional and behavioural functioning and/or impaired functioning (see Report Supplementary Material 14).

Shorter-term outcomes

Eight interventions, with eight RCT evaluations, demonstrated no effectiveness for reducing short-term social–emotional functioning difficulties. 142,158,162,165,172,183,218,221

Five interventions, with five study reports, demonstrated effectiveness in improving some aspect short-term functioning in children and young people. The Fostering Changes intervention reported significantly lower carer-rated scores than children in the control group for emotional difficulties (d = −0.33, 95% CI −0.60 to −0.06), total difficulties (d = −0.32, 95% CI −0.59 to −0.05) and peer problems (d = −0.28, 95% CI −0.56 to −0.01) at 3-month follow-up. 179 No significant group differences were found in an earlier UK evaluation of the same intervention. 221 Children in the ‘extended’ arm of the HealthRHYTHMS musical therapy intervention rated themselves as having significantly fewer interpersonal problems at 6-week follow-up (d = −0.90, 95% CI −1.60 to −0.20), while children in the standard intervention arm showed no difference from control group participants. There were no significant differences from control group participants for either group (standard or extended) on child-rated social adaptation scores or clinician-rated total functioning scores at 6 weeks. 169

In the PMTO intervention, children had significantly lower carer-rated social–emotional functioning scores than the usual care group at 6-month follow-up (d = −0.19, 95% CI −0.35 to −0.04). 166 Young children who had received PCIT were rated by foster carers as having significantly lower scores on the ‘lability/negativity’ scale of the Emotional Regulation Checklist measure than control group participants (d = −0.67, 95% CI −1.15 to −0.19). 212 Finally, adolescents in the intervention arm of the TFTC had lower 6-month follow-up scores than an active control group (TFC) for Behavioural and Emotional Ratings Scale (β = 0.41, SE = 0.12, p < 0.01) and higher scores for the strengths component of the Strengths and Difficulties Questionnaire (SDQ; β = −0.24, SE = 0.11, p < 0.05). 172

Longer-term outcomes

Six interventions, with seven associated RCT evaluations, showed no effectiveness at reducing long-term social and emotional functioning difficulties. 27,31,122,154,162,179,209

Three interventions showed some effectiveness at improving longer-term functioning outcomes. Evaluation of PMTO-reported children had significantly lower social–emotional functioning scores (as rated by carers) than the control group at 12-month follow-up (d = −0.38, 95% CI −0.57 to −0.19), with the between group difference increasing between 6- and 12-month follow-up. 166 Young foster children who took part in the FHF intervention had significantly lower scores on a multi-informant, composite ‘mental health functioning’ outcome, compared with control group participants, at 18 months (d = −0.25, 95% CI −0.47 to −0.04). Linked FHF study reports showed some moderating effects of risk exposure and relationship history on these outcomes. 115,116

Finally, a QED evaluation of the TST assessed the effect of increased child-level exposure to care agencies trained in trauma-informed practices, looking at trajectories in measures of child functioning, emotional regulation and behavioural regulation. 34 Change in functioning was not predicted by overall dosage, but increases in dosage were associated with decreased challenges with functioning (r = −0.37). For emotional regulation, overall dosage was associated with improvements in functioning over time (β = −0.18) but there was no significant correlation between increases in dosage and improvements in emotional regulation. Finally, change in behavioural regulation was not predicted by overall dosage, but increases in dosage over time were associated with temporal improvements in behavioural regulation (r = −0.17).

Meta-analysis of shorter-term and longer-term follow-up

A total of 9 interventions, with 16 RCT evaluations, were eligible for inclusion in the meta-analysis of total functioning. Eight RCT evaluations were ineligible: four were moderation analyses or reused analysis from other included evaluations;115,116,167,210 three evaluations used an active control group;158,172,181 and one did not provide sufficient information to extract effect sizes. 118 We undertook separate meta-analyses to investigate the effect of interventions on social–emotional functioning difficulties at short-term (0–6 months follow-up) and long-term (7 + months) follow-up. Collectively, interventions significantly reduced social–emotional functioning difficulties in the short term (d = −0.18, 95% CI −0.31 to −0.05; Figure 9). The analysis included 28 effect sizes from 10 evaluations, with moderate between-studies heterogeneity (I² = 53%).

FIGURE 9.

Forest plot for meta-analysis of total functioning (short term).

When evaluated at longer-term follow-up, interventions showed some effect at reducing social–emotional functioning difficulties, with the effect approaching but not reaching statistical significance (d = −0.15, 95% CI −0.40 to 0.09; Figure 10). For longer-term follow-ups, we included 14 effect sizes from 8 evaluations and observed substantial heterogeneity (I² = 63%).

FIGURE 10.

Forest plot for meta-analysis of total functioning (long term).

Internalising behaviour problems

A total of 22 interventions, with 22 RCT evaluations and 1 QED evaluation, reported outcomes relating to internalising behaviour problems. Total internalising behaviour problems were most commonly assessed by the internalising sub-scale of the Child Behaviour Checklist (CBCL) tool, with assessments primarily made by parents or carers (see Report Supplementary Material 14).

Shorter-term outcomes

Nine interventions, with nine evaluations, showed no effectiveness in reducing short-term internalising problem behaviours. 113,128,158,163,169,180,181,185,212

Young adolescents in the ‘cognitive’ arm of the cognitive and affective bibliography intervention showed significantly better outcomes at 3-month follow-up compared with the control group for clinician-rated internalising problems (d = −0.77, 95% CI −1.33 to −0.21) and withdrawal (d = −0.75, 95% CI −1.31 to −0.18), but not psychosomatic problems (d = 0.00, 95% CI −0.54 to 0.54). However, participants in the ‘affective’ arm of the same intervention did not differ from control group participants on any of the outcomes at 3 months post intervention. 168

Infants whose carers had received the ABC intervention showed significantly lower rates of carer-rated internalising behaviour problems compared with control participants at 6-month follow-up (d = −1.60, 95% CI −2.22 to −0.98). 184 KEEP reported that children were significantly less likely than the control group to show clinical (d = −0.23, 95% CI −0.45 to −0.01) or borderline-level scores (d = −0.23, 95% CI −0.45 to −0.01) on the internalising behaviour problems scale of the CBCL scale, as assessed by carers at 6 months. There were no group differences in somatic complaints, and the absolute difference in continuous scores on the internalising scale did not reach significance. 29

Finally, young children who had taken part in mentalisation‐based therapy self-reported significantly lower levels of internalising behaviour problems than control group children, at both 12-week (d = −1.04, 95% CI −1.54 to −0.54) and 24-week (d = −1.30, 95% CI −1.80 to −0.80) follow-up. However, assessment of internalising problems made by carers showed the opposite trend, with intervention group children scoring significantly higher than those in the control group at 12-week (d = 0.35, 95% CI 0.00 to 0.70) and 24-week (d = 0.48, 95% CI 0.13 to 0.80) follow-up.

Longer-term outcomes

Seven interventions, with six RCT evaluations and one QED evaluation, showed no effectiveness for reducing internalising behaviour problems at long-term follow-up. 131,163,170,173,174,186,209

The Take Charge intervention showed lower scores for young people than control group peers for carer-rated somatic complaints at 12-month follow-up (d = −0.30, 95% CI −0.65 to 0.06), although the difference did not reach statistical significance. At the 18-month follow-up, the difference between groups persisted and was significant (d = −0.54, 95% CI −0.90 to −0.18).

Meta-analysis for shorter- and longer-term outcomes

A total of 18 interventions, with 18 RCT evaluations, were eligible for inclusion in the meta-analysis of total functioning. Four RCT evaluations were ineligible. One reused analysis from another included evaluation,175 and one did not provide sufficient data to calculate effect sizes,173 while two evaluations used an active control group. 158,181 We undertook separate meta-analyses to investigate the effect of interventions on internalising problem behaviours measured at short-term (0–6 months) and long-term (> 6 months) follow-up. Collectively, interventions significantly reduced internalising problem behaviours in the short term (d = −0.35, 95% CI −0.61 to −0.08; Figure 11). The analysis included 32 effect sizes from 12 evaluations, with substantial between-studies heterogeneity (I² = 74%).

FIGURE 11.

Forest plot for meta-analysis of total internalising problems (short term).

However, there was no evidence that interventions reduced internalising problem behaviours when evaluated at longer-term follow-up (d = −0.03, 95% CI −0.31 to 0.25) (Figure 12). For longer-term follow-up, we included 16 effect sizes from 7 evaluations and observed moderate heterogeneity (I² = 53%).

FIGURE 12.

Forest plot for meta-analysis of total internalising problems (long term).

Externalising behaviour problems

A total of 32 interventions, with 30 associated RCT evaluations and 2 QED evaluations, reported outcomes relating to externalising behaviour problems. Total externalising behaviour problems were most commonly assessed by the externalising subscale of the CBCL tool, with assessments primarily made by parents or carers (see Report Supplementary Material 14).

Shorter-term outcomes

Some 13 interventions, with 13 outcome evaluations, demonstrated no effectiveness at reducing children and young people’s externalising problems in the short term. 29,33,113,128,140,158,165,174,177,179–181,185

Eight interventions demonstrated effectiveness for short-term reductions in externalising problem behaviours. Young adolescents in the ‘extended’ arm of a musical therapy intervention, HealthRHYTHMS, self-reported significantly lower levels of anger at 6-week follow-up compared with control group peers (d = −0.94, 95% CI −1.64 to −0.24). Adolescents receiving the standard intervention did not differ from controls for self-reported anger at follow-up, and neither intervention group differed from controls at 6-week follow-up for self-reported aggression. 169

Carers who had taken part in the ABC intervention rated children as having significantly fewer externalising problem behaviours than controls at 6-week follow-up (d = −1.80, 95% CI −2.44 to −1.16). 184 Adolescents who received the ‘affective’ arm of the cognitive and affective bibliography intervention (including discussions about their emotional reactions) were rated by clinicians as significantly lower than control group participants for total externalising behaviour problems (d = −1.07, 95% CI −1.65 to −0.49), delinquency (d = −0.99, 95% CI −1.57 to −0.41) and aggression (d = −0.96, 95% CI −1.54 to −0.39), 3 months after the intervention. Participants in a parallel ‘cognitive’ arm of the intervention (with more focus on coping techniques) were rated lower than control groups on the same measures, but unlike those in the ‘affective’ arm, the differences did not reach statistical significance. 168

Young adolescents who took part in the Dojo: Biofeedback videogame intervention self-reported significantly lower levels of externalising problem behaviours compared with control group peers, both post intervention and at 6-month follow-up. 143 However, independent ratings from carers showed no difference from control group participants on the same measure at either time point.

Foster carers who took part in a brief skills-based intervention rated their young foster children’s externalising behaviour problems as significantly lower than those in a control group, at 6-month follow-up (d = −0.09, 95% CI −0.16 to −0.02). 183

Young children of foster carers who took part in the kConnect intervention had significantly lower scores for carer-rated ‘affect dyscontrol’ than control group participants at post-intervention follow-up (d = −0.97, 95% CI −1.81 to −0.12),209 while children taking part in the PMTO intervention also had lower carer-rated scores than a control group for externalising problems at 6-month follow-up (d = −0.33, 95% CI −0.66 to 0.00). 163

Finally, evaluation of KEEP reported that a lower proportion of children were rated by their carers at 6-month follow-up as having ‘rule breaking’ scores on the CBCL measure that exceeded the ‘borderline’ threshold (d = −0.293, 95% CI −0.57 to −0.01). 29 However, no differences were found between groups for children who reached the ‘clinical’ cut-off threshold or when continuous scores were compared. Similarly, no differences were found in carer ratings of total externalising behaviour problems or aggression.

Long-term outcomes

A total of 12 interventions, with 11 RCT evaluations and 1 QED evaluation, showed no effectiveness for reduced externalising behaviour problems at long-term follow-up. 35,118,131,157,162,163,173,176,179,186,187,209

Two interventions were effective at reducing externalising behaviour problems. Young children who received FIAP self-rated their externalising behavioural problems (d = −0.43, 95% CI −0.82 to −0.05) and aggression (d = −0.42, 95% CI −0.80 to −0.03) as being significantly lower than control group children at 18-month follow-up. There were no differences between groups for self-rated delinquency and no group differences for any of the measures when assessed independently by carers at 18 months. 170 The CARE intervention reported a significant longitudinal improvement in the slope of incidents relating to aggression towards residential care staff among foster children who attended residential homes whose staff had been trained in trauma-informed care (programme effect: β = 0.87, 95% CI 0.81 to 0.94).

Meta-analysis for shorter- and longer-term outcomes

A total of 23 interventions, with 24 associated RCT evaluations, were eligible for inclusion in the meta-analysis of externalising problems. Six RCT evaluations were ineligible: one reused analysis from another included evaluation,210 two evaluations used an active control group158,181 and three evaluations presented insufficient information to calculate effect sizes. 35,118,173 We undertook separate meta-analyses to investigate the effect of interventions on externalising problem behaviours measured at short-term (0–6 months) and long-term (> 6 months) follow-up. Collectively, interventions significantly reduced externalising problem behaviours in the short term (d = −0.30, 95% CI −0.53 to −0.08; Figure 13). The analysis included 44 effect sizes from 18 evaluations, with substantial between-studies heterogeneity (I² = 73%).

FIGURE 13.

Forest plot for meta-analysis of total externalising problems (short term).

There was, however, no evidence that interventions reduced externalising problem behaviours when evaluated at longer-term follow-up (d = 0.02, 95% CI −0.17 to 0.20; Figure 14). For longer-term follow-up periods, we included 19 effect sizes from 9 evaluations and observed moderate heterogeneity (I² = 45%).

FIGURE 14.

Forest plot for meta-analysis of total externalising problems (long term).

Anxiety and depression

Some 12 interventions, with 12 associated RCT evaluations, reported outcomes relating to anxiety and depression. They were most commonly assessed via the Revised Children’s Manifest Anxiety Scale or CBCL by self-report from children and young people (see Report Supplementary Material 14).

Shorter-term outcomes

Five interventions, with five RCT evaluations, demonstrated no effectiveness for reducing anxiety and/or depression in children and young people. 135,142,143,169,218

Three interventions showed some short-term effectiveness. Children in the PMTO intervention were significantly less likely than control peers to reach the ‘borderline’ threshold on the withdrawn and depressed subscale of the CBCL, as assessed by carers (d = −0.35, 95% CI −0.61 to −0.10). However, no group differences were observed for the ‘clinical’ threshold or differences in continuous scores and there were no group differences for scores on the anxiety and depression subscale. 29 Adolescents in residential foster care receiving the ‘affective’ arm of the cognitive and affective bibliotherapy intervention had significantly lower levels of clinician-assessed scores on the anxiety and depression subscale of the teacher report form compared with controls at 6-month follow-up (d = −0.73, 95% CI −1.30 to −0.17). However, there were no differences compared with control on several other anxiety measures for either the ‘affective’ or ‘cognitive’ intervention groups, at either 3- or 6-month follow-up points. 168 Finally, young children in foster care whose carers had taken part in the CARE intervention were reported by carers to have significantly lower anxiety scores at 3-month follow-up (d = −1.59, 95% CI −3.03 to −0.14). No differences were found for the same outcome at 1-month follow-up, or for depression scores at either follow-up point. 33

Longer-term outcomes

Three RCT evaluations of three interventions showed no effectiveness for reducing depression and/or anxiety in children and young people. 27,157,170

Young adolescent children in foster care who took part in the Take Charge mentoring intervention showed significantly lower scores at long-term follow-up in a series of measures: carer-rated anxiety and depression scores at 12-month (d = −0.65, 95% CI −1.02 to −0.29) and 18-month (d = −0.36, 95% CI −0.71 to −0.00) follow-up; carer-rated withdrawal and depression scores at 12-month (d = −0.41, 95% CI −0.77 to −0.05) and 18-month (d = −0.37, 95% CI −0.73 to −0.01) follow-up; and self-rated anxiety and depression at 12-month (d = −0.44, 95% CI −0.79 to −0.08) follow-up. 159

Meta-analysis of shorter-term outcomes

Eight interventions, with eight RCT evaluations, were eligible for inclusion in the short-term meta-analysis of anxiety and depression. There were insufficient available effect sizes to conduct a meta-analysis of long-term (> 6 months of follow-up) intervention effects. Collectively, interventions significantly reduced scores on measures of anxiety and depression in the short term (d = −0.26, 95% CI −0.40 to −0.13), relative to control groups (Figure 15). The analysis included 40 effect sizes from 8 evaluations, with minimal between-studies heterogeneity (I² = 8%).

FIGURE 15.

Forest plot for meta-analysis of anxiety and depression (short term).

Suicide-related outcomes

Three interventions, with two RCT evaluations and one QED evaluation, assessed suicide-related outcomes. One RCT evaluation of a peer mentoring intervention to reduce teenage pregnancy-assessed attempted suicide,27 and a second musical therapy intervention measured self-reported suicidal ideation. 169 Both evaluations also measured self-harm episodes. No significant group differences were noted. One other QED study of the CARE intervention measured self-harm incidents and found no significant intervention effect. There were insufficient data available to conduct meta-analysis for this outcome domain. 191

Subjective well-being

Three studies reported five equity harms related to subjective well-being, which were all related to Fostering Healthy Futures (FHF) . Evaluations reported that there were no differential effects for a number of sociodemographic and care placement characteristics: children and young people’s gender; ethnicity; intelligence quotient (IQ); baseline mental health problems; quality of relationship with foster carers; placement type (foster care vs. kinship care). 115,147

Three study reports linked to the same evaluation examined the interaction of history of maltreatment and quality life. One study report found that FHF had a stronger effect for children exposed to fewer adverse childhood experiences (ACEs). 147 This was a composite measure of physical abuse, sexual abuse, removal from a single parent household, exposure to community violence, caregiver transitions and school transitions. This finding was not replicated in other analyses. 111,147 There was indication that the intervention was more effective for children with poorer relationship quality with their birth parents and for those who have had fewer caregivers. 115

Mental, behavioural and neurodevelopmental disorders

A total of 8 interventions, with 14 associated study reports, considered equity harms for mental, behavioural and neurodevelopmental disorders.

Children and young people’s age

Four studies reported on the moderating effect of children and young people’s age on mental, behavioural and neurodevelopmental disorders (Figure 16).

FIGURE 16.

Equity harms for mental health, behavioural and neurodevelopmental disorders by children and young people’s age. Notes: Price et al. (2015)182 (¼ bar); DeGarmo et al. (2013)103 (¼ bar); Smith et al. (2011)183 (¼ bar); Dozier et al. (2006)171 (4/4 bar). Height of bars represent the nature of the evidence presented by the trial: full height – significant moderation for a given outcome; three-quarter height – pattern of moderation estimates including some significant moderation; half height – non-significant moderation trending in one direction; one-quarter height – only non-significance reported. Shading of bars represent time to outcome at follow-up: blue – short-term outcomes of < 6 months post baseline; red – long-term outcomes of 6 months or more post baseline.

One study reporting an evaluation of ABC found an interaction effect. 171 The intervention was more effective for toddlers (18–36 months) compared with infants (0–17 months) in reducing total social, emotional and behavioural problems. Three studies reported no interaction effect for total social, emotional and behavioural problems, internalising and externalising problems. 171,182,183

Children and young people’s gender

Three study reports examined equity harms related to children and young people’s gender, which evaluations classified according to the binary of male and female (Figure 17). None of the studies reported differential effects for total social, emotional and behavioural problems, attention deficit hyperactivity disorder (ADHD), externalising problems, internalising problems, PTSD or dissociation. 103,147,176

FIGURE 17.

Equity harms for mental health behavioural and neurodevelopmental disorders by children and young people’s gender. Notes: DeGarmo et al. (2013)103 (¼ bar); Taussig et al. (2019)147 (¼ bar); Marquis (2013)176 (¼ bar). Height of bars represent the nature of the evidence presented by the trial: full height – significant moderation for a given outcome; three-quarter height – pattern of moderation estimates including some significant moderation; half height – non-significant moderation trending in one direction; one-quarter height – only non-significance reported. Shading of bars represent time to outcome at follow-up: blue – short-term outcomes of < 6 months post baseline; red – long-term outcomes of 6 months or more post baseline.

Children and young people’s ethnicity

Three studies explored the moderating effect of children and young people’s ethnicity on mental, behavioural and neurodevelopmental problems (Figure 18). Study reports tended to classify populations according to white and non-white participants. None of the studies found an interaction effect in relation to total social, emotional and behavioural problems, social-emotional functioning, PTSD or dissociation.

FIGURE 18.

Equity harms for mental health behavioural and neurodevelopmental disorders by children and young people’s ethnicity. Notes: Taussig et al. (2019)147 (¼ bar); Price et al. (2015)182 (¼ bar); Akin et al. (2019)167 (4/4 bar). Height of bars represent the nature of the evidence presented by the trial: full height – significant moderation for a given outcome; three-quarter height – pattern of moderation estimates including some significant moderation; half height – non-significant moderation trending in one direction; one-quarter height – only non-significance reported. Shading of bars represent time to outcome at follow-up: blue – short-term outcomes of < 6 months post baseline; red – long-term outcomes of 6 months or more post baseline.

Children and young people’s baseline mental health problems

Six study reports, linked to five interventions considered the moderating effect of baseline mental health problems on mental, behavioural and neurodevelopmental disorders (Figure 19). Two evaluations of MTFC indicated that the intervention was more effective in impacting total social, emotional and behavioural problems among high-risk children, who were classified as those whose carers reported six or more problem behaviours each day. 117,122 To note, in one of these evaluations, there was variance in the differential impact on social and emotional functioning depending on the outcome measurement tool used. 122

FIGURE 19.

Equity harms for mental health behavioural and neurodevelopmental disorders by baseline mental health problems. Notes: Smith et al. (2011)183 (¼ bar); Taussig et al. (2019)147 (¼ bar); Akin et al. (2019)167 (4/4 bar); Biehal et al. (2012)135 (¼ bar); Chamberlain et al. (2008)117 (4/4 bar); Linares et al. (2006)175 (¼ bar). Height of bars represent the nature of the evidence presented by the trial: full height – significant moderation for a given outcome; three-quarter height – pattern of moderation estimates including some significant moderation; half height – non-significant moderation trending in one direction; one-quarter height – only non-significance reported.

Four studies reported no interaction effect: total social, emotional and behavioural problems or functioning,147,167 externalising problems,174,183 internalising problems,183 symptoms of PTSD147 or dissociation. 147

Suicide-related behaviours

No studies reported moderation analysis or interactions effects for outcomes related to suicide-related behaviours.

Contextual characteristics

Only six interventions, with six associated study reports, referenced relevant contextual factors. 124,127,130,131,148,163 This lack of detail was one of the key reasons why they were classified as thin-process evaluations.

Geography was considered in one study report, with issues around the rurality of communities providing a barrier to accessing services. 131

The wider economic context was reported as a relevant factor in one study report, where a major economic recession in the USA led to budgetary constraints and a reduced willingness and capacity among social welfare agencies to implement MTFC. 124 The organisation and resourcing of social welfare systems was considered important in another evaluation of a solution-focused parenting groups intervention, where the structurally restricted and isolated role of the foster carer profession gave a sense of limited autonomy, which could compromise implementation. 148 An evaluation of PMTO addressed the importance of ‘usual care’ when the intervention was transported to the Netherlands from the USA, with the authors reflecting that standard practice is comparatively high in Northern Europe relative to the USA and could therefore potentially mask intervention effectiveness. 163

There were also reported sociocultural factors in relation to the transportation of interventions to new populations and countries. 127,130 Evaluation of Incredible Years in Ireland, an intervention transferred from the USA, ascribed challenges with implementation and acceptability to uniquely local issues, including interprofessional working and the historic non-engagement of biological families in social work. Local teams also reported concerns about the suitability of American-style language and activities for an Irish audience. 130

Implementation

A total of 19 interventions, with 25 associated study reports, reported data on implementation. 34,108,114,123–125,127,128,130,131,133,135,137,140,143,147,148,152,157–160,162,163,221 These data mapped to two categories: recruitment and retention, and fidelity to the intervention. 13 interventions, with 15 study reports, provided data on intervention recruitment and retention, primarily in relation to parents and carers. 108,114,123,127,128,130,131,135,140,143,147,158–160,221

A key issue in recruitment was capacity among targeted carers. Carers often had too much work, too many children, family illness or no interest in the intervention. 127 Foster carers found training in parenting skills as being inconsistent with their role as a professional caregiver, which contributed to drop out. 130 There was also reference to challenges of interprofessional working,130,140,158 including a reliance on CAMHS or social workers for referrals in the UK. 140,158

Retaining participants in interventions was another issue, with notable barriers to retention being children and young people moving care placements or areas,131,143 carer illness221 or inability to secure childcare arrangements. 221 Some interventions worked to retain children and young people by offering transportation, dinner, respite care or payment. 135,147 One intervention promoting mindfulness did not explicitly advertise the programme as an intervention, rather an opportunity to meet peers socially, which the intervention team thought would improve take-up and retention. 135 Retention rates among subgroups of intervention participants were not routinely reported, but the evaluation of the PCIT found that African American carers were more likely to drop out of the intervention. 123

A total of 17 interventions, with 22 linked study reports, considered intervention fidelity. 34,108,114,123–125,128,130,133,137,140,143,147,148,152,157–160,162,163,221 Generally, interventions reported a high level of fidelity,128,140,143,147,148,158,162,221 although one study report indicated issues with therapists not delivering all intervention components, which correlated with carers not completing follow-up assessments. 140

Two study reports, related to the KEEP and MTFC, explored implementation models to support the intervention. One assessed whether or not a ‘train the trainers’ model could achieve equal fidelity compared with a model where the intervention developers provided training, but reported no difference in fidelity. 125 A second study assessed differences in fidelity when delivering standard MTFC implementation training, compared with standard training plus support from two consultants, finding that the involvement of consultants was linked to higher levels of fidelity. 124

Barriers to fidelity were explored in several studies. In one intervention aimed at supporting children and young people’s relationships with birth parents, parents reported not having enough time during visits with their child to implement newly acquired parenting skills. 130 For carers, the solution-focused parenting groups approach reported continued issues with foster carers feeling too restricted in their role to change young people’s disruptive behaviours. 148 Meanwhile, a qualitative exploration of researchers’ and practitioners’ experiences of TFC suggested that carers struggled to balance their caregiver role with what they saw as their ‘professional’ role, and the need to develop a productive relationship with their social worker. 137 Carers also identified issues relating to finding childcare for other children and balancing the needs of the care-experienced child and biological children. 133,134 At the organisational level, perceived barriers included mental health needs being only one of numerous priorities being addressed by child welfare social workers, which could act as a barrier to timely and appropriate referrals. 158

There were perceived facilitators to implementation fidelity, which included flexibility in the frequency and timing of sessions to accommodate participants’ needs. 140,221 Suggestions to improve fidelity to parenting programmes were explored. For TFC, one suggestion was to develop carers’ understanding and knowledge of how to navigate the broader child welfare system and the adoption of a strengths-based approach to support them. 137 The evaluation suggested this could be improved through experiential training, peer support and ongoing skill building.

Acceptability

A total of 13 interventions, with 16 linked study reports, considered intervention acceptability. 108,127,128,130,131,133,135,140,142–144,148,157,158,162,221 Of these, 5 interventions with 5 study reports presented data on acceptability to children and young people,131,135,142,143,158 7 interventions, with 11 study reports, presented acceptability to parents and carers,108,128,130,131,132,140,144,148,157,158,162,221 and 5 interventions, with 6 study reports, considered acceptability to delivery agents and related stakeholders. 127,130,131,133,140,162 Evaluations tended to have limited descriptions of acceptability, with many confined to quantitative assessments of satisfaction.

Overall, levels of acceptability were high across stakeholders. 127,130,131,134,140,142,143,162,204 Qualitative data reported that children and young people valued the relationship-building element of interventions that included working with local people they were familiar with, working with others from similar backgrounds and working with clinicians for mental health support. 131,135

The value attached to relationships was echoed among parents and carers who welcomed the way interventions facilitated new relationships for young people. Carers also appreciated intervention components that provided opportunities for their own relationship development. This included opportunities for peer support, reflection and discussion. 128,130,131 They also valued developing and expanding their parenting skills. 148 Delivery agents recognised that interventions provided additional value to existing services in a way that helped reduce placement breakdowns. 130,140

For parents and carers, challenges to acceptability often related to their identity and relationships. In one intervention, kinship carers found it hard to fit into group sessions alongside other types of carers due to different relationships with biological families. 133 In another intervention, acceptability was impacted by some carers in the sessions feeling criticised for their parenting approaches. 140 When training sessions ended, carers reported struggling to implement newly acquired skills and knowledge as they juggled different parenting strategies for foster children and biological children. 133 Both children and young people, and parents and carers, felt that their intervention experiences could have been enhanced with longer intervention timescales to further develop skills and build relationships. Suggestions included follow-up support, refresher sessions or more permanent, ongoing provision. 131,148

For those delivering interventions, acceptability issues mapped to challenges in delivering interventions in the home setting and, in some instances, the difficulties associated with the emotional burden of managing interventions without adequate training and supervisory support. 131

Study characteristics

Study reports were published between 2003 and 2021; 9 interventions, with 14 associated study reports, were evaluated in the context of the UK and Ireland. 27,118,120,122,126,136,138,139,149–151,154,155,215 Six interventions, with eight study reports, were from the USA. 129,132,141,145,146,156,164,223 One study report, with one intervention, was from Australia. 153

Quality assessment

We quality appraised rich study reports with an adapted qualitative appraisal tool84 (see Report Supplementary Material 19). We made 21 assessments, as 3 study reports were related to the same evaluation of a single intervention. Overall scores were assessed for reliability and usefulness.

For overall reliability, 2 study reports were assessed as having high reliability, 14 as medium, 4 as low and 1 as unclear. For overall usefulness, 5 study reports were assessed as having high usefulness, 14 as medium and 2 as low. Only one of the studies was rated highly in both categories150 and two were rated low. 141,215

There were a number of methodological limitations across the evaluations. In terms of reliability, assessment of sampling indicated widespread challenges in engaging care-experienced children and young people, in addition to their parents and carers. Studies tended to rely on a pragmatic, convenience approach to sampling and, on occasion, reported approaching populations based on ease of access. 120 There were efforts to sample diverse care-experienced populations, but this was sometimes compromised by participant drop out.

The grounding of findings in the data was variable. Higher-rated study reports demonstrated a clear link between qualitative data, interpretation and conclusions. They presented a range of quotes from participants, and the provenance of each quotation was clearly labelled. In lower-rated study reports, there was limited linking of data and the findings. It was often unclear how representative quotations were, usually as a result of non-specific labelling.

In reference to usefulness, there was a general issue with the under-representation of children and young people’s voices. A number of evaluations made a concerted effort to engage with this group,27,120,132,145 but they were clearly missing from other study reports. In some instances, this was due to the evaluation focusing on maximising implementation and intervention functioning, and hence a focus on delivery agents as study participants. 129,141,153 In this case, we assessed usefulness as not applicable. In other study reports, evaluations explored the experiences of intervention participants, which tended to involve a focus on carers. While children and young people were not the direct target population of these interventions, interventions did assess outcomes related to children and young people’s mental health and well-being, and so it may have been appropriate to include their voice. Accordingly, these studies were rated low for usefulness in relation to children and young people’s voices. Furthermore, while some interventions were focused on younger-aged children (e.g. children aged under 5 years) who may have been more difficult to engage, these studies were also assessed as low where there was limited or no consideration of potential opportunities to secure their participation.

Context factors as enablers and inhibitors of implementation and acceptability

The synthesis of qualitative data from the rich-process evaluations generated three overarching themes, or context factors, with five associated domains. These themes explore how key dimensions of context shape intervention implementation and acceptability.

Overview of intervention types and evidence types

We identified 64 separate interventions, with 124 associated study reports. Of these reports, 24 described the intervention’s programme theories, 50 were process evaluations, 85 were outcome evaluations and 1 was a partial economic evaluation. The majority of the 124 studies described interventions that were delivered and evaluated in the USA (n = 77). Of the 64 interventions, there was a clear focus on approaches that target the skills and knowledge of young people (n = 9 interventions), emphasise improvement of carer parenting practices and young people’s relationships (n = 26 interventions), or a combination of the two (n = 15 interventions). Of these interventions, 13 included an explicit programme theory, with 12 prioritising theories that operate within the interpersonal domain (e.g. relational) of the socioecological model. Interventions primarily targeted mental health, behavioural and neurodevelopmental disorders, with 48 interventions assessing children and young people’s total social, emotional and behavioural problems and 26 targeting children and young people’s externalising problem behaviours. Only 11 interventions targeted subjective well-being, with 4 targeting suicide-related outcomes.

Overview of outcome synthesis

We synthesised evidence from 44 RCTs of 35 interventions that evaluated the effectiveness of interventions for improving children and young people’s subjective well-being, mental, behavioural and neurodevelopmental disorders and suicide-related issues. Synthesised evaluations predominantly related to interventions that operated across the intrapersonal and/or interpersonal domain, and were largely conducted in the USA or UK.

Meta-analyses of RCT evaluations showed that interventions collectively improved short-term outcomes (those measured at 0–6 months post baseline) across several mental, behavioural and neurodevelopmental disorder domains. Using GRADE evidence assessments, we judged there to be low or very low certainty for short-term effectiveness in these outcome domains, as a result of concerns relating to RoB and/or imprecision. For longer-term outcomes (> 6 months post baseline), meta-analyses showed no significant effectiveness across any of the assessed mental, behavioural and neurodevelopmental outcome domains. Combined with concerns regarding RoB and imprecision, our GRADE evidence assessment indicated very low certainty that interventions are effective in the longer-term for these child-level outcomes.

Quasi-experimental design evaluations of community-level (n = 2) and organisational-level (n = 1) interventions showed mixed results, with some limited evidence of longer-term improvements in child-level functioning and emotional/behavioural regulation where care agencies receive specialist trauma-informed training.

There were few evaluations that looked at the effectiveness of interventions on children and young people’s subjective well-being outcomes or outcomes relating to suicide and self-harm.

Fourteen evaluations provided moderator analyses or interaction effects that allowed us to assess whether interventions might deliver adverse or inequitable effects. Evidence was mixed, with the majority of evaluations showing no moderating effects. There was some limited evidence that interventions may work more effectively for those with more baseline mental health problems or lower exposure to maltreatment.

Explaining outcomes findings: context factors and process evaluation synthesis

We classified process evaluations as either conceptually and/or empirically thin (n = 27) or conceptually and/or empirically rich (n = 23). Thin evaluations, which were linked to outcome evaluations, tended to have limited data availability that was generalisable beyond the immediate evaluation context. Rich evaluations presented generalisable findings that had explanatory power beyond the proximate evaluation context. Thin-process evaluations generally demonstrated that interventions reported high levels of fidelity, in terms of the quantity and quality of intervention delivery. Equally, acceptability was reported as largely positive across a range of stakeholder groups, although the perspectives of children and young people were largely under-represented. However, there were potential implementation challenges reported in terms of recruitment and retention to interventions and the evaluation study. This was a particular issue for parent and carers who struggled to manage the intervention with a range of other commitments.

The synthesis of rich-process evaluations progressed understanding of wider context factors that might help to explain potential issues around implementation and acceptability. We identified five key context factors: (1) lack of system resources for intervention training and implementation, partly because health promotion is not ‘core business’; (2) intervention burden, which encompasses the time, cognitive and emotional burden associated with implementation and participation. Importantly, young people engaged in mentoring interventions cited concerns about burdening their mentor and hence feeling disinclined to seek support;145 (3) interprofessional relationships, where historical and ongoing tensions between health and social care professionals can inhibit interventions that are reliant on effective multiagency communication and collaboration; (4) care-experienced young people’s identity, where their systematic disenfranchisement can mean they feel unable to express dissatisfaction with an intervention, which might encourage disengagement and (5) carer identity, where interventions may not engage carers as they feel their expertise and history of parenting practices are not valued, creating misalignment between interventions and the wider contexts of their lives.

Through integration of the outcome and process evaluation data, we sought to explain the outcome synthesis. At the level of the individual intervention, there was no clear trend between implementation, acceptability and effectiveness. From the thin-process evaluations, interventions were largely implemented with high levels of fidelity and acceptability whether they were effective or not. However, these thin-process evaluations tended to focus on the immediate period of intervention training. For example, they assessed carers attendance and learning at parenting training courses, rather than carers’ longer-term delivery of their newly acquired skills sets within the real-world contexts of their lives. In contrast, the rich-process evaluations offer insight into the structural challenges to intervention, which may compromise longer-term implementation. This can include the potential incongruence of learning from parenting training with carers’ own values, practices and commitments within their wider family and community,136 which can impede delivery and reduce acceptability. These context factors may help explain the key findings from the outcome meta-analysis, namely that interventions targeting domains of mental, behavioural and neurodevelopmental disorders are effective in the shorter term (0–6 months) but not the longer term (> 6 months).

Overview of economic evaluation

We included one partial economic analysis. Evaluation in Scotland examined the potential costs and consequences of implementing a New Orleans intervention model in Glasgow, comparing estimated costs with estimated costs of the current foster care system (usual care). The study reported that although the New Orleans model would incur greater costs due to being more resource intensive (i.e. involving more healthcare professionals, such as psychologists and psychiatrists), it would significantly reduce the length of stay and probability of return to care. As such implementation could lead to overall savings per child, with a reduction to £88,653 from the current £95,473.

Overview of stakeholder consultations

We conducted seven stakeholder consultations between April and September 2022. The details of the stakeholder groups and structure of consultations is presented in Table 7. Consultations with care-experienced young people were conducted through CASCADE Voices and the Fostering Network in Wales Young Person Forum. Recruitment of participants to these consultations was challenging, reflecting widely reported barriers to engagement with this population during the COVID-19 pandemic. 224 Consultations with experienced foster carers were conducted through the Wales Foster Carer Advisory Forum.

Four stakeholder groups were conducted with policy-makers, professionals and practitioners. First was the Public Health Wales National Health Service (NHS) Wales National Safeguarding Network, which is an independent team of doctors, nurses and a National general practitioner (GP) lead, it provides strategic focus and professional direction to NHS Wales in promoting the welfare and safeguarding of children, particularly those looked after by their local authority. Second was the Welsh Government Social Services and Integration Directorate, which included policy staff with remit for care-experienced children and young people. Third was the Welsh Medical Group, which is an Association for Fostering and Adoption Special Interest Group funded by the Welsh Government. It comprises medical advisors operating in Wales with a role in assessing the well-being of care-experienced children and providing medical advice to local authorities and other adoption and fostering agencies. Fourth was the Public Health Wales Looked-after Children Steering Group, which has clinical representatives from all local health boards in Wales and aims to support health boards and trusts to deliver best practice in improving outcomes for care-experienced children and young people.

Consultations lasted between 30 and 90 minutes as, in some instances, they were part of a wider programme of activity. A member of the CHIMES research team provided a 15- to 30-minute PowerPoint® (Microsoft Corporation, Redmond, WA, USA) presentation outlining key findings from the review (see Report Supplementary Material 21). The structure, language and length of each presentation was adapted to each stakeholder group. Stakeholders were asked to: (1) Consider key context factors identified by the review as influencing implementation and acceptability and reflect upon the relevance of these factors to the UK context. We asked stakeholders if there were additional context factors that should be attended to in future intervention development and research. (2) Appraise the evidence-based interventions identified by the review for potential acceptability, feasibility and effectiveness in the UK context. This discussion point was underpinned by the aims of stage 7 of the TRANSFER model, which recommends discussing the transferability of review findings with stakeholders. 72 Where stakeholders positively appraised an intervention, we asked them to consider potential adaptations that would be required to ensure intervention-context fit in the UK. (3) Identify potential de novo intervention development, including priority intervention types, theories and outcomes.

Note that, while each consultation followed a similar structure, in practice not all discussion points were considered by each group to the same degree. There was variation in the extent of elaboration, explanation and endorsement of different ideas. For example, the young people’s group discussed context factors to a lesser extent. Differences between groups was partly a consequence of the length of the consultations, the fact that different facilitators guided discussions (e.g. the CASCADE engagement manager was led by young people’s interests and priorities) or that stakeholders focused on context factors and interventions more directly related to their own experiences and needs (e.g. foster carers emphasised structural barriers to supporting young people’s mental health and appraised the merits of parenting programmes).

In drawing together, the consultations, we compared and contrasted discussion both within and across groups. The final themes from the discussion were verified against the summary notes, checking for key negative cases. Overall, there was no significant discrepancies between the groups. Rather they had different emphasis on what they felt should be the priority moving forward.

Key context factors and transferability to the United Kingdom context

Consultation groups were presented with an overview of the five key context factors identified by the process evaluation synthesis and the context in which the intervention was delivered and evaluated (Table 8). These were: (1) system resources available for implementation and competing demands; (2) intervention burden; (3) system culture and interprofessional working; (4) the systemic disenfranchisement of children and young people’s identity, values and needs, meaning they feel unable to express dissatisfaction and (5) carer identity values and need, linked to the misalignment between intervention requirements and the wider context of their lives. Most interventions contributing to the context factors were delivered and evaluated in the UK (n = 8 interventions). Table 9 presents the context factors that were discussed as relevant by each group following presentation of all five factors.

Overall, stakeholders identified the need to attend to country-level variations in context, indicating that there may be challenges in adapting international evidence-based approaches, particularly from the USA to the UK. Of note, the Foster Carer Advisory Group reflected on the potential difficulties in transporting parenting interventions from the USA,138,196 due to differences in language and style, variations in parenting practices, and different sets of relationships and resources within the respective social care systems.

It should also be noted that in the first phase of consultations with care-experienced young people, stakeholders queried why the review was considering the transportability of interventions, maintaining that interventions should be co-produced with young people in the UK, as they are frequently devalued and neglected by current social care provision. This reflected a wider limitation with the interventions identified in the present review, namely the non-prioritisation of care-experienced young people’s mental health and well-being and their lack of involvement in the development of relevant interventions and their underpinning theory of change.

Stakeholder groups largely endorsed the five context factors, stating that they reflect many of the central issues influencing service implementation and acceptability in the UK. Across the majority of professional stakeholder groups, participants agreed that a lack of system resources would be the most significant barrier to any intervention delivery. Foster carers in particular maintained that the social care system in the UK is largely ‘broken’, with a lack of support for social workers who are managing complex caseloads and a lack of support for carers who are often managing challenging placements. They suggested that local councils were under increased pressure to return budget underspends to national governments. There were also reported issues around social workers having to constantly ‘firefight’ because social care and CAMHS operate below capacity due to recruitment challenges.

In one of the young person’s groups, participants suggested that there was a higher ratio of children to carers in the UK compared with the USA. With ever increasing numbers in care, this could limit capacity to take on additional interventions. While less specific in nature, Welsh Government stakeholders observed that there were extensive programmes and policies in place to support the mental health of care-experienced young people, but there was an ongoing implementation gap where they were not being taken up. Further work would be needed to consider if any additional resource would be needed to address this issue.

The two young people and one foster carer group emphasised issues around intervention burden. Young people reflected on mentoring-based interventions, observing both the cognitive and emotional burden this could place on the individual mentor when dealing with a child’s difficult experiences, so the mentor would also require supervision. There were questions whether this extra support could be provided. Foster carers mentioned the time and cognitive burden of constantly testing new interventions, which rarely translates into long-term change and improved outcomes for children in their care.

Interprofessional working was discussed in depth by three stakeholder groups (foster carers, Looked-after Children Steering Group, Welsh Medical Group). Foster carers characterised the social work system as ‘risk averse’ and ‘legalistic’, with different professionals conservatively working in their own discrete area without any creativity in collaboration. Carers felt that this translated into carers being mistrusted and receiving stringent deadlines and statutory targets to satisfy professionals in other parts of the system. They expressed a desire for an organisation or the government to take oversight and co-ordinate the system more effectively.

The final two context factors of identity were explored by four of the stakeholder groups (two young people’s groups, foster carers, Welsh Government policy stakeholders). In terms of young people, policy stakeholders noted that drop out from interventions was high and somehow there were challenges in engaging this population. In terms of carer identity, there were concerns among foster carers about the extension of their remit into a therapeutic role where they did not have sufficient skill or training. More significantly, they reported feeling undervalued and deskilled within the system, and that other professionals consistently failed to listen to them. They also reflected on the challenge of practically co-ordinating meetings where carers, social workers and other professionals could all attend. As such, while there was agreement that parenting courses could be helpful, there was a sense that there needed to be clear boundaries to the carer identity and for them to be supported by a culture change in the system.

Stakeholder perspectives on evidence-based interventions

As part of the consultations, we shared an overview of a range of intervention types, theories and specific packages with stakeholders that mapped across the socioecological model. Initially, as per RQ5, we intended to construct an overarching candidate programme theory to present to stakeholders. However, the findings of the review suggested that this was not feasible or appropriate as theories were underspecified, there was limited evidence of combining theories and components, and initial stakeholder feedback indicated that particular approaches (e.g. parenting classes) could be polarising in terms of acceptability and feasibility to different stakeholder groups. As such, we decided to present individual interventions.

The specific interventions presented to stakeholders indicated largely favourable evidence of effectiveness in the review, although for more complex organisational and community interventions, the evidence base was more mixed. The sampling of interventions was also partly informed by 2021 NICE guidance recommending further development and evaluation of mentoring, parenting and interprofessional collaboration interventions to address the mental health and well-being of care-experienced young people. 21 The presented interventions were:

• Intra/interpersonal:

  • Theories: attachment theory;225 positive youth development;226 resilience; social learning theory227

  • Example interventions: FHF; PMTO

• Interpersonal:

  • Theories: attachment theory;225 coercion theory30,101,102,107 positive youth development;226 resilience; social learning theory;227 parental management109

  • Example interventions: KEEP; ABC

• Organisation/community:

  • Theories: attachment theory;225 trauma-informed practice;34 generic system change

  • Example interventions: FIAP; TST; CARE

Stakeholder perspectives on de novo intervention development

Following exploration of evidence-based interventions identified by the review, stakeholders subsequently considered new intervention approaches that they would recommend for development, implementation and evaluation within the UK context.

As a starting point, stakeholders identified principles to underpin future intervention development. First, reflecting findings in the process evaluation synthesis theme on care-experienced young people’s identity is that individuals in care should not be treated as a homogeneous group. Rather there should be sensitivity and flexibility in responding to their needs at a time that suits them. Young people from CASCADE Voices stated that interventions need to be tailored to the stage of the individual care journey, maintaining that support should differ for a young person who is just entering care, someone who is struggling in their placement and a care leaver who is now living independently. This resonates with reports in the process evaluation synthesis, where intervention participants emphasised the need for intervention adaptability and scope for tailoring to meet local needs. 146,153 Second, and primarily considered by policy stakeholders, is the need to develop something scalable that can potentially be delivered on a regional or national scale. In this case, stakeholders from the NHS Safeguarding Network cited the importance of issues around workforce capability and capacity to deliver interventions at scale.

In terms of theory, young people stakeholder groups wanted a clear emphasis on early support and prevention to supplement existing provision around specialist mental health treatment. Building on reflections on theories that underpin existing approaches, stakeholders recommended approaches that promote positive relationships, which may link to attachment theory,225 positive youth development,226 resilience and social learning theory. 227 They also wanted attachment and trauma-awareness integrated into the system more widely. 34,119,170,191

Stakeholders recommended two types of interventions: mentoring, primarily by care-experienced young people and trusted adults, and promotion of interprofessional working and system change. In practice, these recommendations built upon discussion of existing intervention packages and so, while participants suggested de novo intervention work, it is possible that adaptation of the evidence-based approaches identified by the review may be feasible and more resource efficient.

The first type of intervention described models of mentoring. Participants suggested that mentoring should start from the age of 10 years, when children may become more aware of some of the structural disadvantages of being in care and may be developmentally ready to engage in social and emotional skills development. Young people’s stakeholder groups advocated for mentoring delivered by someone with extensive experience or knowledge of the care system (e.g. care leavers) so that they would understand potential issues and challenges. They felt that it was difficult to forge a positive and meaningful relationship with those who had not shared similar life events. Equally, as the mentoring models in the review were often unpinned by social learning theory,227 there was a sense that young people were less inclined to observe behavioural models from individuals to whom who they could not relate, or they may lack the belief that they can replicate certain modelled actions due to their structurally disadvantaged position of being in care. Participants further identified a range of skills and characteristics that they would like a mentor to have, including understanding, trustworthiness, honesty, compassionate nature and ability to speak to the young person as an equal.

Stakeholders explored how mentoring relationships could fit with the wider system of support. The two young people’s groups felt that mentoring could serve as early support while individuals were waiting for referrals to specialist mental health services, such as CAMHS. They also felt that other groups may be brought into the mentoring relationship where appropriate; for example, suggesting that foster carers may be involved in some later sessions to be aware of their child’s progress and needs.

The second type of intervention focused on addressing interprofessional relationships, which may be supported by upskilling professional groups involved in the care system. Stakeholders indicated that components could include training in trauma-informed or attachment-informed practice across local authority health, education and social care teams. It also extended to include recommendations for implementation models that facilitate multiagency wraparound provision for individuals in care, by having clearer and more co-ordinated communication channels, regular interprofessional meetings and potentially shared departmental budgets.

While stakeholder groups explored the potential for new intervention development, it should be noted that not all participants felt that the system needed a new approach. Indeed, the Welsh Government consultation discussion maintained there was a wealth of interventions already being funded and there was probably a gap between what young people perceive as being implemented and what is actually available to them. Stakeholders from the Welsh Medical Group reflected that there were already key policy efforts to integrate a systems approach into social care in Wales, as advocated by the Children’s Commissioner for Wales’ No Wrong Door report,228 which maintains a co-ordinated approach to supporting young people’s needs where all professional groups have a responsibility in working together to improve outcomes. As such the focus may need to be on addressing the implementation gap, and focusing on how awareness and access may be optimised moving forward.

Priority intervention types, theories and outcomes

Following consultations, we constructed two integrative matrices to map stakeholder preferences against the interventions included in the evidence base. 94 The first matrix mapped stakeholders’ priority intervention theories and types. The second mapped stakeholders’ priority outcomes. The aim of this approach was twofold. First, it demonstrated if interventions, and the associated evidence base, are addressing what stakeholders think is important and meaningful in terms of the mental health and well-being of care-experienced young people. Identifying incongruence in the evidence and priorities can help to refine the intervention and research agenda moving forward.

Second, it supplemented the process of identifying potential interventions to progress for future development, adaptation and evaluation. As part of the consultations, the emphasis was primarily on discussing a selection of approaches that demonstrated some evidence of effectiveness. However, we recognised that stakeholders could express a preference for theories or components that may not have been included in the consultation presentation but were eligible for inclusion in the CHIMES review. Constructing the matrix allowed us to iteratively return to the included interventions to identify if there were additional approaches that aligned with stakeholders’ priorities, but which may not have demonstrated effectiveness or even showed harm.

The process for constructing the matrices commenced with classifying priority intervention types, theories and outcomes from stakeholder data. This information was collated from study reports of interventions’ programme theories, thin- and rich-process evaluations; stakeholder consultations in the initial phase of study conduct, and stakeholder consultations during WP6 consultations. These were mapped to the interventions included in the review. Where interventions had been included in the outcome synthesis, we indicated the associated evidence base. They were categorised according to there being clear evidence of effectiveness, mixed evidence (i.e. effectiveness was not reported for all primary outcomes, was not consistent across reports (e.g. child and carers) or was not sustained over time) or no effectiveness. Similarly, we indicated where study reports included evidence that was not an outcome evaluation.

Methods for intervention development and adaptation

While the CHIMES review largely focused on the evaluation of interventions, it generated important insights into the process of intervention development and adaptation. These insights were primarily ascertained through the quality appraisal of intervention theory and process evaluations. Generally, there was a lack of use of relevant frameworks and models to support intervention development,61,98,229 specifically in relation to identifying and articulating the programme theory. Importantly, there was a dearth of reported engagement with stakeholder groups in theory development, specifically children and young people. This should be remedied moving forward to ensure that interventions are contextually relevant and respond to local stakeholders understanding of the problem.

The review also indicated potential issues with the transportability of interventions across contexts. 138,196 In these instances, interventions failed to replicate feasibility, acceptability or effectiveness. Given the predominance of interventions developed and evaluated in the USA, there remains a significant lack of knowledge of how interventions can function in different contexts and the most effective mechanisms for supporting their successful transfer. More work is required in this field to integrate adaptation frameworks into the process of transporting approaches to maximise the likelihood of positive outcomes in new contexts. 64

Recommended interventions for development and/or adaptation

Drawing together the results of the review-level mixed-method synthesis, combined with stakeholder consultations, we can make recommendations in regard to intervention theories, types and outcomes to progress to further intervention development, adaptation and evaluation in the UK context.

In terms of programme theory, it is challenging to identify a central set of theoretical tenets as interventions were largely under-theorised and stakeholder consultation emphasised intervention types rather than mechanisms. However, there were two clear sets of theories that may be taken forward. The first set are orientated to positive relationship-based mechanisms, as expounded by attachment theory,225 social learning theory,227 positive youth development226 and, less specifically, social network approaches. These theories have been a key feature of mentoring from both peers and significant adults. 25–27,111,112,115,116,118,147,154,155 The second set is focused on creating a shared working practice and more efficient interprofessional relationships and decision-making to provide more comprehensive and tailored support to families and children. This may be facilitated by creating a shared theoretical alignment across agencies, such as attachment. 119,191

Interestingly, there was some evidence for interpersonal interventions targeting parents and carers to promote positive parenting. These included a combination of theoretical approaches, from social learning tyheory108 to behavioural management. 109 Stakeholders advocated for parenting interventions more aligned with ‘positive’ approaches of social learning theory, drawing parallels with mentoring by significant adults, with less endorsement of parenting interventions that rely on behavioural management techniques.

Specific recommendations for intervention types to progress to development and evaluation include a focus on mentoring and system approaches for ethos change, which are also recommended by a 2021 NICE review. 52,54 For mentoring approaches, this could include new intervention development. However, there are US-based interventions that could be considered for adaptation to the UK setting, such as TAKE CHARGE or FHF.

Recommendations for potential adaptations include changing the delivery agents to individuals with experience or knowledge of care. Equally, mentoring and social network interventions in the UK have demonstrated a lack of feasibility, largely due to the five context factors identified by the process evaluation synthesis. Adaptation could require contextual restructuring to account for system-level characteristics. 27,118,154,155

For interventions operating across organisations and communities to realign ethos, draw together professional groups and increase resources, evidence of effectiveness has not been fully established. 34,119,187 CARE is the only candidate intervention we identified with a mixed evidence base, showing limited evidence of reducing self-harm in residential settings. 119,191 Equally evaluation of such approaches in the UK have largely focused on feasibility and piloting testing, demonstrating potential acceptability but identifying issues around implementation. 149,150,215 Again, these issues link to the five context factors from the review, especially regarding interprofessional relationships. Hence, while there may be acceptability for this type of approach, developmental work is required to ensure an approach that is responsive to the specific health and social care system in the UK.

Interventions need to target a wider range of outcomes, particularly those that are prioritised by care-experienced children and young people and other stakeholders. They should include a focus on subjective well-being and suicide-related outcomes. Few interventions currently target well-being, although it is being addressed by some mentoring programmes or interventions promoting social networks. 25,26,111,112,115,116,118,145,147,154,155 Equally, suicide-behaviour outcomes are rarely assessed, although they have been theorised as appropriate in interventions that modify organisational and interagency culture and ethos. 119,191

One of the RQs of the CHIMES review (RQ5) was to assess if prioritised theories, types, and outcomes could be combined into an overarching programme theory. Based on the interventions identified in the review, there is no currently available evidence for interventions that combine mentoring approaches with system-level interventions that focus on aspects of culture, multiagency working and interprofessional relationships. However, given the fact that process evaluations of mentoring intervention or social network building relationships report issues around the health of care-experienced young people not being ‘core business’, lack of resources and the systemic devaluing of young people, there is potential for organisational and community-based approaches to facilitate a culture that mitigates some of these structural barriers. 27,118,154,155 The identification of theories at the organisational and community level may be of import. These either sought to reconfigure organisational policies and practices to create a trauma-informed therapeutic ethos119,191 or delivered enhanced implementation models to facilitate the delivery of parenting interventions. 28,102,107,119

Evaluation study designs

Types of evaluation eligible for inclusion in the review were theory, process, outcome and economic. The inclusion criteria for the outcome study designs were RCTs or non-randomised evaluations. No quality threshold was used as part of the inclusion criteria for study design.

There were a number of limitations with existing approaches to evaluation that could be addressed to further strengthen the evidence base. First, there is a need to improve description and reporting of the study designs used in evaluation in this area. In particular, study reports often conflated feasibility testing with outcome evaluations. Relatedly, claims about effectiveness were often made where evaluations did not include a power calculation or were underpowered. As such it would be helpful for intervention studies to engage more clearly with frameworks that recommend and detail different evaluation study designs that are appropriate to different aims and RQs. 62

Second, there are study designs that are currently under-used, which, given the mixed evidence base, could helpfully be employed moving forward. There is a lack of theoretical descriptions, and qualitative research, process evaluations, realist evaluation or mediation analysis within outcome evaluations can all support the articulation and testing of interventions’ causal mechanisms. Theory evaluations also need to include a clear focus on context, working to understand how interventions interact with the system and how this changes over time. 57

There is also a need for more economic evaluations, which reflects a wider issue in the field of social care. 76 A 2021 systematic review, conducted by members of the CHIMES review team, identified that, in the field of children and young people’s social care more widely, there has been a tendency to use cost–benefit analysis and, while this is useful, there is also scope to take more advantage of cost consequence analysis. The review further provide 10 recommendations to improve the conduct of economic evaluations, which include: pilot measures and data sources; engage with stakeholders; explore strategies to maintain recruitment; define and justify the approach; define and justify the time horizon; clearly identify all costs; identify routine data sources; carry out sensitivity analysis; carry out subgroup analysis; and consider using economic models alongside or as an alternative to a within-trial economic evaluation. 76

Intervention reporting and future systematic reviews

Reflecting on the CHIMES review and its associated challenges, we have a number of recommendations that can support the future conduct of systematic reviews at the interface of public health and social care. First, is the need to improve the reporting of interventions. Currently, there is a lack of use of reporting standards for social care interventions, such as the TIDieR framework, to describe theory, components and implementation strategies. 214 Equally, given the reviews’ emphasis on the importance of context and the contextual contingency of intervention effects, future syntheses would benefit from more comprehensive reporting of key context factors within evaluations, potentially drawing on models, such as the CICI framework. 82

Second, we recommend further standardisation and harmonisation of outcome measurements used to assess intervention effectiveness, which can support meta-analyses. While there were a select number of outcome measures routinely used in evaluations, such as the SDQ and Children’s Global Assessment Scales, further work might be undertaken to support the implementation of core outcome measurements across domains of subjective well-being, mental health, behavioural and neurodevelopmental disorders and suicide-related outcomes.

Third, the review has highlighted the utility of complex-systems informed approaches,57,58 that draw together theory, process and outcome evaluations to understand how system characteristics shape intervention implementation and structure outcomes. To date, the reliance on synthesising outcome studies in reviews of the mental health of care-experienced young people has ensured limited understanding of how interventions work in context and their potential transportability to other systems. As we have highlighted the strength of the evidence base in United States and issues about replicability in other countries, we need to foreground questions about the applicability of international evidence to local contexts as part of the review process. Although published after commencement of the present review, we found the TRANSFER framework to be supportive in exploring stakeholder priorities and perspectives to understand issues around transportability, and further integration of such frameworks into reviews would help to progress the evidence base. 72

Summary recommendations for research

We developed a set of six core questions for researchers to consider when developing, adapting, evaluating and reporting interventions, specifically in relation to mental health interventions for care-experienced children and young people. These were informed by our reflections on the evidence base and our experiences of conducting the CHIMES review.

1. Stakeholder involvement in intervention development: were care-experienced children and young people, parents and carers, and health and social care professionals involved in the development of the intervention? Stakeholders, especially young people, have largely been excluded from the process of intervention development.

2. Describing the intervention theory and components: have you described the programme theory and components so that it could be replicated? To date, interventions have been under-theorised and have not used reporting guidelines to describe the theory, components or implementation strategies.

3. Intervention transportability and adaptation: if an intervention has been delivered and evaluated in a different context, have you considered potential similarities and differences between the original and new context and the need for adaptation? Most of the evidence base is from a select number of countries (e.g. USA) and there may be need to adapt interventions to meet the needs of different social care and healthcare systems.

4. Appropriate intervention evaluation study designs: have you selected an appropriate research design that can address the evaluation RQ? Evaluations often make claims about intervention effectiveness based on feasibility or pilot studies, or from underpowered outcome evaluations.

5. Integrated intervention evaluation study designs: does your intervention evaluation integrate description of the programme theory, outcome evaluation, process evaluation and economic evaluation? There is currently a lack of theory and economic evaluations in social care.

6. Complex-systems informed systematic reviews: does your systematic review take account of the contexts in which interventions are developed, delivered and evaluated? Complex-systems informed systematic review can support decision-making about the potentially transportability of the international evidence base to local contexts.