An adapted social communication intervention at home and education to promote social communication change in children with severe autism: the PACT-G RCT

Article history

The research reported in this issue of the journal was funded by the EME programme as project number 13/119/18. The contractual start date was in February 2016. The final report began editorial review in December 2020 and was accepted for publication in March 2021. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The EME editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Copyright statement

Copyright © 2022 Green et al. This work was produced by Green et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This is an Open Access publication distributed under the terms of the Creative Commons Attribution CC BY 4.0 licence, which permits unrestricted use, distribution, reproduction and adaption in any medium and for any purpose provided that it is properly attributed. See: https://creativecommons.org/licenses/by/4.0/. For attribution the title, original author(s), the publication source – NIHR Journals Library, and the DOI of the publication must be cited.

2022 Green et al.

Scientific background

Intervention evaluation research in autism spectrum disorder (hereafter ‘autism’) has recently accelerated, with studies across a range of interventions considered in recent National Institute for Health and Care Excellence (NICE) guidance,2 Cochrane Library reviews3 and other reviews. 4–6 The pattern of findings across a number of early childhood interventions is for reproducible moderate-to-good effects on targeted ‘proximal’ or intermediate outcomes, such as improvement in social interaction and communication in the local treatment context. 7–9 However, for an intervention to demonstrate a tangible impact on a child’s life and overall development, the challenge is to show effects beyond the immediate intervention context, for instance that changes are also seen in non-direct treatment contexts, such as in interactions with other people inside or outside the classroom or in the family home setting, or an impact identified on other developmental function or symptom outcomes over time and in a variety of settings. In this regard, there is much less evidence,4,6 despite the fact that without such changes the claim that autism interventions are truly effective is harder to substantiate.

Reinforcing the transmission of targeted ‘proximal’ intervention effects into functional change seen in the wider context pervading the child’s life is, thus, a key current challenge for autism treatment research,6,10 and raises the question why does this appear to be so difficult in autism? The capacity to generalise acquired skills flexibly across different situations, people and environmental contexts – a central feature of skill acquisition in neurotypical development – has often been suggested to be a core difficulty for individuals with autism. In this context, one can consider two different forms or concepts of generalisation, which will be referenced throughout this report. First, the concept of generalisation of specific behaviours (for instance the observation of a specific newly acquired skill or behaviour in one setting that then occurs in another) has been commonly investigated in behavioural learning theory and could be termed ‘homotypic’ generalisation. Second, the concept of a ‘developmental cascade’ from precursor skills through to developmentally related but different subsequent skills (e.g. specific vowel sounds in infant development through to evolved language) is the subject of much study in general developmental science, and might be characterised as ‘heterotypic’ (‘homotypic’ and ‘heterotypic’ being terms originating in the longitudinal epidemiology literature). Although a recent systematic review concludes that the empirical evidence for difficulty in generalisation in the first sense is inconsistent,11 many single-subject design studies using behavioural learning methods have described difficulties that autistic children have in transferring newly taught skills to different settings, people or materials/activities, and it has been an assumption in developmental science, as well as from clinical experience, that there is additional difficulty in generalisation in both senses for autistic children. However, these are assumptions that have not been fully evaluated.

Possible explanations, particularly for the second, heterotypic, form of generalisation, include the autistic child’s well-documented lack of internal symbolic representation, which is likely to interfere with the consolidation and thus transfer of procedural aspects of skill across contexts into different forms of behaviour, independent of the specific context in which the skill was acquired. Another related barrier may be weaknesses in autistic cognitive central coherence, associated with an over-reliance on concrete behavioural prompts or reinforcers, which can lead to fragmented learning dissociated from the social pragmatic context as a whole rather than a more diverse integration of learning with connected understanding in context. There may be further factors related to a general lack of behavioural flexibility in applying new skills across different environments. Plausible approaches to helping overcome some of these generalisation difficulties in autism include embedding the intervention into the social environment through parent mediated and education staff-mediated learning, which may optimise the interpersonal cues and continuity of learning across contexts. 12 This method supports opportunities for incidental or naturalistic learning, by weaving functional social and communication learning into the child’s daily experiences, where skills are acquired. Working with children in naturalistic environments is now often highlighted as best practice for early intervention. 13

Early social communication intervention, delivered through parents, therapists or teachers, is the only early autism intervention with a ‘consider’ recommendation by NICE. 2 The Preschool Autism Communication Trial (PACT)7 tested a clinic-delivered, parent-mediated social communication intervention against regular care in what was, to the best of our knowledge, one of the largest randomised controlled trials (RCTs) in the field. 14 The therapy showed a substantial impact on the targeted immediate outcome of parental communicative synchrony with the child [effect size (ES) 1.22, 95% confidence interval (CI) 0.85 to 1.59)] and also on the child’s communication initiations with the parent (ES 0.41, 95% CI 0.08 to 0.74). The original publication7 demonstrated that PACT was associated with non-significant effects on the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) social communication domain when considered alone, but subsequent analysis of outcome on the full autism symptom phenotype [measured across both social communication and repetitive behaviour and sensory symptom domains in the ADOS-2 Calibrated Severity Score (CSS)] found a significant intervention effect at the end point, with a log-proportional odds ratio of 0.64. 14 Subsequent follow-up of the PACT7 cohort 6 years after treatment end, when participant mean age was 10.5 years, with blinded assessment of the original randomly allocated groups (80% follow-up completeness on the primary outcome) found evidence suggesting further sustained effect on reduced autism symptom severity on the ADOS-2 (log-odds ES 0.70, 95% CI –0.05 to 1.47), and a significant overall treatment effect when calculated over the treatment and follow-up time period (log-odds ES 0.55, 95% CI 0.14 to 0.91). Non-blind, parent-rated autism symptoms on the Social Communication Questionnaire (lifetime version) (SCQ) (ES 0.40, 95% CI 0.05 to 0.77) and repetitive and sensory behaviours on the Repetitive Behaviour Questionnaire (RBQ) (ES 0.87, 95% CI 0.47 to 1.35) also showed comparable improvement at follow-up. 14

To our knowledge, the PACT intervention was the first study in the field to show that a preschool autism intervention could have downstream developmental effects to reduce child autism symptom severity for a substantial period after treatment end. This PACT treatment7 was 93% clinic delivered and the key outcomes were measured in a research context; there was little measurement of effects within educational or home environments. Consequently, in the Paediatric Autism Communication Trial – Generalised (PACT-G), we aimed to extend the work of the original PACT,7 with the aim of (1) reinforcing the extension of treatment effects within the different naturalistic environments of the child’s early life, in home and educational settings, and (2) responding to moves towards home-based therapy delivery. The intervention was taken into the child’s daily naturalistic home and nursery/school setting, incorporating parent- and education staff-mediated interventions, and embedding individualised strategies within these naturalistic learning contexts.

A further development was the extension of the intervention into the primary school years, up to the age of 11 years. Autism intervention studies to date have been largely limited to preschool (< 5 years) interventions; however, early communication development continues into the school years,15 and social communication skills in the early school-age period are strong predictors for later development. 16 The persisting and significant impairments in social interaction and communication among children with autism argue for a developmentally sustained intervention into middle childhood, utilising the child’s naturalistic learning environments. However, because of the focus of the intervention on early social communication skill development, the cohort of autistic children of school age (> 5 years) receiving the treatment was restricted to those who had no more than a 4 years’ language-equivalent level.

In addition, a mechanism study within PACT-G built on the understanding gained from the mediation analysis in the original PACT. 17 This previous analysis had tested for mediation of primary treatment effects on child autism symptoms (ADOS-G) through the proximal intervention target of improvement in parental synchronous response to child communication, and an increase in child communication initiations with the parent within dyadic communication (both of which were researcher assessed by video-coding blind to intervention group). In this analysis, increases in child initiations with a parent were shown to be strongly (mediated 80%) by the change in parental synchronous response produced by the treatment. These changes in child initiation with a parent then, in turn, were shown to strongly mediate (97%) the change in child autism symptoms. This mediation analysis had supported the theoretical logic model of the PACT7 intervention by showing a causal-effect chain from parent response to child, to child communication with a parent, and from child communication with a parent to a generalised social communication change in child symptoms. A similar mediation effect had been shown previously in the smaller PACT. 18 In the PACT-G mechanism design, we aimed to test whether or not a similar chain of mediation effect would be maintained when measured in the home and educational settings, and whether or not such changes in two parallel naturalistic settings simultaneously might be further additive in mediating an enhanced effect on symptom outcomes in the research context. In addition, the assessment of mediators and outcomes in different simultaneous naturalistic contexts might provide an innovative opportunity to study key mechanisms of child developmental learning across contexts and whether or not different contextual learning might generalise in an additive way to related downstream symptom outcomes.

Background literature on autism intervention

A significant evidence base from RCTs of early interventions for young children with autism has accumulated over the past 15 years. 4,6,19 Many of these interventions aim to ameliorate the impact of core social communication impairments on early social interactions, particularly with parents, who often find their child’s behaviour perplexing and challenging. The tested interventions have varied in design, duration and intensity, and include developmental and behavioural approaches20 that are mediated through parents and delivered directly by therapists, and there is significant study-to-study variation in the effects found. However, robust evidence now exists that some aspects of social communication can be improved for many young children with autism. 4

The most consistent findings are improvements in dyadic interaction between children and their parents or another adult. 7,9,21–25 Although many studies report improvements in language and communication on unblinded parent-report measures,7,14,25–27 findings on observational or standardised measures of communication and language ability are more variable. Thus, some studies report improvements7,14,23,26–28 whereas others do not. 7,21,24 Only a few studies have demonstrated treatment reductions in autism symptom severity,14,27 whereas most have not. 25,26,28 A recent review4 contains a thorough review of this literature in the context of study design and reporting quality.

Despite this promising evidence base, there is wide recognition that the early autism intervention research field faces challenges. 29–31 Even trials demonstrating group-level improvements report modest ESs,4 reflecting, in part, that a significant proportion of participants in many studies do not benefit. There are sparse examples of designs that would better inform ‘personalised medicine’ approaches (i.e. ‘who benefits from which treatments?’), including comparative or equivalence trials, and rigorous examination of which children benefit from these interventions (moderating effects) and how (mediating effects). Only two long-term follow-ups have been conducted to assess maintenance of effects,14,21 and there has generally been a lack of independent replication of substantial findings. In addition, reporting standards have been variable and there has been inadequate measurement of and attention to the critical importance of testing if and how interventions have an impact on everyday functional abilities that extend beyond the proximal intervention context. 11

In an earlier study, we found that a therapist-directed, parent-mediated intervention that uses video-guided feedback to and coaching for parents, following a transactional developmental model,32 resulted in improvements in parental synchrony and child initiations when interacting with each other. We also found moderate ES reductions in overall autism symptom severity on a blinded observational measure at treatment end point (log-proportional odds ES 0.64, 95% CI 0.07 to 1.20), which was sustained at the 6-year follow-up (log-proportional odds ES 0.70, 95% CI –0.05 to 1.47), with an overall log-proportional odds ES of 0.55 (95% CI 0.14 to 0.91; p = 0.004), as was the improvement in child initiations. 14

A qualitative interview study with parents who had received PACT therapy within the previous PACT7 confirmed the largely positive evaluation of the core effects of the therapy on communication, interaction and child progress, as well as emphasising valued improvements in the parents’ sense of the relationship with their child and enhanced parental and family well-being. However, the study revealed practical difficulties for some families in attending therapy sessions within a clinic environment,33 including the inconvenience (travel, child care), unfamiliarity with and lack of preparedness for autism in therapy venues, and challenges with occupying the child during video-feedback sessions and discussion. Delivery of the intervention within the home and educational settings and the use of video telecommunication software aimed to address some of these practical challenges, while also, importantly, in theory, providing a method of enhancing the adoption and generalisation of therapeutic strategies within the child’s family and everyday naturalistic environments. The use of teleconferencing technology was also thought to have the potential to remove practical barriers, as video material of adult–child interactions could be made in either the home or educational setting prior to the session, and then shared and discussed between the therapist and parent/member of education staff through teleconferencing at a later time without the child present. The advantages and disadvantages of these adjustments found in practice are discussed further below.

Interventions for autism in the educational setting

There are a number of programmes delivered within UK educational contexts to support children with autism. 34 The focus of many of these is on enhancing cognition and learning and reducing challenging behaviours. The majority of interventions fall into two broad categories: comprehensive treatment models (CTMs) and focused interventions. CTMs tend to be developmentally focused, multicomponent educational programmes, aiming to improve functioning across cognition, emotional and social development. They are often embedded within the classroom setting itself and delivered by members of staff. 35 A common UK example of a CTM is the Treatment and Education of Autistic and related Communication-Handicapped Children (TEACCH) programme, which incorporates the use of classroom-based visual schedules and prompts. 36 In the USA, Social Communication, Emotional Regulation, and Transactional Support (SCERTS)37 and the Learning Experiences and Alternative Program for Preschoolers and their Parents38 have also been used; these are also occasionally available in UK specialist autism schools. Although these approaches may be based on different conceptual models,39 they share similar general goals of embedding group, class-based support for children with autism within the educational setting.

Focused interventions tend to target specific skills;40 for example, the Picture Exchange Communication System (PECS) is used to support picture exchange as an alternative form of making a child’s needs known. 41 Alternative and augmentative language systems (e.g. Makaton)42 and the use of technology, such as voice output communication aids, are also sometimes available in the UK. A teacher-delivered interactive communication intervention for specific language impairments [the Social Communication Intervention Project (SCIP)]43 is also applicable to high-functioning autistic children in middle childhood. Peer-mediated interventions promote naturalistic peer socialisation and support in the school environment44,45 and have face validity. 46 In contrast to CTMs, the model by which many of these interventions are delivered is through consultation with schools by specialist health professionals (e.g. psychologists, speech and language therapists or trained teachers), but with the aim of collaborating with ‘non-specialists’ in their implementation [e.g. peers and learning support assistants (LSAs)]. 47 Funded LSAs are often allocated for differentiated support and learning in the classroom for individual children. There is a relative lack of high-quality implementation trials of such interventions in educational settings. 34,48

Education provision for children with autism in the UK

In the UK, 71% of children with a diagnosis of autism attend a mainstream educational setting. 49 Despite the benefits of inclusion in mainstream schools (notably the, potentially, wider participation in the educational experience, peer role models, acceptance and opportunities to develop academic, social and emotional skills50) access to specialist professional support and guidance is often very limited. Depending on the level of special educational needs (SEN), pupils receive non-specialist LSA support, but this is highly variable between schools. 51,52 LSA roles are wide-ranging and include supporting children with complex health, medical, behavioural and learning needs. 53 LSAs in special schools are more likely to work in a team with the teacher and support the whole class of children than they are to work one to one with an individual child.

Trial organogram

The organisation of the three-site trial is shown in Figure 1, with the trial principal investigators named and their roles given. The full trial team is named and acknowledged in Acknowledgements. The study progress Gantt chart is shown in Table 1.

FIGURE 1.

Trial organogram.

Stop–go 1: educational engagement

Stop–go 2: pilot phase recruitment

Design

The trial was a three-site, two-group, RCT of the experimental treatment plus treatment as usual (TAU) compared with TAU alone. Children aged 2–10 years with autism were recruited to the trial in the local areas following referral by clinical specialists and education professionals. Consented families were then randomised to three sites around the UK to receive either the PACT-G social communication intervention in addition to TAU (referred to as the PACT-G group) or TAU alone (referred to as the TAU group). Assessments were administered on entry (baseline) to the trial, at the 7-month mid-point and at the 12-month end point. An embedded mechanism study aimed to test specific mediation hypotheses in the trial and to, potentially, use the experimental intervention design with repeated measures to contribute to a general understanding of the generalisation of acquired skills in autism development across both time and setting, and also in terms of downstream predicted developmental cascades to symptom and adaptive outcomes for the child.

Settings

The settings for the trial were family homes, mainstream nurseries and preschools, specialist nurseries, mainstream schools with specialist autism units and specialist SEN/autism school settings, all of which were in Greater Manchester, London or Newcastle upon Tyne.

Sample size

The PACT7 showed an ES of 1.22 (95% CI 0.85 to 1.59) on parental synchrony [Dyadic Communication Measure for Autism (DCMA)], which mediated 70% of the ES of 0.41 (95% CI 0.08 to 0.74) on child communication, which in turn mediated 72% of the ES of 0.24 (95% CI 0.59 to 0.11) on symptom outcome (ADOS-2). The intervention strategies in PACT-G were specifically targeted to enhance generalisation of the child communication to increase primary outcome effects in home, educational and research settings. Therefore, we expected the ES for the symptom outcome to be substantially above 0.24 and clinically meaningful.

For PACT-G, power was calculated using the sampsi command in Stata^® (StataCorp LP, College Station, TX, USA), for an analysis using analysis of covariance with α = 05, with pre- and post-measures correlated 0.67 (from PACT7). With 110 patients followed up in each group (70/70 preschool-aged and 40/40 school-aged children), 80% power was retained for an ES of 0.28 and 90% power for an ES of 0.33. Allowing for 10% attrition (compared with 4% in PACT7), we proposed to recruit 244 families (rounding up to 82 per site: 52 with preschool-aged children and 30 with school-aged children).

Interventions

The PACT-G intervention is an adaptation and extension of the original, largely clinic-delivered, PACT7 intervention into home and educational settings (93% of original PACT7 sessions were delivered in a clinic setting, whereas all of the PACT-G sessions were delivered in a home setting, in an educational setting or by teleconferencing). PACT7 is a ‘caregiver-mediated’ intervention; in this context, the carers are the parents and education professionals who are the primary recipients of the individualised intervention sessions. PACT-G utilises videos of caregiver–child interaction to provide feedback and guide caregivers to adapt their communication with the child. Therapists support caregivers to adopt evidence-based strategies in their communication so as to enhance child communication development,7,54 and to practise these strategies initially during the 30-minute home practice sessions and eventually throughout the day. Recent research indicates that optimal interaction with a sensitive and responsive communication partner can be effective in increasing communication and social interaction skills in children with autism. 4 In the original PACT,7 this caregiver-mediated approach was found to be effective in increasing the quality of parental communicative responses to the child, which in turn led to increased child-initiated communications with the parent and generalised effects on reduced child autism symptom severity. 7

The PACT-G intervention retains these effective elements, but encourages generalisation of the child’s newly acquired skills into other settings. For instance, the design of PACT-G includes more integration of PACT7 techniques into daily routines and play, and the addition of an adult from the child’s educational setting supports skill development in school. 55,56 The intervention is intended to begin with the caregiver at home and then extend into the child’s educational setting. However, sometimes the contingencies of school term times meant that sessions were coincident in timing. Furthermore, the PACT-G intervention incorporated more recent advances in research17 since the publication of the original PACT. 7 These included focusing on specific strategies to enhance the child’s response to shared attention with objects and play as precursors to the early stages of language development. 57,58 These modifications allow more differentiation of the starting point of the intervention to ensure that it is appropriate for the individual child’s initial level of object play and social engagement.

In common with the original PACT7 therapy, PACT-G therapy takes a staged approach that is based on theoretically informed precursor skills for typical communication development, as well as on addressing atypical communication and language development in autism. 15,59,60 Caregivers are initially encouraged to adopt strategies for establishing essential foundation skills for the child’s communication and language development, including shared attention57 and initiating social communication skills,37 before moving on to strategies that support and enhance the child’s receptive and expressive language development8,58 and reciprocal conversation. 43

In contrast to many manualised intervention programmes, PACT7 and PACT-G do not provide a curriculum or prescribed topic guide for sessions. The manual instead outlines six stages, each with a ‘toolbox’ of strategies that could be used to facilitate development. Therapists are trained to minutely analyse videos of caregiver–child interactions to assess the areas for further development and the potential strengths. 61 Based on these observations, therapists then establish which of the manualised therapy targets are most appropriate for that dyad at that stage of development, and agree these as targets for intervention with the caregiver. Therefore, targets and strategies are individualised for each child, depending on their interactive style. Progress is evaluated and subsequent targets for each dyad are based on that pair’s progress, allowing for a highly individualised intervention approach that accommodates each individual, but can remain within a well-defined framework. This is described further below.

The PACT-G method of video feedback facilitates a collaborative partnership, empowering caregivers. 22,33 A new caregiver–child interaction video is recorded at each session, and the therapist guides the caregiver in observing and analysing the most successful moments of dyadic interaction on video, thereby helping the caregiver to see how their use of strategies promotes and enhances the development of the child’s communication. Insights from this discussion are then used to set individual step-by-step caregiver goals (i.e. strategies they will adopt) within the stages of the manual that are appropriate to the identified individual needs of the child. Between sessions, caregivers are encouraged to practise using these strategies with the child, initially in protected time and later at opportune moments throughout the day. The PACT-G intervention is designed for preschool- and primary school-aged children, from those with little communication or no language to children who use spoken sentences. Progression through the stages of PACT-G is individually differentiated so it is related to both the caregiver and the child’s readiness to progress. Many dyads may remain at the early stages focusing on shared attention, caregiver-adapted responding and child communication initiation; others may progress to later PACT-G stages of language expansion and reciprocal conversation.

The PACT-G intervention manual provides clearly defined stage aims, child outcomes, therapy targets and a range of caregiver strategies. Each of the six stages includes a list of objective measures to determine a dyad’s readiness to progress to the next stage.

Intervention delivery

Training and fidelity of treatment

Training in the PACT-G therapy was conducted centrally by the lead speech and language therapists, who undertook overall co-ordination of the therapy in the trial and organised quarterly across-site therapist meetings. Therapists were regularly supervised by the clinical lead in each site. All therapy sessions were videotaped, and randomly selected tapes of one home and one education in-person session per therapist were independently rated using the PACT-G fidelity rating scale (see Appendix 4) at regular intervals across the trial period. Therapists and research staff were trained in practices that minimise non-compliance and dropout. Arrangements were also made at each site to ensure that PACT-G therapists had no contact with TAU only (control intervention) families. Therapy compliance and receipt of other interventions outside the protocol were monitored.

Treatment as usual

The control intervention was TAU only. We had detailed information on TAU in the preschool population from the group’s previous work on the original PACT and in older children from the PACT 7–11 follow-up study. 65 Data on the services received by all families recruited to PACT-G were collected by the research team.

Avoidance of contamination

There were separate clinical and research leads at each site and separate training and supervision structures. Researchers were housed separately from the staff involved in the delivery of the PACT-G intervention. Mid- and end-point research interviews and assessments were conducted so as to avoid inadvertent divulging of information that could reveal treatment status. The assessment suite and materials used were different in type and location to those used for the therapy intervention in home or educational settings to avoid any familiarity effect for children in the treatment group.

Measures

Other measures

Procedures

Patient and public involvement input into original intervention development

Ongoing sequential development of the PACT intervention has been carried out in close collaboration with families and service users. At an early stage, the team co-developed with service users the first user-nominated measure of treatment outcome in autism,97 which helped focus on key parent-nominated priorities. Parent reports on the experience of receiving PACT therapy in the initial stages were evaluated through independently conducted interviews and the results were fed back into the design of the next iteration of the therapy within PACT-G. 33 This feedback, including around clinic visits and home-based practice, informed the design of the home-based aspects in PACT-G.

Patient and public involvement review following the pilot phase

This planned review included input into research design and adaptation of the intervention prior to finalisation of the trial protocol.

Patient and public involvement input to the Trial Steering Committee and Local Steering Groups

Two parent representatives were members of the TSC. Both were parents of children with autism and one was a qualified teacher with experience of working in both mainstream and specialist schools. At all meetings, the parent representatives contributed to discussions and provided valuable input on aspects of the trial design and the therapy process. Some specific areas of feedback from the parent service user TSC members that led to changes to procedures included:

• Discussion around how researchers could sensitively introduce some of the research assessments, which led to SOP researcher guidelines on how to introduce research measures.

• Ways to keep parents engaged with the trial team to maximise participant retention, especially in the TAU group. This led to the purchase of trial-branded merchandise (coffee cups, magnets, pens, etc.), which kept particpants aware of the trial and acted as a small token of appreciation.

• Discussion about borderline eligibility issues, such as significant parent mental ill health, child epilepsy and child genetic conditions. This led to a tightening of the eligibility criteria, ensuring that referring clinicians and trial personnel understood the clear and consistent eligibility criteria for participation.

• Reflections on potential parental experiences of the HSC and some of the parent–education staff dynamics that might affect the delivery and success of these sessions. This resulted in greater awareness among the trial team and increased sensitivity to some of these issues.

Aims of formal analysis

Phase 2: objective 2 mechanisms evaluation

Mediation analysis17 gave detailed insight into an attenuated transmission or generalisation in the original PACT across changes in person, task and context (see Chapter 1). In PACT-G, we tested for enhanced generalisation into home by keeping parent–child dyadic cues constant but increasing functionally relevant interaction contexts, and into education by introducing education staff (i.e. LSA)–child dyadic interventions and enhancing relevant parent–education staff (i.e. LSA) communication. The mechanism study investigated the mediation process in this model and, through that, aimed to illuminate key basic knowledge about developmental generalisation of acquired skills in autism. Some of the pathways of interest are illustrated in Figure 2. If the efficacy analysis showed significant between-group differences in the mediators [DCMA and/or BOSCC at home (path a) and in education contexts (path c); see Figure 9], then we used parametric regression models to:

1. test for mediation of the intervention on primary symptom outcome (ADOS-2) through DCMA and/or BOSCC at home (paths a, e and f)

2. test for mediation of the intervention on primary symptom outcome through DCMA and/or BOSCC in educational setting (paths c, d and f)

3. test for mediation of intervention on DCMA and/or BOSCC in educational setting through DCMA and/or BOSCC at home (paths a, b and c)

4. use structural equation modelling to examine multiple pathways through DCMA and/or BOSCC at home and in the educational setting to primary symptom outcome (paths a–f).

FIGURE 2.

Key mediation pathways tested in the PACT-G mechanism study.

We repeated these four steps using researcher BOSCC as the outcome variable in place of the ADOS-2. As all of the measures are continuous, the indirect effects were calculated by multiplying relevant pathways. Bootstrapping was used to produce valid standard errors for the indirect effects. All analyses were adjusted for baseline measures of the mediators (BOSCC/DCMA), primary outcome (ADOS-2) and putative measured confounders. Mediation analyses are potentially biased by measurement error in mediators and hidden confounding between mediators and outcomes; we built on our previous methodological and applied work in this context to include repeated measurement of mediators and outcomes to account for classical measurement error17 and baseline confounding. 102

Primary outcome analysis

The ADOS-2 is a structured researcher-led assessment that maximises ascertainment of both social communicative competency and autism-related atypicality. Both tasks and scoring vary with the child’s verbal ability according to the ADOS-2 module completed, and the same module was used at baseline and end point. Stratified by module (and, for module 1, by child’s level of language to allow for the three items that differ for children with/without words), a module and language-level stratified regression of the end-point ADOS-2 SA plus RRB raw total score included treatment assignment, randomisation stratifiers (dummy variables for site, age group and gender), baseline ADOS-2 score and any other baseline variables found to predict missingness. Residual plots were used to determine whether or not prior transformation of the ADOS-2 scores was required. Using the within-group end-point standard deviation (SD), an ES was calculated for each module stratum. A single estimate pooled across modules was calculated using a weighted mean, where the weights were the inverse of the variance of each stratum-specific estimate. A CI for this pooled estimate was obtained using 1000 bootstrapped samples. The stratified analysis is illustrated below in Figure 3. To be consistent with the treatment main effects analysis, the test of difference in treatment effect by age group was based on the bootstrap p-value over 5000 replicates of the pooled within-ADOS-2 stratum estimate of the treatment difference.

FIGURE 3.

Stratified structure of analysis model. Analysis of primary outcomes: weighted sum of a, b and c (each standardised by within-group variance of outcome). 1m nv, module 1 non-verbal; m1 v, module 1 verbal; m2, module 2.

Non-adherence and complier-average causal effect estimator

The total number of therapy sessions meeting the satisfactory quality and fidelity criterion was used to define the dose of PACT-G therapy received (see Table 13). We estimated the effect of each satisfactory therapy session using instrumental variable regression (the model being identified by assuming the exclusion restriction that the offer of treatment does not itself influence the outcome) after accounting for receipt of treatment. The model had two correlated equations, the first for the number of satisfactory sessions received and the second predicting outcome, with treatment received replacing treatment assignment. The distributional form for the number of sessions was examined to determine whether or not this could be assumed Gaussian, or overdispersed Poisson with, the model being estimated using Stata sem or gsem, as appropriate. A dose–response model, assuming a linear dose–response relationship, was used to calculate how many sessions would be required to achieve a criterion level of clinical efficacy – defined here as the ES included in the power calculation, that is, an ES of 0.28.

We defined a more approximate variable for TAU families, where significant use of PACT-like treatment was identified. If this exceeded 10% of TAU participants using PACT, then an extended complier-average causal effect (CACE) model was estimated.

Analysis and presentation of secondary outcomes

Secondary outcomes with baseline, mid-point and end-point data:

• the parent BOSCC total

• the LSA BOSCC total

• the researcher BOSCC total (no mid-point)

• parent synchrony from the DCMA

• LSA synchrony from the DCMA

• child initiations from the parent DCMA

• child initiations from the LSA DCMA.

Secondary outcomes with baseline and/or end-point data:

• Language composite – MCDI receptive and expressive scores, the one-word tests and the PLSs.

• Anxiety composite – DBC anxiety and emotional subscale of the SDQ. DBC anxiety was collected at the end point only.

• SCQ – collected at baseline only.

• Repetitive behaviour composite – parent SCQ and insistence on sameness and sensory motor scores from the RBQ. Parent SCQ was collected at baseline only.

• Adaptive behaviour composite – VABS composite standard scores from parent and teacher and the prosocial subscale of the SDQ parent and teacher.

• Parent well-being composite – WEMWBS and TOPSE.

• Child health-related quality of life – CHU9D.

• Disruptive behaviour composite – SDQ conduct subscale and disruptive/behaviour problems from the DBC. The DBC was collected at the end point only.

Secondary outcomes are presented either as a single-measure estimate from an analysis of covariance or, where multiple measures contributed, formed into composites and analysed using structural equation modelling (SEM), as illustrated below in Figure 4 for a composite with three baseline and four end-point components.

FIGURE 4.

Structure of SEM-based models for the analysis of composites.

Assessing quality of outcome measures

Dyadic interaction

Participant flow

Participant flow is summarised in Figure 5. The first randomisation into the trial occurred on 18 January 2017, and last randomisation on 19 April 2018. The follow-up mid-point assessments ran from 15 August 2017 to 7 December 2018 and the end-point assessments ran from 5 January 2018 to 3 May 2019. Study withdrawal rates were very low and, as shown in Table 7, were predominantly from the TAU group. Although there were small variations in the numbers of mid-point and end-point assessments completed in the different settings, completion rates were very high. Only one participant did not contribute data to the analysis of the primary outcome.

FIGURE 5.

The PACT-G CONSORT flow diagram. a, One family inadvertently received PACT-G treatment.

Baseline comparability of randomised groups

Table 8 summarises baseline characteristics by treatment group. Participants were predominantly white (60.1%) and male (79.4%) and spoke only English at home (79.4%). There was very little missing baseline demographic information. The participant population was ethnically diverse, in keeping with the large urban context of two of the sites and the mixed urban–rural setting of the third.

Tables 9 and 10 present selected baseline measures by age group. With a few exceptions, the children in the older age group scored ≥ 2 SDs below the population mean for the Mullen Scales of Early Learning developmental quotient (DQ) and development of communication. Children in the younger age group showed somewhat lower autism severity than those in the school-aged group, but were nevertheless markedly intellectually and communicatively impaired.

Table 10 shows that, with respect to autism severity, the groups were similar and at a level consistent with the inclusion criteria of severe autism.

Although the same module was used at baseline and at the end point, two items used in the module 1 ADOS-2 algorithm for calculating the ADOS-2 total score for ‘no-words’ children were replaced by different items when the ‘some words’ level of language was achieved, and vice versa if language regressed. Table 11 shows the modules and level of language of participants at baseline and the end point.

Education provision

The educational settings across both groups of the trial, characterised at baseline, are shown in Table 12. The majority of preschool-aged children recruited to PACT-G attended a mainstream nursery [for both the PACT-G (n = 45) and TAU (n = 48) groups]. These settings consisted of privately owned independent day nurseries and local authority day nurseries. A small number of children in this age group attended a specialist nursery (PACT-G n = 3; TAU group, n = 2). These settings were either charity-led or local authority nurseries. One child attended a child-minding setting (TAU). All of these settings deliver a child-led curriculum within the Early Years Foundation Stage (EYFS). 104

The majority of school-aged children in the PACT-G (n = 44) and TAU (n = 56) groups attended a special educational setting. These settings were either for children with a range of disabilities or were specialist autism schools. Other children in this age group attended a mainstream primary school (PACT-G group, n = 25; TAU group, n = 25) or a resourced specialist provision within a mainstream primary school (PACT-G group, n = 4; TAU group, n = 4). These children access the national curriculum (key stage standards) and have SEN support at school or have an education, health and care plan in place. 91

Change in education professional within therapy

Eighty-five (34.3%) children across the trial experienced a change in the education professional delivering PACT-G between baseline and the mid-point, and 94 (37.9%) children experienced a change in education professional between the mid-point and end point. Twenty-two (20%) children experienced changes at both time points. When this change occurred, the therapist began the PACT-G therapy again with the new education professional. For the purposes of assessing the adequacy of the delivery of the PACT-G therapy, a pragmatic assumption was made that a notional two sessions would be needed for the replacement education professional to achieve minimum familiarity with the method delivery of the therapy and to establish a consistent interaction with the child.

Implementation by stages of PACT-G therapy across home and educational settings

As noted in Chapter 5, Interventions, the PACT therapy proceeds on the basis of manualised sequential stages. All dyads begin at stage 1; the progression that they make depends on child characteristics such as cognitive ability, developmental level and autism severity, as well as the characteristics, progress and adaptations of their carer or LSA as the therapy progresses. Not all dyads are expected to achieve stage 6. As the home and education implementations of PACT-G were separate but involved the same child as the interactive dyadic partner, it is possible to compare the stage progress made for an individual child in each context. Table 15 shows the proportion of children reaching each stage at home and in the educational setting, or at a group level. Although the median stage reached by both parents and LSAs was the same, namely stage 3, a mixed ordinal logistic regression (not prespecified) suggested that parents attained a significantly higher stage than the LSAs (p = 0.014).

TABLE 15 - Stage reached in PACT-G implementation

End-point stage	Children reaching each stage, n (%)
	Parents	LSAs
1	12 (10)	24 (20)
2	29 (24)	27 (22)
3	44 (36)	42 (34)
4	17 (14)	15 (12)
5	15 (12)	9 (7)
6	4 (3)	4 (3)
Total	121	121

Intervention manual fidelity

In-person sessions

Ten therapists delivered a total of 1397 in-person intervention sessions during the trial. Fidelity checks were carried out at three time points across the trial: 6, 12 and 18 months from the beginning of the main trial phase. This was to make sure that fidelity was maintained and did not drift during the trial. At each time point, one education session and one in-home session per active therapist was randomly selected to give a representative picture. Of the 45 sessions selected, two could not be rated because of technical difficulties and COVID-19 restrictions, giving 43 sessions (21 in home settings and 22 in educational settings), representing 3.1% of the total in-person sessions delivered in the trial (2.7% in home settings and 3.5% in educational settings). Scores could range from 0 to 16, with a score of 13 (representing > 80% items passed) set as the minimum fidelity threshold. The mean fidelity across all rated sessions was 14.67 (median 15, range 10–16), with the fidelity threshold being met for 37 (86%) sessions and not met for 6 (14%) sessions (Figure 6).

FIGURE 6.

Distribution of fidelity ratings (n = 43; score > 13 represents acceptable fidelity).

Within the overall total, 17 out of 21 (81%) in-home sessions reached an acceptable mean fidelity score of 14.3 (median 15, range 10–16) and 20 out of 22 (91%) education sessions reached an acceptable mean fidelity score of 14.95 (median 16, range 10–16). Table 16 shows that this did not vary substantially by therapist.

TABLE 16 - Fidelity ratings by therapist

Therapist	Mean fidelity (range; median)
1	14.3 (10–16; 15)
2	15.5 (14–16; 16)
3	15.25 (13–16; 16)
4	14.67 (12–16; 15)
5	13.8 (11–15; 15)
6	15 (14–16; 15)
7	15.75 (15–16; 16)
8	13.6 (10–16; 14)
9	14.5 (13–16; 14.5)
10	15.5 (15–16; 15.5)

Remote sessions (video conferencing or telephone)

Informal fidelity monitoring of a small number (n = 11; 2%) of the remote sessions indicated that, where parents and LSAs had supplied suitable interactive play videos and there were no technical difficulties, the sessions could be of similar quality to face-to-face sessions. However, given the small sample, we can make no inference as to the overall quality of remote session delivery in this trial.

Assessing quality of outcome measures

Primary outcome analysis

Univariate logistic regression was used to detect if any further baseline variables were predictive of missing primary outcome data. No baseline variables were seen to predict missingness beyond those that would already be included in the model (stratifiers and baseline measures of the scale), so no others were included.

The end-point ADOS-2 algorithm score was treated as a continuous variable, with the baseline score as a covariate and randomisation factors included. Three regression relationships were estimated, one for each of the strata defined by the ADOS-2 module [module 1 (non-verbal), module 1 (verbal), module 2] at the end point (Table 23). Regression residual plots were checked within each strata. These showed no evidence of non-normality (see Appendix 1). A single estimate of the treatment effect was obtained by pooling the three stratum-specific ES estimates using the minimum variance estimator. As shown in Table 19, the ITT analysis did not detect an effect of the intervention on the primary outcome of ADOS-2 at the end point. To be consistent with the treatment main effects analysis, the test of difference in treatment effect by age group was based on the bootstrap p-value of > 5000 replicates of the pooled within-ADOS-2 stratum estimate of the treatment difference.

We mapped the change in the ADOS-2 algorithm scores to the CSS, which allows comparison across ADOS-2 modules. Shown in Figure 7, there was little difference between the distributions expected for the study participants undergoing PACT-G treatment and those undergoing TAU, with expected rates in the non-autism 1–3 range of 2.6% and 2.5%, in the autism 4–5 range of 7.3% and 6.9%, in the autism 6–8 range of 78.3% and 75.5%, and in the high-severity cut-off range of 11.8% and 15.1%, respectively.

FIGURE 7.

Distributions of CSS outcome. Plotted by treatment group at the end point for all participants, as implied by the estimated effect on the ADOS-2 algorithm total score (the mapping of scores was carried out using Stata’s multiple imputation command using nearest neighbour matching of predicted scores and the published 18-stratum total score to the CSS algorithm).

Primary outcome complier-average causal effect estimator

In the PACT-G group, out of 121 observations, there was a median of 10 (IQR 8–12) acceptable parent sessions, with an IQR of 2–12 and, from 117 observations, a median of 8 (IQR 5–10) acceptable LSA sessions, with an IQR of 0–12. An instrumental variable estimator of a CACE was calculated. Treatment receipt was measured as the number of acceptable sessions with a parent and LSA. We report the estimated effect of receiving a total of eight sessions across home or educational settings (the TAU group being considered as receiving no sessions), which was the minimum number of sessions therapists and the research team took to constitute a delivered treatment. The across-strata, pooled CACE estimate for a dose of eight sessions was non-significant (–0.02; p = 0.755). To reach the minimum clinically significant change of 0.28, 112 sessions would be required.

Secondary outcomes

Table 24 shows the effect estimates for all secondary measures and composites.

Significant changes are seen on the parent and LSA DCMA measures, which are the proximal target behavioural mediators of the therapy (Table 22). Similarly, the child initiations, as measured using the DCMA with both parents and LSAs, also show significant effects. The BOSCC measure, an observational measure of autism-specific behaviours, showed no significant change in any of the three settings of interaction with a parent, LSA and researcher. Parent-reported measures of language, anxiety, repetitive and adaptive behaviour, and an overall measure of child health-related quality of life showed no effect. There was significant improvement in parent- and teacher-reported child disruptive behaviour and parental well-being (Table 25).

Overall outcomes

Figure 8 shows the pattern of treatment effect across all planned outcomes, with directions of effects signed with beneficial effects on the right-hand side of the null.

FIGURE 8.

The ESs of the primary and secondary outcomes. rev, reversed.

Adverse events

Table 26 shows AEs collated by treatment group. There were 46 AEs, with six of these classified as serious. Seven events were possibly related to the study, and these were balanced across treatment arms. One serious AE was deemed possibly related to the study: an impact on a parent’s mental health because of the autism diagnostic process and collaborative nature of therapy received.

TABLE 26 - Adverse events

Event characteristic	Treatment group, n (%)		Overall, n (%)
	TAU	PACT-G
Is the event serious?
No	20 (90.9)	20 (83.3)	40 (87.0)
Yes, requires hospitalisation	2 (9.1)	2 (8.3)	4 (8.7)
Yes, persistent or significant disability	0 (0.0)	2 (8.3)	2 (4.3)
AE-type events (people)
Reduction in school attendance	1 (4.5)	0 (0)	1 (2.2)
Other issues with school	1 (4.5)	1 (4.2)	2 (4.3)
Relationship breakdown (parent)	0 (0)	5 (20.8)	5 (10.9)
Significant change in child behaviour or well-being	13 (59.1)	11 (45.8)	24 (52.2)
Significant family illness	1 (4.5)	1 (4.2)	2 (4.3)
Death in immediate family	1 (4.5)	1 (4.2)	2 (4.3)
Other personal/family issue	4 (18.2)	3 (12.5)	7 (15.2)
Other	1 (4.5)	2 (8.3)	3 (6.5)
Relationship to study
Possibly related	4 (18.2)	3 (12.5)	7 (15.2)
Not related	18 (81.8)	21 (87.5)	39 (84.8)

Mechanism study

In this planned analysis of treatment effect mechanism, we examined several putative mechanisms between randomisation and outcomes in a series of mediation analyses. As in the primary outcome analysis, estimates are made for each ADOS-2 or BOSCC module, which are then combined into a total score using an inverse variance-weighted estimator. The findings are presented with no adjustment for measurement error in the mediator, and the results are displayed as unstandardised effect estimates.

For each analysis, the results show the estimated effect of treatment on the mediator, the estimated effect of the mediator on the outcome, the estimated indirect effect (the product of the preceding two estimates), the direct effect that represents any other pathway from treatment to outcome that does not act through the mediator being considered and the total effect (the sum of the direct and indirect effects). In Tables 27–33, we present the effect estimates, standard errors, 95% CIs and p-values.

The first model examines mediation of treatment to DCMA parent synchrony at 7 months to ADOS-2 at 12 months (Figure 9).

FIGURE 9.

Model pathway from treatment to ADOS-2 outcome.

To summarise the findings presented in Table 27, there is no total effect to be mediated (as demonstrated in the primary ITT analysis), but there is a significant effect of treatment on parent synchrony at 7 months (A path) in favour of the PACT-G intervention.

TABLE 27 - Mediation of DCMA parental synchrony at 7 months on end-point ADOS-2 outcome at 12 months

SE, standard error.

ADOS-2	A path (SE); (95% CI); p-value	B path (SE); (95% CI); p-value	Indirect effect (SE); (95% CI); p-value	Direct effect (SE); (95% CI); p-value	Total effect (SE); (95% CI); p-value
Module 1 (non-verbal), n = 91	0.10 (0.03); (0.04 to 0.15); 0.001	0.81 (2.29); (–3.68 to 5.30); 0.724	0.08 (0.22); (–0.36 to 0.51); 0.727	–0.52 (0.63); (–1.76 to 0.71); 0.406	–0.45 (0.59); (–1.60 to 0.71); 0.450
Module 1 (verbal), n = 82	0.17 (0.04); (0.10 to 0.24); < 0.001	2.85 (2.76); (–2.55 to 8.26); 0.300	0.49 (0.49); (–0.46 to 1.45); 0.312	–1.65 (0.93); (–3.47 to 0.16); 0.074	–1.18 (0.81); (–2.77 to 0.42); 0.147
Module 2, n = 58	0.12 (0.03); (0.06 to 0.18); < 0.001	–0.93 (4.07); (–8.91 to 7.04); 0.819	–0.11 (0.49); (1.06 to 0.84); 0.818	1.60 (0.96); (–0.29 to 3.49); 0.096	1.48 (0.83); (–0.15 to 3.12); 0.075
Overall	0.12 (0.02); (0.09 to 0.16); < 0.001	1.24 (1.62); (–1.93 to 4.40); 0.444	1.33 (1.42); (–1.45 to 4.12); 0.347	–0.32 (0.46); (–1.22 to 0.58); 0.486	–0.16 (0.41); (–0.97 to 0.65); 0.699

We found no association between parent synchrony at 7 months and ADOS-2 at 11 months (B path), and the indirect effect is non-significant, which we concluded was because of the non-significant B paths.

We see the opposite signs for the B path in module 2 than those seen in module 1: an increase in parent synchrony leads to lower ADOS-2 scores in module 2, but to higher ADOS-2 scores in module 1. This leads to the indirect effects having opposite signs between module 2 and module 1, and opposite signs to the direct and total effects, where module 1 showed improvement and module 2 showed worsening.

This direct effect reduced symptom severity in module 1, but increased symptom severity in module 2 [the positive coefficient indicating a worse (higher) score in the PACT-G group]. The negative coefficient for the overall direct effect indicates a reduction in ADOS-2 independent of the mediator, although this effect is non-significant.

In the analysis shown in Table 28, we consider child initiations at 7 months as the mediator, and find that the A path shows significant treatment effect on child initiations at 7 months in module 1 (verbal) and module 2, but not in module 1 (non-verbal). Overall, there is a significant effect in favour of the PACT-G intervention. As with parent synchrony, there is no association between child initiations at 7 months and ADOS-2 at 12 months. There are no significant mediated effects, and the direct effects have the same pattern as with parent synchrony, described previously.

TABLE 28 - Mediation of DCMA child initiations with a parent at 7 months on end-point ADOS-2 at 12 months

ADOS-2	A path (SE); (95% CI); p-value	B path (SE); (95% CI); p-value	Indirect effect (SE); (95% CI); p-value	Direct effect (SE); (95% CI); p-value	Total effect (SE); (95% CI); p-value
Module 1 (non-verbal), n = 91	0.03 (0.03); (–0.03 to 0.09); 0.336	1.11 (2.15); (–3.10 to 5.31); 0.605	0.03 (0.08); (–0.11 to 0.18); 0.654	–0.52 (0.59); (–1.68 to 0.65); 0.386	–0.45 (0.59); (–1.60 to 0.71); 0.450
Module 1 (verbal), n = 82	0.14 (0.04); (0.07 to 0.22); < 0.001	2.42 (2.52); (–2.52 to 7.36); 0.337	0.35 (0.38); (–0.39 to 1.09); 0.356	–1.45 (0.89); (–3.18 to 0.29); 0.103	–1.18 (0.81); (–2.77 to 0.42); 0.147
Module 2, n = 58	0.10 (0.05); (0.01 to 0.19); 0.033	–4.56 (2.50); (–9.45 to 0.34); 0.68	–0.44 (0.31); (–1.04 to 0.16); 0.151	1.88 (0.83); (0.26 to 3.50); 0.023	1.48 (0.83); (–0.15 to 3.12); 0.075
Overall	0.08 (0.02); (0.04 to 0.12); < 0.001	–0.20 (1.37); (–2.88 to 2.48); 0.882	1.30 (1.42); (1.48 to 4.08); 0.359	–0.10 (0.42); (–0.93 to 0.73); 0.820	–0.16 (0.41); (–0.97 to 0.65); 0.699

Table 29 shows that there is a significant effect of treatment on LSA synchrony at 7 months (A path), but there is no association between LSA synchrony at 7 months and ADOS-2 at 12 months (B path). The indirect effect is non-significant because of the non-significant B paths. In this analysis, we see opposite signs for the B path in module 1 (verbal) than in module 1 (non-verbal) and module 2.

TABLE 29 - Mediation of DCMA LSA synchrony at 7 months on end-point ADOS-2 at 12 months

ADOS-2	A path (SE); (95% CI); p-value	B path (SE); (95% CI); p-value	Indirect effect (SE); (95% CI); p-value	Direct effect (SE); (95% CI); p-value	Total effect (SE); (95% CI); p-value
Module 1 (non-verbal), n = 96	0.10 (0.04); (0.03 to 0.17); 0.007	–2.82 (1.76); (–6.26 to 0.62); 0.108	–0.28 (0.20); (–0.68 to 0.12), 0.167	–0.15 (0.61); (–1.34 to 1.04); 0.805	–0.45 (0.59); (–1.60 to 0.71), 0.450
Module 1 (verbal), n = 82	0.13 (0.03); (0.07 to 0.20); < 0.001	3.67 (2.79); (–1.80 to 9.15); 0.189	0.49 (0.39); (–0.28 to 1.25); 0.210	–1.64 (0.87); (–3.35 to 0.07); 0.061	–1.18 (0.81); (–2.77 to 0.42); 0.147
Module 2, n = 59	0.15 (0.03); (0.09 to 0.21); < 0.001	–0.89 (4.13); (–8.98 to 7.20); 0.829	–0.14 (0.63); (–1.37 to 1.10); 0.830	1.63 (1.08); (–0.49 to 3.75); 0.131	1.48 (0.83); (–0.15 to 3.12); 0.075
Overall	0.13 (0.02); (0.09 to 0.17); < 0.001	–0.98 (1.40); (–3.72 to 1.77); 0.485	–0.15 (1.42); (–2.93 to 2.63); 0.915	–0.24 (0.45); (–1.13 to 0.65); 0.600	–0.16 (0.41); (–0.97 to 0.65); 0.699

The analysis in Table 30 shows a similar pattern to previous findings, demonstrating significant effects of the treatment on child initiations with a LSA at 7 months (A path). As with parent and LSA synchrony, there is no association between child initiations at 7 months and ADOS-2 at 12 months (B path) and, overall, there are no significant mediated effects.

TABLE 30 - Mediation of DCMA child initiations with a LSA at 7 months on end-point ADOS-2 at 12 months

ADOS-2	A path (SE); (95% CI); p-value	B path (SE); (95% CI); p-value	Indirect effect (SE); (95% CI); p-value	Direct effect (SE); (95% CI); p-value	Total effect (SE); (95% CI); p-value
Module 1 (non-verbal), n = 96	0.13 (0.05); (0.03 to 0.22); 0.011	–0.69 (1.31); (–3.26 to 1.89); 0.600	–0.09 (0.17); (–0.42 to 0.25); 0.608	–0.39 (0.61); (–1.58 to 0.79); 0.516	–0.45 (0.59); (–1.60 to 0.71); 0.450
Module 1 (verbal), n = 82	0.15 (0.05); (0.05 to 0.24); 0.002	–0.60 (1.99); (–4.51 to 3.30); 0.762	–0.09 (0.30); (–0.67 to 0.49); 0.764	–1.08 (0.87); (–2.78 to 0.62); 0.212	–1.18 (0.81); (–2.77 to 0.42); 0.147
Module 2, n = 59	0.09 (0.05); (0.00 to 0.19); 0.042	3.19 (2.64); (–1.98 to 8.36); 0.227	0.30 (0.29); (–0.26 to 0.86); 0.292	1.18 (0.86); (–0.51 to 2.86); 0.171	1.48 (0.83); (–0.15 to 3.12); 0.075
Overall	0.12 (0.03); (0.07 to 0.18); < 0.001	–0.10 (1.01); (–2.08 to 1.89); 0.925	–0.73 (1.42); (–3.51 to 2.04); 0.606	–0.17 (0.43); (–1.01 to 0.67); 0.695	–0.16 (0.41); (–0.97 to 0.65); 0.699

We repeated these analyses using the same modelling approach, replacing the outcome measures as the BOSCC instead of the ADOS-2. An example is shown in Figure 10. Estimates are shown for each BOSCC module, and then combined into a total score using an inverse variance-weighted estimator.

FIGURE 10.

Model pathway from treatment to BOSCC outcome.

Table 31 shows the results for four different mediators. As these mediators are unchanged, we have already demonstrated that there is a significant treatment effect on each of these (the A paths).

TABLE 31 - Mediation of treatment effects on researcher BOSCC at 12 months

Bold indicates significance.

Mediator at 7 months		A path (SE); (95% CI); p-value	B path (SE); (95% CI); p-value	Indirect effect (SE); (95% CI); p-value	Direct effect (SE); (95% CI); p-value	Total effect (SE); (95% CI); p-value
Parent synchrony	Module 1, n = 191	0.12 (0.02); (0.07 to 0.16); < 0.001	0.09 (3.56); (–6.89 to 7.07); 0.980	0.01 (0.42); (–0.81 to 0.84); 0.980	–0.21 (1.12); (–2.39 to 1.98); 0.854	–0.19 (1.02); (–2.19 to 1.81); 0.854
	Module 2, n = 57	0.13 (0.03); (0.07 to 0.18); < 0.001	–10.23 (16.27); (–42.12 to 21.66); 0.529	–1.30 (2.08); (–5.38 to 2.77); 0.531	–0.02 (3.86); (–7.60 to 7.55); 0.995	–1.28 (3.33); (–7.81 to 5.25); 0.700
	Overall	0.12 (0.02); (0.09 to 0.16); < 0.001	–0.38 (3.48); (–7.20 to 6.43); 0.912	–0.21 (0.50); (–1.18 to 0.76); 0.671	–0.19 (1.07); (–2.29 to 1.91); 0.858	–0.28 (0.97); (–2.19 to 1.63); 0.773
Child initiations with a parent	Module 1, n = 191	0.07 (0.03); (0.02 to 0.12); 0.003	–6.82 (3.16); (–13.01 to –0.63); 0.031	–0.51 (0.29); (–1.08 to 0.07); 0.084	0.27 (1.03); (–1.75 to 2.28); 0.796	–0.20 (1.02); (–2.20 to 1.80); 0.843
	Module 2, n = 57	0.11 (0.05); (0.02 to 0.20); 0.023	–18.32 (9.61); (–37.16 to 0.51); 0.057	–1.98 (1.32); (4.57 to 0.61); 0.133	1.19 (3.48); (–5.63 to 8.02); 0.732	–1.01 (3.42); (–7.73 to 5.70); 0.767
	Overall	0.08 (0.02); (0.04 to 0.13); < 0.001	–7.94 (3.00); (–13.82 to –2.06); 0.008	–0.78 (0.34); (–1.44 to –0.11); 0.022	0.34 (0.99); (–1.59 to 2.28); 0.730	–0.27 (0.98); (–2.18 to 1.65); 0.784
LSA synchrony	Module 1, n = 191	0.11 (0.02); (0.06 to 0.16); < 0.001	3.99 (3.22); (–2.33 to 10.30); 0.216	0.43 (0.37); (–0.28 to 1.15); 0.235	–0.62 (1.08); (–2.73 to 1.49); 0.563	–0.18 (1.02); (–2.17 to 1.82); 0.863
	Module 2, n = 57	0.17 (0.04); (0.10 to 0.24); < 0.001	–41.82 (13.45); (–68.18 to –15.45); 0.002	–6.92 (2.68); (–12.17 to –1.67); 0.010	7.58 (4.44); (–1.13 to 16.28); 0.088	–0.47 (3.86); (–8.03 to 7.09); 0.903
	Overall	0.13 (0.02); (0.09 to 0.17); < 0.001	1.50 (3.13); (–4.64 to 7.64); 0.633	–0.45 (0.46); (–1.34 to 0.44); 0.324	–0.17 (1.05); (–2.22 to 1.88); 0.873	–0.19 (0.98); (–2.12 to 1.73); 0.843
Child initiations with a LSA	Module 1, n = 191	0.12 (0.03); (0.06 to 0.19); < 0.001	–4.23 (2.38); (–8.90 to 0.44); 0.076	–0.52 (0.32); (–1.16 to 0.12); 0.109	0.29 (1.05); (–1.76 to 2.34); 0.784	–0.18 (1.02); (–2.18 to 1.82); 0.857
	Module 2, n = 57	0.14 (0.05); (0.04 to 0.24); 0.008	–9.30 (9.23); (–27.38 to 8.78); 0.313	–1.30 (1.39); (–4.02 to 1.42); 0.350	–0.14 (3.66); (–7.32 to 7.05); 0.971	–1.37 (3.41); (–8.05 to 5.31); 0.687
	Overall	0.13 (0.03); (0.07 to 0.18); < 0.001	–4.55 (2.31); (–9.06 to –0.03); 0.049	–0.67 (0.37); (–1.40 to 0.06); 0.073	0.26 (1.00); (–1.71 to 2.22); 0.800	–0.28 (0.98); (–2.20 to 1.64); 0.774

We find there are overall effects of child initiations with both the parent and the LSA on the BOSCC score at 12 months. This leads to a statistically significant indirect effect of the treatment through child initiations with the parent (p = 0.022) and weak evidence for an indirect effect of child initiations with the LSA (p = 0.073). All of the direct effects are not statistically significant and, as noted in the primary analysis, there are no statistically significant total effects on the BOSCC.

We tested if there was a mediation of treatment effects between the DCMA measures of parent/LSA synchrony and child initiations at 12 months. The results are shown in Table 32.

TABLE 32 - Mediation of parent/LSA synchrony at 7 months on end-point child initiations at 12 months

Outcome at 12 months	Mediator at 7 months	A path (SE); (95% CI); p-value	B path (SE); (95% CI); p-value	Indirect effect (SE); (95% CI); p-value	Direct effect (SE); (95% CI); p-value	Total effect (SE); (95% CI); p-value
Child initiations with a parent	Parent synchrony	0.12 (0.02); (0.08 to 0.16); < 0.001	0.46 (0.08); (0.30 to 0.62); < 0.001	0.05 (0.01); (0.03 to 0.08); < 0.001	0.02 (0.02); (–0.02 to 0.07); 0.341	0.08 (0.02); (0.03 to 0.12); 0.001
Child initiations with a LSA	LSA synchrony	0.11 (0.02); (0.07 to 0.15); < 0.001	0.31 (0.09); (0.14 to 0.49); < 0.001	0.03 (0.01); (0.01 to 0.06); 0.003	0.02 (0.03); (–0.04 to 0.07); 0.508	0.05 (0.03); (0.00 to 0.11); 0.048

This test demonstrates that there is mediation of the effect of treatment on child initiations with a parent at 12 months through parent synchrony at 7 months, and there is mediation of the effect of treatment on child initiations with a LSA at 12 months through LSA synchrony at 7 months.

One of the assumptions underpinning the analysis approach is the absence of measurement error in the mediator. If present, this could cause a bias in the estimation of the B path and the direct effect, and lead to misleading conclusions. To check this assumption, we extended the structural equation models to allow for measurement error in the mediator by using an approach outlined in Pickles et al. 17 This involves fitting a latent variable model for the repeated measures of the mediator, constraining the factor loadings and measurement error variance across time and ADOS-2 group. We allowed the variance of the factors to vary across ADOS-2 group. The mediator in this model is the latent measure of DCMA at 7 months, which is taken to be a measurement error-free construct. As previously, estimates are shown for each ADOS-2 module, and then combined into a total score using an inverse-weighted estimator. A schematic of the model is shown in Figure 11.

FIGURE 11.

Model allowing for measurement error using repeated measures.

The pattern of results, shown in Table 33, is consistent with those seen previously for the effect of treatment on ADOS-2 mediated through parent synchrony at 7 months. There is no evidence of an association between the mediator and outcome, implying that measurement error was not attenuating this association previously. We display this for one mediation analysis for illustrative purposes.

TABLE 33 - Mediation of treatment to parent synchrony at 7 months to end-point ADOS-2 at 12 months

ADOS-2	A path (SE); (95% CI); p-value	B path (SE); (95% CI); p-value	Indirect effect (SE); (95% CI); p-value	Direct effect (SE); (95% CI); p-value	Total effect (SE); (95% CI); p-value
Module 1 (non-verbal); n = 96	1.48 (0.59); (0.31 to 2.64); 0.013	0.06 (0.51); (–0.95 to 1.06); 0.909	0.09 (0.76); (–1.40 to 1.57); 0.910	–0.53 (1.02); (–2.53 to 1.46); 0.599	–0.45 (0.60); (–1.63 to 0.73); 0.457
Module 1 (verbal); n = 82	2.14 (0.63); (0.91 to 3.36); 0.001	3.36 (8.41); (–13.12 to 19.83); 0.690	7.17 (18.95); (–29.97 to 44.31); 0.705	–8.25 (18.87); (–45.23 to 28.73); 0.662	–1.13 (0.85); (–2.79 to 0.53); 0.181
Module 2, n = 59	1.84 (0.73); (0.40 to 3.28); 0.012	–2.08 (4.95); (–11.78 to 7.62); 0.674	–3.83 (10.06); (–23.55 to 15.89); 0.704	5.34 (10.18); (–14.62 to 25.30); 0.600	1.51 (0.86); (–0.17 to 3.18); 0.079
Overall	1.80 (0.53); (0.75 to 2.85); 0.001	0.05 (0.51); (–0.94 to 1.04); 0.925	0.23 (1.17); (–2.06 to 2.53); 0.842	–0.50 (1.01); (–2.47 to 1.47); 0.620	–0.16 (0.41); (–0.97 to 0.65); 0.699

Figure 12 displays a summary of the mediation analysis findings from Tables 27–33. These need to be understood in relation to the hypothesised chain of effect (found in our previous studies7,14,17,18,54) from the intervention to (1) caregiver adult response to the child in the dyadic context, measured at 7 months; (2) child response to the caregiver adult in that same context, also measured at 7 months; and (3) child response to another adult (researcher) in a generalised research context, measured at 12 months.

FIGURE 12.

Summary of mediation analysis findings from Tables 27–33. Statistically significant paths (p < 0.05) are shown as dark-blue arrows.

In all of these analyses, there was a consistent statistically significant effect of treatment on caregiver synchronous response with the child, indicating that the PACT-G intervention led to a between-group difference effect on the initial target mechanism, as found in our previous studies. The analyses also demonstrated the second hypothesised effect, that is from caregiver synchrony to later child communication initiation, so that the effect on child initiation was mediated through synchrony with the respective partner.

We did not find any evidence of a downstream effect from the mediator to the ADOS-2 outcomes. However, in keeping with our previous work, there was a significant association between child initiation with both the parent and the LSA in home and educational settings, and later symptom change on the BOSCC at 12 months, and a significant indirect effect was estimated through the parent synchrony measure.

Aims

This trial addressed a number of contemporary challenges in autism intervention research. The first was extending a manualised, largely clinic-based treatment within the naturalistic community settings of home and education, and, for added efficiency, including use of remote video-conferencing sessions, in addition to in-person delivery. Second, we introduced parallel home and education therapy for the child, with the aim of reinforcing the child’s developmental learning across contexts and persons. The purpose of this was to address a key difficulty, suggested in many single-subject design studies, that autistic children are thought to have in generalising and implementing acquired skills across contexts11 – a difficulty assumed to place significant limitations on the functional effectiveness of interventions in development. Third, the study extends the intervention into the early school years for children with intellectual disability and autism, whereas most early psychosocial interventions to date have been implemented in the preschool period. In addition, our large and detailed Mechanism study, one that we believe to be by far the most extensive yet in the autism field and perhaps across wider developmental science, provided the opportunity to, for the first time, look in detail at autism symptoms across different contexts and, from our mediation analysis, to investigate processes of generalisation of change through intervention.

This trial addresses current NHS priorities in child development and mental health, for instance the emphasis on therapy delivery within educational contexts in The NHS Long Term Plan. 105 The trial’s focus on autism symptoms as the primary outcome is consistent with our previous work in addressing the core defining characteristics of autism, and supports the mechanistic approach to analysing both proximal and distal effects of intervention and mediation. The data-rich, repeated-measures design provided enhanced power to account for the measurement error in behavioural studies of this kind and allowed mediation/mechanism modelling of the findings.

Strengths and limitations

Families, clinicians and education

The authors gratefully acknowledge the invaluable contribution of all of the participating families, as well as the participating nurseries and schools and referring clinicians.

PACT-G group, therapy, data collection, trial management and administration

Hilary Beach (Manchester University NHS Foundation Trust), Claire Bennett (University of Manchester), Imogen Crook (University of Manchester), Hannah Danvers (Manchester University NHS Foundation Trust), Kate Dartnall (Manchester University NHS Foundation Trust), Hannah Foote (Guy’s & St Thomas’ NHS Trust), Jessica Graham (Newcastle University), Sarah Jamieson (Cumbria, Northumberland, Tyne & Wear NHS Foundation Trust), Anna Kappa (University of Manchester), Anna Knight (Cumbria, Northumberland, Tyne & Wear NHS Foundation Trust), Jo Lowe (University of Manchester), Ruth Madeley (Cumbria, Northumberland, Tyne & Wear NHS Foundation Trust), Deborah Maskell (University of Manchester), Olivia Mitchell (University of Manchester), Francisca Monteiro (King’s College), London, Cat Papastavrou Brooks (University of Manchester), Amelia Pearson (University of Manchester), Leanne Rogan (Newcastle University), Jessica Rose (Guy’s & St Thomas’ NHS Trust), Lisa Slater (Cumbria, Northumberland, Tyne & Wear NHS Foundation Trust), Susanna Vosper (Manchester University NHS Foundation Trust) and Helen Wilson (University of Manchester).

PACT-G additional data collection and video coding

We gratefully acknowledge the contribution of Alana Barnett, Lydia Johnson-Ferguson, Rachel Lisle, Hafsa Rahman and Hafiza Sadiq. The DCMA coders were Lucy Dempster and Laura Kidd. The UK BOSCC coders were Young Ah Kim, Nicole El Hawi, Chloe Hayes, Leonie Luk, Laurie Preston, Rachel Randle and Owen Waddington. The US BOSCC coders were Alex Francisco Aspuru, Niki Bahri, Nurit Benrey, Katherine Byrne, Gabrielle Gunin, Alison Holbrook, Alapika Jatkar, Kassandra Martinez, Allison Megale, Nicole Rosen and Maira Tafolla.

We gratefully acknowledge invaluable training and outstanding collaboration from Professor Cathy Lord, Dr Alison Holbrook (UCLA), Dr Sue Fletcher Watson (Edinburgh) and Sheri Stegall (Western Psychological Services, USA).

Governance

We sincerely thank the following members of the TSC and support during the work: Professor Stuart Logan (chairperson), Professor Anne O’Hare, Professor Liz Pellicano, Professor Emily Jones, Louisa Harrison and Kellie Bell for their valuable guidance. We also thank the members of the Data Monitoring and Ethics Committee, Professor Paul Ramchandani (chairperson), Professor Amanda Farrin and Professor Jacqueline Barnes, for their helpful contribution.

Sponsor and monitor

Manchester University NHS Foundation Trust, Central Research Office, Nowgen Building, Grafton Street M13 9WL. Telephone: 01612764125.

Trial adoption

By NIHR Clinical Research Network: March 2016.

Final trial protocol

V6, July 2018; available from corresponding author (see Appendix 5 for summary of protocol amendments during the trial).

Trial website

http://research.bmh.manchester.ac.uk/pactg/ (accessed June 2021).

Contributions of authors

Jonathan Green (https://orcid.org/0000-0002-0143-181X) (Professor, Child Psychiatry) conceived and led the design of the study and the application for funding, and was chief investigator. He was responsible for the drafting and revision of the report and for the final report submission.

Kathy Leadbitter (https://orcid.org/0000-0002-0744-2800) (Research Fellow, Psychology) conceived and designed the study, and managed and conducted the trials. She contributed to training, supervision and quality assurance of data collection and observational coding; the interpretation of results; and the writing of the report.

Ceri Ellis (https://orcid.org/0000-0002-8735-9993) (Research Associate, Psychology) contributed to the trial design, data collection, coding, data entry, and the writing and editing of the report. She also provided comments on drafts.

Lauren Taylor (https://orcid.org/0000-0002-1821-6163) (Research Assistant, Psychology) was responsible for overseeing the day-to-day running of the research aspects of the study at the London site. This involved participant recruitment and retention, data collection, data entry and data management.

Heather L Moore (https://orcid.org/0000-0002-2999-8332) (Research Assistant, Psychology) contributed to the design of the study and the evaluation of the feasibility and acceptability of the pilot study, and was responsible for data collection and data entry at the Newcastle upon Tyne site.

Sophie Carruthers (https://orcid.org/0000-0003-1616-415X) (Research Assistant, Psychology) contributed to the acquisition, analysis and interpretation of data; and contributed to the drafting of parts of the report.

Kirsty James (https://orcid.org/0000-0002-2657-8148) (Research Assistant, Statistics) co-analysed the quantitative data for the primary and secondary outcomes of the trial.

Carol Taylor (https://orcid.org/0000-0003-3317-2207) (Research Assistant, Speech and Language Therapy) contributed to the design of the study, contributed to the development of the intervention and its adaptation for schools, evaluated the feasibility and acceptability of the pilot study and contributed to the design. She also assessed and monitored the intervention fidelity and the reliability of measures, and contributed to the writing of the report.

Matea Balabanovska (https://orcid.org/0000-0002-3585-1935) (Research Assistant, Psychology) contributed to data acquisition (a substantial part of the BOSCC data, and other data collected during clinic assessments) and the management of BOSCC coding, including main coding processes, video allocations, reliability calculations and also had a minor contribution to the writing of the report (the breakdown of BOSCC videos).

Sophie Langhorne (https://orcid.org/0000-0003-1293-3310) (Research Assistant, Psychology) contributed to observational coding of DCMA data, quality assurance of observational video-coding data and the writing of the report.

Catherine Aldred (https://orcid.org/0000-0002-9686-8685) (Lecturer, Speech and Language Therapy) contributed to the design of the study, the development of the intervention and its adaptation for schools, older children and online use. She was responsible for the training and supervision of the RAs and therapists, the assessment and monitoring of intervention fidelity, liaising with stakeholders, resolving issues and data collection, and contributed to the writing of the report.

Vicky Slonims (https://orcid.org/0000-0003-3339-2365) (Consultant, Speech and Language Therapy) contributed to the original ideas, as well as the design and planning of the study, the development, delivery, measurement and evaluation of the therapy; and contributed to the interpretation and reporting of results, and the plans for implementation and dissemination.

Victoria Grahame (https://orcid.org/0000-0003-4541-7574) (Consultant, Clinical Psychology) was the clinical lead at the Newcastle upon Tyne site, provided weekly supervision to the therapists and also contributed to writing of this report.

Jeremy Parr (https://orcid.org/0000-0002-2507-7878) (Professor, Paediatrics) was the co-investigator and contributed to the funding application, trial design and delivery, attended local and trial co-investigator meetings, co-supervised the Newcastle upon Tyne research team and contributed to drafts of the final report.

Neil Humphrey (https://orcid.org/0000-0002-8148-9500) (Professor, Education) consulted on the adaptation for education/school settings and on the implementation-related aspects of the study.

Patricia Howlin (https://orcid.org/0000-0002-1804-6053) (Professor, Psychology) advised on study design and implementation, assessed treatment fidelity and contributed to the writing of the final report.

Helen McConachie (https://orcid.org/0000-0002-0713-3987) (Professor, Psychology) contributed to the design and planning of the study, co-led and supervised the coding of adult–child interactions and contributed to the writing of the report.

Ann Le Couteur (https://orcid.org/0000-0001-9991-3608) (Professor, Child Psychiatry) contributed to the conception and design of the research and, as research lead for the Newcastle upon Tyne site, was responsible for the training and supervision of the RAs based in Newcastle upon Tyne. She also contributed to the training of RAs across all sites on some of the research measures, contributed to the interpretation of the study findings and to the writing and reviewing of the content of the final report.

Tony Charman (https://orcid.org/0000-0003-1993-6549) (Professor, Psychology) obtained funding, was involved in designing the study and then read, made revisions to and approved the final version of the manuscript.

Richard Emsley (https://orcid.org/0000-0002-1218-675X) (Professor, Methodology and Statistics) contributed to the design of the trial, co-conducted the statistical analysis and also contributed to reading, evaluating and approving the final report.

Andrew Pickles (https://orcid.org/0000-0003-1283-0346) (Professor, Methodology and Statistics) contributed to the design and implementation of the study and led the majority of the analysis and reporting.

Publication

Green J, Leadbitter K, Ellis C, Taylor L, Moore HL, Carruthers S, et al. Combined social communication therapy at home and in education for young autistic children in England (PACT-G): a parallel, single-blind, randomised controlled trial. Lancet Psychiatry 2022;4:307–20.

Data-sharing statement

We will make data available to the scientific community with as few restrictions as feasible, while retaining exclusive use until the publication of major outputs. All data request should be submitted to the corresponding author for consideration. Access to anonymised data may be granted following review.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Disclaimers

This report presents independent research. The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, the MRC, the EME programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, the EME programme or the Department of Health and Social Care.

Change in teaching assistant

Changes in the school-based interaction partner during the course of the trial occurred more frequently than expected. Eighty-five (34.3%) children experienced a change in LSA between baseline and the mid-point, and 94 (37.9%) children between the mid- and end point. Twenty-two children experienced changes at both time points.

Structural equation model

Generalisation was examined by fitting models of the form shown in Figure 3. The model was stratified as a two-group model by the BOSCC module and estimated by maximum likelihood. In the light of changes in interaction partner, the model included dummy variables to allow for a mean difference where a change in partner had occurred. The models were estimated in Mplus using maximum likelihood estimation with robust standard errors, and standardised estimates are reported. At the mid-point, the path from parent to LSA is included but the reverse reciprocal was not significantly different from zero and, therefore, not included.

The model was run with the social communication subscale of the BOSCC. The length and setting of the interaction was considered to elicit too few RRBs to make their separate analysis reliable.

FIGURE 13.

Structural equation model fitted to baseline, mid-point and end-point BOSCC data from parent, teacher and researcher. R, researcher; P, parent. Numerical suffix = month from start of therapy. Circles represent latent variables, squares represent observed variables. Light blue arrows represent the paths from intervention to BOSCC assessment, and orange arrows represent the generalisation paths tested. Covariation paths are dashed double-headed curved paths.

Results

Summary

These analyses were set out to explore the extent to which children’s social communication, as measured by the BOSCC, generalised across contexts from interaction with familiar adults (i.e. parent and LSA) to an unfamiliar setting and unfamiliar adult (i.e. the researcher). This was tested using the social communication subscale of the BOSCC. The initial model, considering only two contexts at a time, showed that social communication with a parent generalised to later interaction with a researcher, such that increased social communication skills in interactions with the parent transferred to improved social skills with the researcher. This appeared also to be true for social communication during interactions with a LSA. In each of these analyses, the generalisation arising from the excluded interaction partner may be considered an omitted confounder. A trivariate analysis with all three raters is required to confirm the effects. When all three contexts were included in the same model, although the generalisation of social communication from parent to researcher remained (both directly and overall including the indirect effect), the effect from LSA to researcher was no longer significant. Generalisation was also shown to be significant for the parent mid-point to LSA mid-point. This analysis provides supportive evidence for cross-context generalisation in behavioural change, particularly from the home setting to educational setting. However, although we account for both measurement error and baseline confounding, our analyses have not yet eliminated the possible confounding effects of context general shared developmental change in autism severity as a source of bias. We hope to further develop our analyses of the mechanisms of cross-context and cross-behavioural (DCMA/ADOS-2) change to address this source of developmental confounding.