Positive behaviour support training for staff for treating challenging behaviour in people with intellectual disabilities: a cluster RCT

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 10/104/13. The contractual start date was in November 2012. The draft report began editorial review in May 2017 and was accepted for publication in October 2017. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Rumana Omar reports membership of the Health Technology Assessment General Board. Michael King is a member of the PRIMENT Clinical Trials Unit, which is funded by the National Institute for Health Research.

Copyright statement

© Queen’s Printer and Controller of HMSO 2018. This work was produced by Hassiotis et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2018 Queen’s Printer and Controller of HMSO

Introduction

Intellectual disability (ID) is characterised by significant impairments in cognitive, social and practical skills including activities of daily living and work tasks. 2 Prevalence estimates suggest that as many as 2 in 100 adults have ID worldwide, although prevalence rates may vary according to country, age and socioeconomic status. 3 Individuals with ID have complex and unique care needs, given their pronounced vulnerability to biological, psychological and environmental stressors compared with the general population. 4 A reduced ability to cope with these stressors makes individuals with ID more likely to experience challenging behaviours. 5 Between 10% and 15% of adults with ID present with challenging behaviour, most commonly aggression, which may result in long-term hospitalisation, often in out-of-area facilities, restrictive care practices and neglect as well as an increase in receipt of antipsychotic medication and service use. 6–9 Therefore, effective treatment approaches for challenging behaviour are vital. However, the wide range of factors that could trigger challenging behaviour, as well as the many psychiatric conditions associated with ID, make a clear understanding of challenging behaviour in ID difficult. Although there have been a number of interventional approaches over the years, the literature on pharmacological and psychosocial interventions is limited to observational studies and single-site randomised controlled trials (RCTs) with short follow-up periods, which therefore are subject to significant bias. 10–13 Furthermore, these studies did not include a health economic evaluation, which is important in guiding policy-makers and service commissioners.

Cost of challenging behaviour to society

The presence of challenging behaviour increases the cost of care for people with ID, mostly due to increases in support and long-term inpatient care, often in expensive out-of-area placement;14–17 and also increases family-carer burden. 18 Hunter19 argues that the limited economic data currently available make use of information based on services that are many years old and superseded by advances in community health and social care.

The psychosocial intervention with the greatest evidence base for efficacy is Positive Behaviour Support (PBS). It is described in detail in the following section.

Positive Behaviour Support

Positive Behaviour Support has arisen out of the tenet that challenging behaviour is shaped by personal and psychological experiences and helps the person to exert some control over their environment. It is considered a cornerstone of good-quality care that is also supported by policy objectives to maintain individuals in their local communities and encourage commissioning that rewards skilled care provision. 20,21 The challenging behaviour may be a response to environmental cues or ‘schedule induced’, that is, may be the result of interactions between the individual and the environment. 22 These concepts have influenced the definition of challenging behaviour as ‘behaviour of such intensity, frequency or duration that the physical safety of the person or others is placed in serious jeopardy or behaviour which is likely to seriously limit or deny access to the use of ordinary community facilities’. 23 PBS is ‘the application of the science of applied behaviour analysis (ABA) in the support of people with challenging behaviour’ and is an essentially complex intervention in terms of its components and range of outcomes. 24 PBS is a flexible, multicomponent approach that takes a lifespan perspective and emphasises prevention, and so far is the only treatment with evidence of efficacy. It focuses on reducing challenging behaviour and improving quality of life in individuals with intellectual disabilities25 and other population groups across the lifespan, for example in education or in individuals with brain injury. 26–29 It aims to help professionals and family or paid carers understand the behaviour that an individual displays by focusing on the individual’s interaction with his/her environment and identifying the context in which the behaviour takes place by means of a functional assessment that should lead to personalised, non-restrictive approaches to challenging behaviour designed to foster prosocial actions.

Staff training in Positive Behaviour Support

Positive Behaviour Support can be implemented in a number of ways, including by a single practitioner co-ordinating all elements of the framework and leading each stage of the process,30–32 by professional teams in which different members contribute to different elements of the PBS framework or process,13,33 and systemwide, whereby the PBS framework is implemented at varying levels of intensity via a tiered model of prevention that covers an entire organisation or geographical area. 33–35 Specific staff competencies have an impact on the effectiveness of PBS in improving challenging behaviour. However, as a result of the resources required to deliver PBS, this type of support is not always available. Despite PBS being a well-known intervention framework, only about half of adults with ID and challenging behaviour may receive it. 36 Even in areas with specialist support teams providing applied behaviour analysis or PBS, patients often have to wait several months to receive help. Therefore, training paid carers and professional staff in PBS is thought to increase awareness of good care for this population group and extend the expertise in managing challenging behaviour in the community. McClean et al. 36 and Grey and McClean37 have reported on training 132 paid carers in a non-randomised clinical study (n = 60). The authors found significant reductions in challenging behaviour in groups supported by trained carers compared with controls. However, the instrument used to measure the primary outcome does not have established psychometric properties and the study was uncontrolled and included training paid carers (rather than professionals) who are likely to require a different set of skills and knowledge from the outset. The authors estimated that paid carer training in PBS may lead to savings of €2000 per person treated.

A pilot RCT of PBS incorporating applied behaviour analysis delivered by a specialist behaviour team in one area in England showed significant reductions in irritability, lethargy and hyperactivity. 11 A naturalistic 2-year follow-up of the same cohort showed a continued positive effect of the intervention on reducing challenging behaviour compared with TAU. 13

The National Institute for Health and Care Excellence (NICE) in the UK has produced a guideline38 that recommends the implementation of PBS in routine care and highlights the importance of working with the individual and their carers, understanding the function of the behaviour and ensuring that interventions are provided in the least restrictive manner. However, it has been acknowledged that staff in community ID services are not sufficiently skilled to deliver PBS. Training programmes for front-line staff have been developed and delivered nationally, showing an increase in knowledge and perceived confidence in understanding and managing challenging behaviour. 39

In the light of the widespread implementation of PBS, a rigorous evaluation of the clinical effectiveness of staff training in PBS was required. Furthermore, the economic evaluation would test whether or not any reductions in challenging behaviour resulted in reduced service-use costs and improved health-related quality of life. Therefore, we conducted a real-world multicentre evaluation of the clinical effectiveness and cost-effectiveness of staff training in PBS for treating challenging behaviour in adults with ID compared with TAU in ID services in England.

Trial objectives

Rationale for the long-term follow-up

It has been suggested that some treatments may have delayed beneficial or harmful effects, which may not be known until well after the trial has been completed, showing that the true value of a therapy may change in the light of long-term data. 41,42 There is some evidence from cancer research, cardiovascular medicine and mental health research for long-term treatment effects. A recent RCT of prevention of depression in primary care43 showed little difference in the incidence of major depression after 6 months, but a substantial reduction after 18 months. The collection and analysis of longitudinal data can provide further insight into not only clinical, but also economic, outcomes.

The PBS study is the first large-scale RCT of a complex behavioural intervention for adults with ID and challenging behaviour. Conducting an additional final follow-up assessment would provide further information about the clinical and economic long-term outcomes as well as any potential treatment effect of PBS compared with TAU over time, and would therefore contribute to the existing knowledge of evidence-based care for people with ID who have challenging behaviour.

Trial design

This was a multicentre, single-blind, parallel two-arm cluster RCT evaluating clinical outcomes of manual-based face-to-face staff training in PBS for managing challenging behaviour in adults with ID. A cluster randomised designed was deemed appropriate for this type of complex educational intervention in order to avoid transfer of the intervention skills between the two different arms. 44

Sample size

The primary outcome is the ABC-C_T measured repeatedly at 6 and 12 months following recruitment. The pilot study11 generated a mean baseline Aberrant Behaviour Checklist – Community (ABC-C) score of 45.4 [standard deviation (SD) 26.4]. A SD reduction of 0.45 on the ABC-C score in the PBS arm compared with the control arm is considered to be clinically important. Using an analysis of covariance approach, based on a correlation of 0.48 between the baseline and post-intervention ABC-C measurements (estimated from the pilot study), 80 participants per arm were required to detect a SD difference of 0.45 with 90% power and 5% significance level. Inflating for clustering within the community ID services, using the formula proposed by Eldridge et al. ,45 which accounts for variable cluster sizes and an intracluster correlation of 0.062 (estimated from the pilot study), an average cluster size of 12 (we expected it to be 13, however, allowing for 10% attrition and rounding it up to the nearest integer we have used a cluster size of 12 in our calculations) and a SD for the cluster size of 3, a total of 276 participants were required. However, this sample size can be reduced as each participant provided two measurements of ABC-C score. Using a correlation of 0.6 between the 6 and 12 months’ post-intervention ABC-C measurements (estimated from the pilot study) and a cluster size of 2, a total of 442 ABC-C measurements; thus, 221 participants, were required. In performing this calculation, we have assumed that there would be no treatment by time period interaction over 12 months, which was supported by the pilot studies. 11,13 To allow for 10% attrition over the 12-month period, a total of 246 participants had to be recruited in to the trial thus requiring 19 clusters. The sample size calculation was based on the program and formulae in Stata® version 12 (StataCorp LP, College Station, TX, USA).

Service and participant recruitment

Twenty-three community ID services in England were recruited to the study. All participating services treat adults with ID and challenging behaviour, which tends to be remitting and relapsing so that patients may experience periods of exacerbation of behaviours but also periods of relative stability. The services were recruited through the Clinical Research Networks across several regions in England (London, Leicestershire, Kent, Surrey, Bradford, and Coventry and Warwickshire), which cover urban, semi-rural and rural areas. The number of registered adults with ID ranged from 100 to 1000 and ID services employed a median of 23 full-time equivalent health and/or social care staff (range 4–70). A maximum of 16 participants (range 5–16) with ID were recruited from each cluster.

Two volunteer health staff in each service (henceforth called therapists) from a variety of professions (i.e. psychiatrists, psychologists, nurses, occupational therapists and speech and language therapists) received PBS training. We considered it beneficial to have two therapists per team, as this would enable peer support and cover in the event that one of the therapists subsequently became unavailable, thus ensuring continuity of the intervention and implementation. Interested staff required the agreement of their line managers to take part.

Each therapist had a maximum caseload of eight individuals at any time during the trial, which meant a maximum of 16 service users having treatment per team. Therapists may have continued to provide generic input for other service users, such as taking part in dysphagia assessments, assessment of capacity or day activities/home environment if practicable. However, as therapists who received PBS training were delivering an intensive intervention (suggested time of 12.5 hours per participant excluding travel and writing up), we asked their clinical managers to reduce their routine caseloads to allow them sufficient time to deliver the intervention. We anticipated that, once the training was completed, 12–14 service users per month would be taken on for treatment across the intervention sites.

Randomisation and masking

Once the clusters were recruited and potential participants had been screened for eligibility by their participating ID services and had provided verbal consent to be approached about the study, they were randomised using an independent web-based randomisation system (Sealed Envelope Ltd, London, UK) and random permuted blocks on a 1 : 1 basis. The following data were taken into account in our randomisation planning: team size (number of full-time equivalent staff), clusters located in or out of London and number of service users registered per team (cluster). We calculated the ratio of staff to service users in in-London and out-of-London teams and no difference was found. We stratified the randomisation by calculating the staff-to-patient ratio for each cluster, thus creating a binary factor that indicated whether a cluster was below or above the median ratio. A second wave of randomisation was prepared because of delays in the recruitment of some of the participating teams. The sites were informed of their treatment allocation by the trial manager.

Although clusters, participants and carers were aware of arm allocation, the research assistants (RAs) and clinical studies officers (CSOs) from the National Institute for Health Research (NIHR) Clinical Research Network conducting the study assessments were blind to treatment arm allocation. They were asked to guess treatment allocation for each participant at each follow-up assessment and to report any incident of unblinding. In the analysis, we compared treatment allocation guesses to identify potential bias attributable to unblinding.

Possible sources of bias

One possible source of bias we took into account was the transfer of PBS-trained therapists between the intervention and control teams during the trial. We examined the health staff turnover rates in a number of services that had expressed interest in participating and this was well below 13%, which is considered very low based on agreed service performance data such as balanced scorecards. Therefore, the chance of such leaks of the intervention were considered to be very small. Any therapist changes were recorded throughout the study duration and none found a new post within teams in the control arm. We judged that contamination between study arms was unlikely given the more intensive nature of the intervention. However, PBS principles are being taught widely and, therefore, some knowledge of PBS in teams in the control arm was unavoidable. Selection bias resulting from recruiting participants after cluster allocation is revealed was avoided by completing recruitment and screening assessment prior to randomisation. A degree of variation in participant characteristics between the study arms can be accounted for by the cluster RCT design. 46

Inclusion criteria

Exclusion criteria

Interventions

Frequency and duration of follow-up

All participants received a baseline assessment and were followed up at 6 and 12 months after baseline. We allowed for a window of ± 4 weeks around the due date for each follow-up assessment with every individual participant.

Ethics issues, research governance and consent

The study received ethics approval from the National Research Ethics Service Committee London – Harrow (reference 12/LO/1378). Research and development approvals were obtained from all NHS trusts involved in the study. Details can be found in Appendix 2. The study was sponsored by University College London (UCL). It was conducted in compliance with Good Clinical Practice,49 the Research Governance Framework50 and according to the standard operating procedures of the PRIMENT Clinical Trials Unit at UCL. All data were stored securely and anonymised in accordance with the Data Protection Act. 51

The study’s Trial Steering Committee (TSC) consisted of two independent and two non-independent members and met twice per year. The study’s Data Monitoring and Ethics Committee (DMEC) consisted of three independent members and convened twice a year.

Easy-to-read information sheets and consent forms for the study were prepared with the assistance of the study service users reference group Camden SURGE (Speaking Up Rights Group Experts), The Advocacy Project. RAs and CSOs were trained in obtaining informed consent (including assessing capacity to consent), good clinical practice and data collection. Initially, they contacted those who had given verbal consent to be approached about the study by telephone to discuss the study. Subsequently, they sent written information about the study procedures to family or paid carers and participants with ID, and followed this up with a telephone call ≥ 7 days later to discuss whether or not they were interested in taking part in the study. The RAs or CSOs then visited the participants and carers in their home (or place of work for paid carers) and obtained their written informed consent to take part in the study. If the participants with ID lacked capacity as per the Mental Capacity Act 2005,52 a person was identified or nominated to act as consultee on their behalf.

We notified the participant’s general practitioner (GP) of his/her participation in the trial. All of the research team members followed the required risk assessment procedures including the guidelines for risk management and safeguarding processes.

Serious adverse events

Reports of serious adverse events (SAEs) were collected for the duration of the trial. They were defined as events that:

• resulted in death

• were life-threatening

• required hospitalisation or prolonged existing hospitalisation

• resulted in persistent or significant disability or incapacity

• were otherwise considered medically significant by the investigator.

Any events that were related to the trial (i.e. resulted from the administration of any of the research procedures) and were unexpected (i.e. the type of event was not listed in the protocol as an expected occurrence) were reported to the Research Ethics Committee in line with their reporting timelines. The SAE form is shown in Appendix 3.

In addition, we collected information on the following events for participants for the duration of the study: medications taken including psychotropic medication, out-of-area placements and police contacts.

Outcome measures and instruments

Data entry

Data were entered into a study-specific web-based database developed by Sealed Envelope Ltd. Data entry was undertaken by the study researchers. As agreed with the trial statisticians, the trial manager conducted source data verification (SDV) checks on 100% of the primary outcome measure (ABC-C) at all three assessment time points; for secondary outcome measures, 100% SDV checks were conducted for 20% of all study participants at all three assessment time points. The trial manager prepared a SDV check report that was discussed with the trial statisticians. No further SDV checks were required.

Statistical analysis

Blinding and unmasking

The researchers’ treatment arm allocation guesses were evaluated using a chi-squared test.

All statistical tests and CIs are two-sided. Statistical analysis was performed using Stata® version 14 (StataCorp LP, College Station, TX, USA). We conducted all analyses by treatment allocation. A detailed statistical analysis plan was developed and discussed with the Trial Management Group and further agreed with the DMEC and the TSC prior to the analysis of unblinded data.

Health economic evaluation

The primary aim of the economic evaluation was to calculate the mean incremental cost per QALY gained by the intervention arm compared with the control arm from a health and social care perspective. Utility scores calculated from proxy responses to the EQ-5D-Y were used to calculate QALYs over 12 months.

A secondary aim of the economic evaluation was to calculate the mean incremental cost per QALY gained of staff training in PBS compared with TAU from a societal cost perspective. In addition to health and social care costs, the societal perspective includes the cost of housing, criminal justice costs and out-of-pocket costs for health and social care.

The mean incremental cost per change in ABC-C from a health and social care cost perspective and societal cost perspective will also be reported in Chapter 3.

Participants with autism

All participants were screened for the presence of an ASD using the autism subscale of the Mini PAS-ADD. However, as this is unreliable in ascertaining all those who are likely to have an ASD, we examined the clinical record documented by the researchers in the Case Report File. These records stated if participants had a diagnosis or suspected diagnosis of ‘autism spectrum disorder’, ‘ASD’ or ‘Asperger syndrome’ and the clinical records had a higher yield of cases than screening alone (37 vs. 113). We therefore concluded that an autism spectrum group should be defined using clinical data, and used a two-stage process to validate our assumptions: first, two raters (AH and AS) independently grouped the clinical diagnoses and relevant terms into those that could help to define an autism and non-autism group, reaching consensus by discussing any cases that were unclear (see Appendix 5); second, we tested the Mini PAS-ADD ASD scores between the two categories. The autism group had higher overall scores than the non-autism group (mean of 8.14 vs. 5.76; Figure 1). A total of 113 participants were thus identified as having broadly defined autism (46%).

FIGURE 1.

Comparison of Mini PAS-ADD scores between participants with and without autism. Kernel density plot of Mini PAS-ADD score. N, threshold of 4; P, threshold of 1; R, threshold of 3.

Changes to study protocol

The following changes were made to the original study protocol:

• Five community ID teams dropped out prior to randomisation.

• In the Trial Management Group meeting on 10 May 2013, it was agreed that the screening assessment would be completed before randomisation and this was subsequently amended in the study protocol.

• In addition to interviewing staff, service managers and carers, we also intended to interview service users with severe ID as part of the process evaluation. This has been amended in the protocol.

• A protocol addendum describes the rationale and procedure for using video recordings for qualitative interviews with service users with severe ID in order to capture visual responses by service users during the interview.

• A no-cost long-term follow-up of the participants was conducted at a mean of 36 months post randomisation. The rationale for this is described in Chapter 1. The procedures of the long-term follow-up are further described in the following section.

Long-term follow-up

Once the National Research Ethics Service and Health Research Authority approval had been obtained for the long-term follow-up, we contacted all available participants and carers who had taken part in the PBS study to discuss the additional final follow-up assessment with them. We obtained either verbal or written consent for the long-term follow-up assessment prior to conducting it. Verbal consent was obtained from carers and consultees of participants lacking capacity. We obtained written consent from participants who had capacity to consent. For written consent, we followed the same consent procedures as per the original protocol. Prior to obtaining verbal consent, we sent the relevant information sheet in advance and subsequently contacted the person by telephone in order to gain their consent to participate. The verbal consent process was audio-recorded and the recordings were securely stored.

Follow-up assessments were conducted either face to face or over the telephone. The follow-up data are limited to the primary outcome (ABC-C40) as well as service use (CSRI64), health-related quality of life (EQ-5D-Y58) and medication use including psychotropic medication (Table 3).

Furthermore, we collected information on SAEs including hospital admissions. A total of 184 participants (75%) were seen at a mean of 36 months after entry to the study (range 19–44 months). Fifty-nine participants dropped out (reasons include refused consent, died, were uncontactable and did not returned consent forms). The PRIMENT Clinical Trials Unit continued to provide statistical and health economical support for the analyses. The DMEC and TSC agreed to, and remained involved in, the study overview during the long-term follow-up.

Process evaluation

The aim of process evaluations was to facilitate the understanding of complex observed phenomena needed to guide policy-makers in allocating resources and contributing to improvements in delivery and implementation of evidence-based clinical practice. 73

Interventions usually aim to alter the regular functioning of a system with unpredictable consequences and non-linear outcomes. 74 The Medical Research Council framework73 established a series of dimensions to assess the complexity of the intervention:

• the number of skills required to deliver the intervention

• the number of groups affected by the intervention

• the variability and number of outcomes (i.e. unattended outcomes)

• how flexible the intervention is to be adapted to different circumstances.

Process evaluations can further facilitate the understanding of how the intervention may be implemented elsewhere75 by examining the context, implementation and mechanisms of impact. The process evaluation covered the domains of (1) context, (2) implementation and (3) mechanisms of impact.

Logic model

Although all interventions assume that change is gained on delivery, the theory behind the functioning of interventions is far more complex and it requires a logic model, that is, a description of the mechanisms of the intervention to allow the consideration of all of the interplaying factors and expected outcomes. 73

The logic model is based on Kirkpatrick’s Four-Level Training Evaluation Model. 78 This logic model is further used as a reference framework against which we compare the views of our participants to assess the effectiveness of the training in PBS (Figure 2), comprising the following.

• Level 1: inputs. This level measures what has been put in place to enable the carrying out of the training and intervention. This includes the quality and number of professionals acting as therapists, the trainers employed and the type of training delivered, the quality and quantity of support provided by the PBS research team (teleconference meetings, telephone calls and site visit), the quality and type of support provided for clinical supervision for therapists training in PBS, the quality and type of mentoring scheme offered by the trainers throughout the engagement of therapists with participants for the duration of the trial.

• Level 2: processes. This level measures how the training was delivered and what has been delivered.

• Level 3: actions. This level measures what elements of the interventions have been implemented by therapists and the quality of the plans implemented.

• Level 4: results. This level measures what outcomes have been achieved as a consequence of the delivery of PBS. This last level of investigation takes into account the qualitative interviews of stakeholders in the intervention arm.

FIGURE 2.

Logic model for PBS-based staff training intervention.

Each level measures key dimensions of delivery of the intervention: the initial inputs provided in order for the intervention to be delivered, the training received by therapists, the intervention including the specific components implemented and the resulting outcomes.

Patient and public involvement

A local group of service users with ID from the Camden SURGE, The Advocacy Project, provided advice for the entire duration of the trial. One of the study’s RAs and the chief investigator met with the group on a regular basis. The group provided support for the development of accessible information materials for the study instruments, participant information sheets and consent forms, the study website, topic guides for qualitative interviews, recruitment and the dissemination of findings. Reports from the meetings between the RA and the group were provided to the TSC.

Mrs Vivien Cooper, OBE, the founder and Chief Executive of the Challenging Behaviour Foundation, is one of the study’s co-applicants. Mrs Cooper was involved in the development of the study, advised on recruitment, reviewed study-related documentation such as information sheets, consent forms and qualitative interview topic guides and was involved in the dissemination of the findings.

The study team produced a newsletter twice a year that was disseminated to the participants, their family or paid carers and the local investigators.

Participants

A total of 23 clusters took part in the study, of which 11 were allocated to the intervention plus TAU arm and the remaining 12 to TAU-only arm. In the 11 intervention clusters, one therapist provided training in four clusters, two therapists provided training in another four clusters and three therapists provided training in the remaining three clusters. A total of 382 potential participants were screened, of whom 246 (64%) consented to take part in the trial. One participant was erroneously consented as they did not meet the inclusion threshold on the ABC-C_T and, therefore, was excluded from the analysis.

The median number of participants who were recruited per cluster was 13 [interquartile range (IQR) 6–14]. The details are shown in the trial Consolidated Standards of Reporting Trials (CONSORT) flow diagram (Figure 3). Recruitment took place from 2 June 2013 to 24 November 2014. The 6-month follow-up assessments were conducted between 10 December 2013 and 21 May 2015, and 12-month follow-up assessments were conducted between 3 June 2014 and 30 November 2015.

FIGURE 3.

Trial flow chart. n_fc, number of family carers; n_pc, number of paid carers; n_su, number of service users; n_t, number of teams. From Hassiotis A, Poppe M, Strydom A, Vickerstaff V, Hall IS, Crabtree J, et al. Clinical outcomes of staff training in positive behaviour support to reduce challenging behaviour in adults with intellectual disability: cluster randomised controlled trial. Br J Psychiatry, vol. 212, iss. 3, pp. 161–8, 2018,84 reproduced with permission.

From Hassiotis A, Poppe M, Strydom A, Vickerstaff V, Hall IS, Crabtree J, et al. Clinical outcomes of staff training in positive behaviour support to reduce challenging behaviour in adults with intellectual disability: cluster randomised controlled trial. Br J Psychiatry, vol. 212, iss. 3, pp. 161–8, 2018,84 reproduced with permission.

A total of 215 participants (87%) completed the 6-month follow-up and 225 (92%) completed the 12-month follow-up assessments. There was no difference in attrition between the arms (7% in the intervention arm and 9% in the control arm). Table 4 shows the baseline demographic and clinical characteristics of the participants.

Primary outcome

At baseline, the median ABC-C_T was 60 (IQR 43–80) in the intervention arm, compared with 68.5 (IQR 47–87.5) in the control arm. In the intervention arm, the median ABC-C_T reduced to 50.5 (IQR 30–75) at 6 months and to 49.0 (IQR 32–73) at 12 months. In the control arm, the median ABC-C_T was 54 (IQR 37–81) at 6 months and 55 (IQR 42–75) at 12 months. The primary model used 439 ABC-C_T measurements from 233 participants over the two follow-up time points. There was no statistically significant difference between arms in ABC-C_T over 12 months (mean ABC-C_T difference –2.41, 95% CI –8.79 to 4.51; p = 0.528) (Table 5 and Figure 4).

FIGURE 4.

Aberrant Behaviour Checklist – Community total score over 12 months. From Hassiotis A, Poppe M, Strydom A, Vickerstaff V, Hall IS, Crabtree J, et al. Clinical outcomes of staff training in positive behaviour support to reduce challenging behaviour in adults with intellectual disability: cluster randomised controlled trial. Br J Psychiatry, vol. 212, iss. 3, pp. 161–8, 2018,84 reproduced with permission.

From Hassiotis A, Poppe M, Strydom A, Vickerstaff V, Hall IS, Crabtree J, et al. Clinical outcomes of staff training in positive behaviour support to reduce challenging behaviour in adults with intellectual disability: cluster randomised controlled trial. Br J Psychiatry, vol. 212, iss. 3, pp. 161–8, 2018,84 reproduced with permission.

The intracluster correlation coefficient (ICC) for the ABC-C_T at the service level was 0.021 (95% CI 0.001 to 0.286). The ICC for the repeated measures within participants was 0.625 (95% CI 0.542 to 0.702).

Subgroup analysis

The estimates of the intervention effect on participant sociodemographic and clinical characteristics are shown in a forest plot (Figure 5).

FIGURE 5.

Subgroup analysis. From Hassiotis A, Poppe M, Strydom A, Vickerstaff V, Hall IS, Crabtree J, et al. Clinical outcomes of staff training in positive behaviour support to reduce challenging behaviour in adults with intellectual disability: cluster randomised controlled trial. Br J Psychiatry, vol. 212, iss. 3, pp. 161–8, 2018,84 reproduced with permission.

From Hassiotis A, Poppe M, Strydom A, Vickerstaff V, Hall IS, Crabtree J, et al. Clinical outcomes of staff training in positive behaviour support to reduce challenging behaviour in adults with intellectual disability: cluster randomised controlled trial. Br J Psychiatry, vol. 212, iss. 3, pp. 161–8, 2018,84 reproduced with permission.

Sensitivity analysis

A series of analyses were conducted as follows: (1) adjusting for area deprivation, (2) adjusting for the nature of the respondent (participant or carer), (3) adjusting for the unbalanced baseline characteristics (ethnicity and participant’s cohabitant), (4) adjusting for the percentage of PBS plans written, (5) a model including two random effects and (6) imputing missing values with ‘Baseline Observation Carried Forward’. All of these analyses gave similar results, with differences in ABC-C_T between the arms ranging from –3.45 to –0.81. None of the service user baseline demographics nor the presence of a mental disorder predicted missing data and, therefore, no further analyses were conducted.

Exploratory multivariate analyses

Multivariate analysis examined the effect of training staff in PBS on the individual domains of the ABC-C. The inappropriate speech domain was not included in the multivariate model as it had low correlations (0.300, 0.094, 0.175 and 0.360) with the (1) irritability, agitation and crying, (2) lethargy and social withdrawal, (3) stereotypic behaviour and (4) hyperactivity and non-compliance domains, respectively. The intervention had a similar effect on all four domains; it varied from a standardised difference of –0.016 (95% CI –0.22 to 0.19) for the lethargy and social withdrawal domain to –0.050 (95% CI –0.25 to 0.14) for the stereotypic behaviour domain between the two arms. These analyses are shown in Table 6.

Secondary outcomes

There was no difference in the prevalence of mental disorders (measured using the Mini PAS-ADD) or frequency of activities (measured using the GCPLA) over 12 months. In total, 69 family carers were included in the study, 19 in the intervention arm and 50 were in the control arm. The majority (n = 59, 86%) were female with a median age of 54 years (IQR 48–59 years). As a result of the small numbers in the intervention arm, only descriptive analyses of these secondary outcomes were performed. There was no difference in the Uplift/Burden Scale and GHQ-12 scores over the 12 months. A total of 175 paid carers took part in the study, 89 in the intervention arm and 86 in the control arm. Two-thirds of the paid carer participants (n = 108, 67%) were female and the median age was 41 years (IQR 32–53 years). Over the 12 months, 86 paid carers (49%) left their posts (control arm, n = 49; intervention arm, n = 37) and, therefore, no further analyses were carried out. Details of the treatment effect on the secondary outcomes are given in Table 7.

Subsample with autism spectrum disorders

A total of 113 participants (46.1%) constituted the ASD+ group (control arm, n = 66; intervention arm, n = 47) and 132 participants constituted the ASD– group (control arm, n = 71; intervention arm, n = 61). The baseline demographic and clinical characteristics of a subgroup of participants with broadly defined ASD compared with those without can be found in Table 8.

At baseline, the median ASD+ ABC-C_T in the intervention arm was 66 (IQR 45–85) compared with 75 (IQR 51–93) in the control arm. In the intervention arm, the median ABC-C_T reduced to 55 (37–78) at 6 months and to 53 (35–72) at 12 months. In the control arm, the median ABC-C_T was 62 (IQR 43–90) at 6 months and 60 (IQR 43–66) at 12 months. The difference in the reduction in challenging behaviour over 12 months between the intervention arm and the control arm was not statistically significant (mean difference –2.10, 95% CI –11.3 to 7.13; p = 0.655) (Figure 6).

FIGURE 6.

Aberrant Behaviour Checklist – Community total score over 12 months: ASD.

A total of 102 participants with an ASD received any type of medication, 74 (65%) received antipsychotic medication and 80 (71%) also received other psychotropic medication.

With regard to the subgroup comparisons between the ASD+ and ASD– groups for common mental disorder (CMD) and severe mental illness (SMI), differences at baseline were not statistically significant (CMD, p = 0.625; SMI, p = 0.615). The analysis over 12 months revealed that the CMD and SMI symptoms were stable in both the ASD+ and ASD– group (Table 9).

Patterns of prescribing did not differ between the ASD+ and ASD– groups, irrespective of the presence of mental health comorbidity.

Eighty-five per cent of individuals with CMD in both the ASD+ and ASD– groups were prescribed medication. Sixty-five per cent of individuals in the ASD+ group received antipsychotic medication, compared with 60% of individuals in the ASD– group, whereas 63% in the ASD+ group received other psychotropic medication (i.e. anxiolytic medication), compared with 66% in the ASD– group.

A similar number of participants with SMIs in the ASD+ and ASD– groups received medication (90% in the ASD+ group vs. 89% in the ASD– group), with 75% of the ASD+ group and 70% of the ASD– group receiving antipsychotic medication, and 75% of the ASD+ group and 78% of the ASD– group receiving other psychotropic medication.

In the group with neither CMD nor SMI (i.e. without any evidence of mental illness diagnosis), 91% of participants from both the ASD+ and ASD– groups were prescribed medication; 70% of the ASD+ group and 65% of the ASD– group were prescribed antipsychotic medication, and 73% of the ASD+ group and 80% of the ASD– group received other psychotropic medication.

Serious adverse events

Twenty-nine participants experienced a total of 45 SAEs, including the death of one participant in the intervention arm. Twenty-six of the SAEs were in the intervention arm and 19 were in the control arm. Two participants were admitted to hospital because of mental illness and challenging behaviour and the remaining SAEs were hospitalisations attributable to physical ill-health including falls, or acute exacerbation of chronic conditions, for example respiratory disease and epilepsy. Thirteen participants (intervention arm, n = 3; control arm, n = 10) moved from their original addresses to a new home because of either closures of previous accommodation or a change in the participant’s needs.

Blinding and unmasking

There were six cases of unmasking researchers to the participant’s trial arm allocation; another researcher collected data from those sites. Researchers accurately predicted the arm allocation of 123 participants (59%) at 6 months and 126 participants (56%) at 12 months; thus, the predictions were considered to be no better than chance.

Health economic evaluation

Cost-effectiveness: incremental cost-effectiveness ratio, cost-effectiveness acceptability curves and cost-effectiveness plane

As reported in Primary outcome, the average decrease in the primary outcome of ABC-C_T was 2.14 (95% CI –8.79 to 4.51). This translates to a mean incremental cost per 1-point reduction in ABC-C over 12 months of £578 from a health and social care perspective and £411 from a societal cost perspective. At a threshold for willingness to pay for a point change in ABC-C_T of £5000, there is an 87% probability that the intervention is cost-effective from a health and social care perspective and an 88% probability that it is cost-effective from a societal cost perspective (Figure 7).

FIGURE 7.

Cost-effectiveness acceptability curve of willingness to pay for a 1-point change in the primary outcome (ABC-C_T).

Over 12 months, the incremental cost per QALY gained of training in PBS compared with TAU is £17,871 excluding medication costs and £16,276 including medication costs from a health and social care perspective. The probability that PBS is cost-effective for a range of values of willingness to pay for a QALY is presented in Figure 8. Figure 9 shows the results of the bootstrap analysis on a CEP. At a threshold for willingness to pay for a QALY of £20,000 per QALY gained, there is a 60% chance that the intervention is cost-effective from a health and social care perspective. When out-of-pocket costs, costs for voluntary services, the criminal justice sector, accommodation and unpaid carer time are included, the incremental cost per QALY gained is £11,566. From a societal cost perspective, there is a 54% probability that training staff in PBS is cost-effective at a cost-effectiveness threshold of £20,000. The lower probability of training staff in PBS being cost-effective from a societal perspective (even although it has a lower ICER) is attributable to the wider variation in costs.

FIGURE 8.

Cost-effectiveness acceptability curve for a range of values of willingness to pay per QALY gained.

FIGURE 9.

Cost-effectiveness plane of bootstrapped values for training in the intervention arm compared with the control arm: health and social care costs only.

Long-term follow-up

Participants and outcomes

The final assessments took place between July 2016 and March 2017. On average, the third follow-up took place 36 months (SD 4.7 months) after the baseline assessment. A total of 184 individuals were seen (intervention arm, n = 79; control arm, n = 105). A total of 50 participants dropped out and a further 11 were uncontactable. Table 14 shows the baseline characteristics of the participants who were seen at 36 months and those who dropped out.

TABLE 14 - Baseline characteristics of participants seen compared with those who dropped out at 36 months

Of those with physical health problems, the number of people with the named problem is given.

Characteristics	Total (N = 245)	Completed 36 months (N = 184)	Missing at 36 months (N = 61)
Demographic
Age (years), median (IQR)	37 (25–51)	38 (25–51)	34 (24–50)
Sex: male, n (%)	157 (64)	118 (64)	39 (64)
Ethnic origin: white, n (%)	176 (72)	131 (71)	45 (74)
Service-reported level of ID, n (%)
Mild	41 (17)	24 (13)	17 (28)
Moderate	77 (31)	64 (35)	13 (21)
Severe	127 (52)	96 (52)	31 (52)
Living situation, n (%)
Living alone	39 (16)	29 (16)	10 (16)
Living with parents/others	206 (84)	155 (84)	51 (84)
Current accommodation, n (%)
Residential	105 (43)	87 (47)	18 (30)
Supported living	69 (28)	51 (28)	18 (30)
Family home	64 (26)	44 (24)	20 (33)
Own flat/house	7 (2)	2 (1)	5 (8)
Clinical
Mini PAS-ADD, n (%)
CMD	117 (49)	83 (46)	34 (57)
SMI	47 (20)	33 (18)	14 (23)
Autistic spectrum	50 (21)	37 (20)	13 (22)
ABS (median, IQR)	48 (29–68)	48 (28–68)	44 (32–71)
WASI score (full scale IQ 4) (N = 95), median (IQR)	44 (40–52)	43 (40–49)	53 (42–65)
Physical health problems, n (%)	180 (74)	132 (72)	48 (81)
Mobilitya (N = 180)	64 (36)	41 (31)	23 (48)
Sensory	43 (24)	33 (25)	10 (21)
Epilepsy	67 (37)	48 (36)	19 (40)
Incontinence	78 (43)	53 (40)	25 (52)
Other	103 (57)	78 (59)	25 (52)
ABC-CT score, median (IQR)
Total score	64 (44–86)	63 (45–86)	65 (43–82)
Irritability (15 questions)	20 (13–29)	21 (13–29)	20 (14–25)
Lethargy (16 questions)	12 (7–21)	12 (7–20)	14 (7–21)
Stereotypy (7 questions)	5 (2–10)	5 (2–10)	6 (2–9)
Hyperactivity (16 questions)	20 (12–26)	19 (12–26)	21 (12–27)
Inappropriate speech (4 questions)	4 (1–8)	5 (1–8)	3 (1–7)

Further reductions in challenging behaviour were noted in both arms of the study but, as previously stated, the mean difference in ABC-C_T between arms was non-significant. The median ABC-C_T was 49 (IQR 32–78) and 38 (IQR 25–56) in the control and intervention arms, respectively. The mean difference was –3.70 (95% CI –9.25 to 1.85; p = 0.191). Figure 10 shows the change in ABC-C_T at all four assessment points. The ABC-C subscale scores at long-term follow-up are shown in Appendix 6.

FIGURE 10.

Aberrant Behaviour Checklist – Community total scores over 36 months.

All other sensitivity and subgroup analyses followed the same pattern as in the main study, revealing no significant differences between the study arms.

Regarding the longer-term impact of the intervention on medication, the proportion of participants in the intervention arm being on any medication, antipsychotic medication and other psychotropic medication remained unchanged over time, as shown in Table 15.

TABLE 15 - Recorded medication at all assessment points

Medications	Time point, n (%)
	Baseline	6 months	12 months	36 months
Any
All	217 (89)	195 (91)	205 (91)	165 (90)
Control	122 (89)	108 (92)	115 (92)	96 (91)
Intervention	95 (88)	87 (89)	90 (90)	69 (87)
Antipsychotic
All	162 (66)	139 (65)	145 (64)	123 (67)
Control	89 (65)	77 (66)	79 (63)	69 (66)
Intervention	73 (68)	62 (63)	66 (66)	54 (68)
Other psychotropic
All	177 (72)	163 (76)	172 (76)	142 (77)
Control	100 (73)	92 (79)	97 (78)	84 (80)
Intervention	77 (71)	71 (72)	75 (75)	58 (73)
Antipsychotic and at least one other psychotropic
All	136 (56)	121 (56)	125 (56)	110 (60)
Control	75 (55)	68 (58)	67 (54)	61 (58)
Intervention	61 (56)	53 (54)	58 (58)	49 (62)
Antipsychotic with no other psychotropic
All	26 (11)	18 (8)	20 (9)	13 (7)
Control	14 (10)	9 (8)	12 (10)	8 (8)
Intervention	12 (11)	9 (9)	8 (8)	5 (6)

Resource use and quality-adjusted life-years

The total cost of health and social care at 36 months for the previous 6 months (excluding the cost of medication and training in PBS) was £1540 in the intervention arm and £1100 in the control arm. The adjusted mean incremental health and social care cost of the intervention compared with TAU is £501 (95% CI –£270 to £1274). From a health and social care perspective, adding 6-month and 12-month costs and PBS training and delivery costs, the mean incremental cost per participant in the intervention arm compared with the control arm is £1931 undiscounted (95% CI £260 to £3402) and £1898 discounted (95% CI £480 to £3318).

From a societal perspective, the adjusted mean incremental total societal cost of the intervention compared with TAU is –£4874 (95% CI –£14,485 to £4736). When 6-month and 12-month costs and PBS training and delivery costs are added, the mean incremental cost per participant is –£1235 undiscounted (95% CI –£19,347 to £16,874) and –£879 discounted (95% CI –£18,405 to £16,648). Details are given in Appendix 7.

At 36 months, complete data were available to calculate QALYs for 73 participants (68%) in the intervention arm and 90 participants (66%) in the control arm. The discounted adjusted difference in utility scores between the two arms was 0.160 (95% CI –0.034 to 0.355). Over the duration of the trial, 12 participants died (intervention arm, n = 5; control arm, n = 7). Including those participants in the calculation of QALYs (assuming a straight line from the participant’s last utility score to 0, representing death), there was a difference in adjusted discounted QALYs of 0.150 (95% CI –0.064 to 0.364 QALYs).

The discounted incremental cost per QALY gained of training in the PBS arm compared with the control arm over 36 months from a health and social care perspective, excluding medication costs, is £12,653. The probability that PBS is cost-effective for a range of values of willingness to pay for a QALY is presented in Figure 11, which shows discounted costs and QALYs. At a threshold for willingness to pay for a QALY of £20,000 per QALY gained, there is a 67% chance that the intervention is cost-effective from a health and social care perspective.

FIGURE 11.

Cost-effectiveness acceptability curve per QALY gained at 36 months (discounted).

Process evaluation

Mechanisms of impact: stakeholder interviews

A total of 13 service users, 11 family carers, 10 paid carers, 12 service managers, 12 therapists and four PBS trainers were interviewed to explore their experience of taking part in the study.

Three rounds of revisions of the codebook were necessary to reach agreement among co-raters for the coding of the interviews. A subsequent inter-rater reliability exercise revealed a range of 0.8–1 for the different subgroups of stakeholders.

The views of the service user advisory group from Camden SURGE, The Advocacy Project, were recorded at regular meetings convened to discuss the interpretation of emerging themes and subthemes from the qualitative interviews. The SURGE members also took the opportunity to reflect on their own past experience of having been in hospital or having received treatment for challenging behaviour and how it matched or differed from the experiences of the study participants.

Summary of main findings

In this cluster RCT, we evaluated the clinical outcomes of a training programme in PBS that was delivered to qualified health professionals, who are specialists in working with adults with ID, supported by a training manual and mentoring to reduce challenging behaviour in adults with ID. To our knowledge, this is the first large-scale, pragmatic RCT of this intervention. The results revealed that training in PBS was not clinically and statistically significant in reducing challenging behaviour, measured by the ABC-C_T over 12 months. Thus, we conclude that the training in PBS of qualified health professionals in this study was no more effective than TAU in reducing challenging behaviour. Secondary outcomes were also similar between the two arms. Crucially, compliance and treatment fidelity to the PBS model, as well as the reach of PBS, appeared to be low and may go some way to account for the lack of effect of the intervention. The 36-month follow-up showed that the differences in the primary and secondary outcomes between the study arms remained non-significant and the initial gains in health-related quality of life also dissipated.

A subsample of participants with broadly defined ASD presented with higher challenging behaviour scores than participants without an ASD. However, the intervention did not significantly reduce challenging behaviour in participants with an ASD.

Resource use was similar across both arms but costs were lower, particularly from a societal cost perspective, although not significantly so, than resource use costs in the control arm. The mean incremental cost per QALY gained for training in PBS compared with TAU was £16,276 from a health and social care cost perspective and £11,566 from a societal cost perspective, with both estimates falling below the cost-effectiveness threshold of £20,000 per QALY. Although this finding is mostly driven by a greater number of QALYs over 12 months for participants in the intervention arm, it should be considered with caution because of the proxies being unblinded to arm allocation and the study not being powered to detect significant differences in costs. The relatively low probability of cost-effectiveness at a threshold of £20,000 per QALY gained is primarily attributable to the large variability in costs between participants, as reflected in the wide 95% CIs for the difference in costs between the intervention and control arms. As a result, there is great uncertainty regarding the cost-effectiveness of the intervention compared with TAU. The relatively high cost per participant of the training and delivery of PBS of £1597 also has an impact on the cost-effectiveness of the intervention.

Process evaluation

Despite the lack of clinical effectiveness, a number of findings from this investigation are quite promising. The training in PBS was very well received by therapists, and the majority of therapists completed the training workshops. With respect to the experience of those who were interviewed, several positive outcomes relating to the training and interventions within the PBS framework were found. Despite their different roles, therapists reported an increased knowledge of the nature of the challenging behaviour experienced by participants and of the intervention itself. The training programme was also reported to be an important asset for the service, as it enhanced the skills of health staff and improved the quality of the care delivered. The PBS framework was perceived as helpful in co-ordinating different approaches delivered by a multidisciplinary team.

Challenges were instead found in practice post training, despite great efforts to improve buy-in and implementation. The setting in which the intervention was delivered was highly complex and the intervention was superseded by, or competed with, other service requirements. This suggests that, although the training was carried out successfully, its delivery by therapists at the level of participants with challenging behaviour and their family or paid carers was inadequate. This may have also been compounded in the few cases in which the person did not show challenging behaviour at the time of contact with the therapists as well as by the need for extra resources to manage the workload.

However, although all of the reviewed PBS plans were rated as unlikely to affect behaviour, some limitations of the BIP-QE II may have had an impact on the ratings. The effectiveness of the BIP-QE II has been mainly evidenced in educational settings and with a young population, whereas the participants taking part in this trial were adults with ID living in a variety of community settings (i.e. supported living and residential care). 85 It is possible that training in PBS may need to be more tailored to the setting and other factors across the lifespan and this, in turn, may improve the quality of the support plans.

Although there is an indication of poor-quality PBS plans, it should be noted that there is a lack of research suggestive of a positive correlation between the (better) quality of PBS plans and better outcomes in the person’s challenging behaviour. Thus, we cannot be certain that this was a definitive reason for the lack of effectiveness.

Paid and family carers and participants with ID reported certain techniques they have learnt to help the participant with ID manage their anger or other behaviours. Such strategies were usually along behavioural lines, for example ‘time out’. However, other aspects of PBS, such as functional analysis and its association with improving the understanding of challenging behaviour, were reported by service managers and therapists. It is possible that lay people and participants with ID are not able to differentiate the varying domains of PBS or even identify them from a list of generic behavioural approaches. For example, the use of a time plan for activities for the participant was seen as helpful in creating structure in their daily routine and, in turn, facilitating the caring process. PBS plans were also valued as a helpful tool that enabled the understanding of the participants’ needs. Other elements that were valued were the relationships with the therapists, which, through non-specific treatment effect, may help to engage individuals and their families and promote a therapeutic alliance.

The preliminary analysis of the impact of the intervention in adults with ID and autism (ASD+) also shows that it was not clinically effective or cost-effective for this group either, although they appeared to have a higher ABC-C_T than the ASD– group. Surprisingly, the ASD+ group reported a lower level of access to any services than the ASD– group, except for art therapy (ASD+ 8% vs. ASD– 6%) and contact with social workers (ASD+ 50% vs. ASD– 49%) and psychologists (ASD+ 18% vs. ASD– 17%).

Finally, medication use did not change overall over time, with the same proportions of participants being prescribed psychotropic medication over 12 and 36 months’ follow-up. This pattern was the same in the ASD+ and ASD– groups.

Results in context

MacDonald and McGill86 conducted the only systematic review to date on outcomes of training staff in PBS. The authors argue that the training in PBS of community services staff increases their competence in managing challenging behaviour and reduces the use of restrictive practices and the reliance on other professional support. However, the research is undertaken within fairly controlled environments with little evidence of how to translate practice into real-world conditions. The therapist professions, study settings and training courses in the present study mirror those reported in the review. 86 Despite the review finding that the training led to reductions in challenging behaviour, when this was measured and reported, none of the included studies used a randomised or quasi-randomised design and the follow-up was limited to 6 months, if included at all. Therefore, previously reported significant effects of the intervention on challenging behaviour are likely to be attributable to study bias. 37

There was a risk that some form of PBS-based care would be gradually adopted in some of the clusters in the control arm during the trial and that this may have reduced any differences between the trial arms. However, we explicitly excluded teams that had implemented PBS provided by practitioners, either embedded in services or through tertiary specialist teams. This was further confirmed by the responses to a survey of the participating ID teams prior to the study commencing, which explored the pre-existing behavioural approaches, training and resources in each cluster. The local and principal investigators at each site reported that they had no specific service pathways relating to challenging behaviour; PBS-trained staff had left the services and specialist teams for challenging behaviour, when previously available, had been decommissioned.

It is possible, nevertheless, that the results of the present study have been affected by improvements in the quality of care in the TAU-only clusters, which may have attenuated the impact of more specialist interventions such as staff training in PBS. Our study began just after the Winterbourne View scandal,21 which exposed the poor care and abuse perpetrated on adults with ID and challenging behaviour in an inpatient facility in England. By the time that the report21 was completed 4 years later, NHS England and the Department of Health and Social Care had made serious efforts to counteract complacency and improve the standards of practice for people with ID. 87 However, such policy changes are slow to be implemented and to reach a sustained local impact; therefore, we argue that such changes are unlikely to account for the findings. Furthermore, we did not measure staff skills or knowledge per se, so those aspects that may have been improved by the training in PBS were not captured. 88,89 It is often repeated that functional analysis is an important element of behavioural approaches and this may not have been carried out consistently in this study. However, a recent multilevel analysis of one experimental study showed that functional analysis does not moderate the relationship between an intervention and its impact on challenging behaviour, so it is unlikely that omission of a functional analysis would have had a significant impact on participant outcomes. This is further supported by McClean and Grey,90 who carried out a 26-month follow-up of paid carers as part of a 5-year rolling PBS training programme. They found that no specific PBS components accounted for reductions in challenging behaviour.

This is the first RCT that we are aware of to conduct an extensive cost-effectiveness analysis of a PBS-related intervention, with most studies in this area predominantly using a before-and-after design. 19 Whereas before-and-after studies have hypothesised a decrease in accommodation costs for participants who are provided with PBS, this study has echoed the findings of other economic evaluations of person-centred interventions for participants with challenging behaviour in that there is no reduction in costs. 16 This is likely to be because of changes in community care for people with challenging behaviour and ID, which require significant changes in the commissioning of appropriate services, therefore taking time to implement fully. Other research in which resource use has been investigated91 suggests that the type of challenging behaviour, for example externalising, and whether or not mental illness/other mental disorder is present, is linked to higher costs. Although we did not look specifically at types of behaviour in this study, some costs may potentially have been driven by the severity of behaviour and/or comorbid conditions. Given the paucity of economic studies in the field of ID in general, and in the context of psychosocial intervention effectiveness in particular,92 our findings provide the first indications of resource use and cost-effectiveness of a widely rolled-out intervention.

Strengths and limitations

Our study had several strengths, including the recruitment of the required number of participants and clusters, testing a single primary outcome, achievement of a low attrition rate at an individual level and an a priori analysis plan, all of which reduce the risk of bias. Furthermore, the ICC for the primary outcome is smaller than originally assumed for the sample size calculation and the sensitivity analyses support the main findings of the study, suggesting that the findings are robust.

Our design aimed to maximise the external validity and generalisability of the study. The trial was conducted within the NHS in 23 community ID services in different parts of England, covering urban, semi-rural and rural areas, from which we recruited 246 service users. Despite the geographical spread of the services, we found that services were not very different. Although there were some minor differences in baseline participant characteristics, adjusting for these variables in the main analysis had no bearing on the study outcomes. Our inclusion and exclusion criteria closely resembled the cases that were likely to be referred for assessment and treatment to those services. The overall median level of challenging behaviour of the participants was higher than that reported in a trial of antipsychotic medication by Tyrer et al. 61

In order to guard against the tendency for the impact of training to dissipate over time, we set up mentoring and peer support during the study period. 1,28,93 Our process evaluation indicated that training in PBS was well received by the therapists and the delivery of PBS was welcome and highly valued by the service managers of participating services, carers and participants with ID.

The study also has limitations, including low fidelity and a less than optimal engagement of the therapists with the intervention. We found that 30% of participants received all elements of PBS as specified in the training and 43.5% received only partial input, mainly observations. In addition, there was a low uptake of the mentoring support that was offered by the PBS trainers. A reason for poor implementation was the difficulty experienced by some of the participating NHS organisations to assist their staff in undertaking the additional responsibilities of the study procedures by failing to reduce therapist caseloads. Therefore, the therapists either undertook the delivery of PBS in addition to their normal workload or were unable to implement it. Furthermore, therapists were embedded in the ordinary community ID team and, therefore, clinical procedures including managing emergencies and interprofessional referrals may have also hindered the delivery of the intervention as intended. This may be a reflection of the realities of implementing PBS in community ID services without the provision of additional resources, such as specific posts for accredited behavioural therapists.

Regarding the lack of impact on the subgroup with broadly defined ASD, it may be that interventions that are not specific to ASD are less likely to help improve behaviour. However, the findings in this respect are preliminary but worth considering in the context of investigating treatment approaches for this population.

There was a considerable turnover of paid carers in the study. This may also have affected some of the delivery of PBS, as the therapists were meant to work closely with family and paid carers, and any change in paid carers would have required the therapists to continue this work with a new carer, possibly having an impact on the continuity of their work. Also, because of the considerable turnover in paid carers and the small number of family carers in the intervention arm, we were unable to further explore the secondary outcomes in paid and family carers.

As a result of the nature of the intervention, it was not possible to blind carers and participants to treatment arm allocations. There were six cases in which researchers were unblinded to a cluster’s treatment arm allocation, which indicates a small possible risk of bias. In such cases, another researcher subsequently collected data from those sites. Moreover, our analysis showed that researchers’ predictions of treatment arm allocation were considered to be no better than chance.

Costing for accommodation was based on very broad assumptions because of poor data on accommodation costs. There was no evidence of changes in participants’ accommodation throughout the study. We were unable to provide any estimates of the impact on employment or state benefits as a result of the intervention due to poor reporting and a large number of missing or inconsistent data. This points to the need for better methods of collecting data on accommodation, employment and state benefits for people with ID. The self-reported questionnaire for unpaid carer time was limited to four questions and it is potentially unreliable as it has not been extensively validated. 94 It was reported that 14 participants did not have challenging behaviour at the time of first contact with the therapist. This may have been attributable to a number of factors such as (1) challenging behaviour being a relapsing–remitting disorder and, therefore, not being acutely present at the time; (2) other interventions may have been delivered by other team members with positive outcomes; and (3) the behaviour being mild enough not to cause any harm or severely reduce quality of life and so treatment was not seen as priority by the carers. Paid carer turnover was significant and limited the undertaking of some of the PBS-related tasks such as goodness of fit, which requires discussion of the plans and their implementation in the patient’s home or activity setting. Finally, because of the design and logistics of the study, we were unable to train new staff to undertake PBS in place of the therapists who left. Therefore, the therapist cohorts were fairly static and any changes meant that participants allocated to the intervention arm did not receive it as we were unable to replace therapists. The study began at a time of increasing awareness of PBS as a preferred psychosocial approach to manage challenging behaviour and ended at a time when training in PBS at scale was implemented. 95 These changes in health and social care service approaches may have closed the gap between the intervention and TAU. There is currently a debate on whether PBS is delivered as it should be or if cursory attention is paid to its delivery across multiple services. Although we had considered this issue at the inception of the trial, we believe that we had offered significant incentives and support to motivate and entice services to participate. However, we were unprepared for the wide variation in delivery that we encountered and the opposition at times from services allocated to the intervention to make adjustments to accommodate research activity. Although our results may be disappointing to many in the PBS community, they nevertheless present a fair account of how PBS as an intervention may be corrupted96 and the extent to which it requires further modification in both content and delivery in order to provide the benefits that its proponents advocate. At 12 months, the intervention appeared to increase QALYs and, in this context, society may be willing to pay a higher price given the intractable problem of challenging behaviour. However, it is also important to take into account the fact that the measures used for cost-effectiveness (i.e. EQ-5D-Y) may have uncertain validity in this population with long-term disability.

Implications for practice

Our study is important because it was randomised and had a longer follow-up time than most studies in ID literature. It raises important issues in relation to the expectations that providing training in PBS to health professionals in community ID services may be sufficient to confer significant reductions in challenging behaviour beyond what is currently achieved by TAU. The UK is relatively unique in the provision of multidisciplinary teams that are specialised in working with individuals with ID.

Our findings are relevant to many professional groups, ranging from service planners for adults with ID to policy-makers who are responsible for setting nationwide directives about the care of adults with ID and challenging behaviour, including that of universal PBS training. For example, the PBS and Autism Training Fund, part of the Positive and Safe initiative that was launched in 2014,48 has supported training events for 2900 health and social care staff and family carers at a cost of > £500,000. 93 During the 36-month follow-up, we found that all but one of the clusters in the control arm had received such training since the end of recruitment in 2014. In the light of the main study findings, however, there is doubt as to the added benefit of such training.

It is also striking that, over the time of the study, the proportion of participants on psychotropic medications has remained stable. This may be attributable to medication reduction not being a specific target of the treatment or that issues around medication review/reduction may require another approach to address systemic beliefs about their benefits or otherwise. The identification of patient-reported outcome measures is another important consideration, as there may be discrepancies between what professionals and patients and their families think of as recovery-focused achievements.

Recommendations for future research

We believe that we have carried out a carefully conducted clinical trial, which has moved from small uncontrolled or feasibility studies to a fully powered pragmatic multicentre RCT in NHS community ID services. The behavioural aspects of the PBS framework have sufficient evidence of efficacy behind them and, potentially, an explanatory trial rather than a pragmatic one might have been a useful alternative design. A further concern is whether an implementation plan should have been considered at the outset to explore any issues that may arise in the intervention delivery. 97 Such blended designs may consider both the conduct of the clinical trial and issues in its implementation or may test both domains at the same time. We did collect implementation data but, because of the limitations, we were unable to react to the challenges in a flexible way, as reported in Chapter 2, Positive Behaviour Support-based staff training (in addition to treatment as usual). The trial raises important questions about (1) the content of staff training in PBS, (2) the optimisation of implementation in ordinary care and (3) the elements of a person’s life that it is most likely to have an impact on. It is crucial to understand the mechanisms by which PBS has an impact on behaviour as this would lead to targeted implementation and would clarify training objectives. As this is the first cost-effectiveness study of training in PBS, further studies about its delivery in different settings should include a health economic component to ensure that resources are prioritised and allocated in the most effective way.

Positive Behaviour Support is the only intervention with efficacy evidence, but not evidence of clinical effectiveness, in adults with ID and challenging behaviour. The 2015 NICE guideline38 on behaviour that challenges failed to find other candidate psychosocial interventions to ameliorate this long-term condition. Evidence-based care must be a top priority in the care of people with ID as it is for the care of other population groups. It is therefore imperative to develop new, or optimise existing, interventions for this population in order to improve care and quality of life in addition to making the best use of resources.

Trial collaborators

Contributions of authors

Angela Hassiotis (Professor in Learning Disability and Consultant Psychiatrist) was the chief investigator for the study, designed the study with input from the other authors, was involved in all aspects of management and implementation of the study and drafted and edited the final report.

Michaela Poppe (Trial Manager) co-ordinated the study as the trial manager and drafted the final report.

Andre Strydom (Consultant Psychiatrist and Reader in Intellectual Disabilities) was a principal investigator for the study, was involved in study design and management and provided key revisions and detailed feedback for the final report.

Victoria Vickerstaff (Medical Statistician) was a statistician for the study and conducted the statistical analysis, contributed to the funding application and protocol and drafted sections of the final report.

Ian Hall (Lead Clinician) was a principal investigator for the study and was involved in study design and management, advised on the methodology and analysis of qualitative interviews and provided key revisions and detailed feedback for the final report.

Jason Crabtree (Lead Consultant Psychologist) was a principal investigator for the study and was involved in study design and management, advised on the methodology and analysis of qualitative interviews and provided key revisions and detailed feedback for the final report.

Rumana Omar (Professor of Medical Statistics) was the senior statistician for the study and conducted the statistical analysis, contributed to the funding application and protocol and provided key revisions and detailed feedback for the final report.

Michael King (Professor of Primary Care Psychiatry and Joint Director of PRIMENT Clinical Trials Unit) was the triallist for the study and was involved in study design, management and analysis, and provided key revisions and detailed feedback for the final report.

Rachael Hunter (Senior Research Associate, Health Economics) was the health economist for the study and was involved in study design and management, conducted the health economic analysis and drafted sections of the final report.

Alessandro Bosco (Research Assistant) was a qualitative research assistant for the study and carried out the process evaluation (including conducting and analysing all qualitative interviews) and drafted sections of the final report.

Asit Biswas (Consultant Psychiatrist) was a principal investigator for the study and was involved in study design and management and provided key revisions and detailed feedback for the final report.

Victoria Ratti (Research Assistant) was a research assistant for the study and was responsible for participant recruitment, collected the study data and contributed to the protocol paper submission.

Jessica Blickwedel (Research Assistant) was a research assistant for the study and was responsible for participant recruitment, collected the study data and contributed to the protocol paper submission.

Vivien Cooper (Chief Executive of the Challenging Behaviour Foundation) is a family carer of a person with ID and a co-investigator for the study. She was involved in study design and management, advised on the methodology and analysis of qualitative interviews and provided key revisions and detailed feedback for the final report.

William Howie (Consultant Psychiatrist in Learning Disability) was a principal investigator for the study and was involved in study design and management and provided key revisions and detailed feedback for the final report.

Mike Crawford (Professor of Mental Health Research) was a principal investigator for the study and was involved in study design and management and provided key revisions and detailed feedback for the final report.

Publications

Ali A, Blickwedel J, Hassiotis A. Interventions for challenging behaviour in intellectual disability. Adv Psychiatr Treat 2014;20:184–92.

Ratti V, Hassiotis A, Crabtree J, Deb S, Gallagher P, Unwin G. The effectiveness of person-centred planning for people with intellectual disabilities: a systematic review. Res Dev Disabil 2016;57:63–84.

Ratti V, Vickerstaff V, Crabtree J, Hassiotis A. An exploratory factor analysis and construct validity of the Resident Choice Assessment Scale with paid carers of adults with intellectual disabilities and challenging behavior in community settings. J Ment Health Res Intellect Disabil 2017;10:198–216.

Hassiotis A, Poppe M, Strydom A, Vickerstaff V, Hall IS, Crabtree J, et al. Clinical outcomes of staff training in positive behaviour support to reduce challenging behaviour in adults with intellectual disability: cluster randomised controlled trial. Br J Psychiatry 2018;212:161–8.

Data sharing statement

Appropriately anonymised copies of the research databases will be made available in UCL Discovery (http://discovery.ucl.ac.uk) (https://doi.org/10.14324/000.ds.10041531). Please contact the corresponding author with queries.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care.