Cognitive behavioural therapy for clozapine-resistant schizophrenia: the FOCUS RCT

Article history

The research reported in this issue of the journal was funded by the HTA programme as project number 10/101/02. The contractual start date was in December 2012. The final report began editorial review in August 2017 and was accepted for publication in February 2018. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Anthony P Morrison reports personal fees from the provision of training workshops in cognitive–behavioural therapy (CBT) for psychosis and royalties from books on the topic, outside the submitted work. Andrew Gumley reports grants from the National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme (grant number 13/15/04) outside the submitted work. Douglas Turkington reports personal fees from Insight–CBT partnership (Insight Healthcare, Newcastle upon Tyne), outside the submitted work. Gemma Shields reports grants from NIHR during the conduct of the study. Graeme MacLennan reports grants from the NIHR HTA programme during the conduct of the study. Hamish J MacLeod reports that he occasionally provides CBT for psychosis workshops and receives fees for this work. John Norrie reports personal fees from the NIHR Editors Board and grants from NIHR HTA General Board Deputy Chairperson, outside the submitted work, and has membership of the HTA Funding Boards Policy Group and Pre-Exposure Prophylaxis Impact Review Panel. Linda Davies reports grants from the NIHR HTA programme during the conduct of the study. Paul French has membership of the HTA prioritisation Panel. Paul Hutton reports that he sits on an Expert Steering Group for Professor Jill Stavert’s Centre for Mental Health and Incapacity Law Rights and Policy at Edinburgh Napier University, and that he is a member of a committee developing National Institute for Health and Care Excellence (NICE) guidelines on supporting decision-making for people who may lack mental capacity. Robert Dudley reports receiving a NIHR Comprehensive Local Research Network Greenshoots award to fund time to support his contribution to the FOCUS (Focusing on Clozapine Unresponsive Symptoms) trial, royalties from Guilford Press and personal fees from Trinity College Dublin, outside the submitted work. Samantha Bowe reports personal fees from Pennine Care NHS Foundation Trust and personal fees from Cheshire & Wirral Partnership NHS Foundation Trust, outside the submitted work. Thomas RE Barnes reports personal fees from Sunovion (Marlborough, MA, USA) and Otsuka (Tokyo, Japan)/Lundbeck (Copenhagen, Denmark), outside the submitted work.

Copyright statement

© Queen’s Printer and Controller of HMSO 2019. This work was produced by Morrison et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2019 Queen’s Printer and Controller of HMSO

Psychosis and schizophrenia

The term psychosis is used to refer to a mental health problem that can involve changes to a person’s perceptions and/or to their thoughts. A person who has experience of psychosis may report perceptual changes such as hearing a voice that another person cannot hear, or seeing something that another person cannot see. Perceptual changes may also occur to a person’s taste, smell and/or bodily sensations. Such perceptual changes are typically referred to as hallucinations. A person experiencing psychosis may also report beliefs that others around them consider unusual, that are out of keeping with their social or cultural background and that are lacking rational grounds (often referred to as delusions). Common delusional beliefs include feelings of being persecuted, ideas of reference and feelings of importance. 1 Persecutory beliefs are thought to be the most commonly occurring of the range of delusional beliefs. 2 Experiences of hallucinations and delusions can be distressing and confusing and can have a negative impact on functioning. In addition, a person with experience of psychosis may report changes in their ability to concentrate or may communicate in a manner that is hard for other people to understand, which is commonly referred to as thought disorder. In both clinical practice and research, delusions, hallucinations and thought disorder are typically referred to as positive symptoms of psychosis. In addition, people who experience psychosis may report flat affect (blunted affect), a decrease in verbal output and verbal expressiveness, loss of motivation (including social withdrawal) and loss of enjoyment. 3 These experiences are often referred to as negative symptoms. This term was originally proposed because these changes refer to the loss or absence of usually present functions or characteristics. 4–6 Estimates suggest that 15–20% of people who receive a schizophrenia diagnosis will experience persistent negative symptoms. 7,8

Psychosis is considered to exist on a continuum,9 from the occasional occurrence of psychotic phenomena in the general population9–12 to persistent and frequent psychotic symptoms resulting in distress and, in many cases, the need for care from a mental health service. Someone who experiences psychosis and presents to mental health services for care may receive a schizophrenia spectrum diagnosis. Experiences of psychosis are often categorised using a diagnostic classification system, namely the International Classification of Diseases, Tenth Revision (ICD-10),13 or the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-V). 14 Schizophrenia spectrum diagnoses include schizophrenia, schizoaffective disorder, delusional disorder and schizophreniform disorder. However, there has been considerable debate over the use of such diagnostic terms and classification systems. 15 One argument is that these classification systems suggest that psychosis difficulties are dichotomous rather than continuum based. 9 In addition, concerns have been raised regarding the reliability of the diagnostic classification systems used. 16 The diagnostic label of schizophrenia has become stigmatising, with intrinsic negative associations about prognosis17 leading some countries to drop the term schizophrenia. 18 The use of diagnostic terminology has, therefore, been contentious and Murray,19 in 2017, highlighted the burgeoning evidence that schizophrenia is not a dichotomous condition, but rather the severe end of a continuum. Murray19 expects to see the end of the concept of schizophrenia in the future. However, diagnosis and diagnostic terminology is commonly utilised throughout both clinical practice and research. For the purpose of this report, we wish to acknowledge this debate and, throughout, we will adopt respectful terminology. When reviewing the literature relevant to the Focusing on Clozapine Unresponsive Symptoms (FOCUS) trial, we use the terms psychosis and schizophrenia, depending on the sample of participants recruited to the studies being referred to. In relation to the sample of participants included in the FOCUS trial, we refer to the population as people who meet the criteria for a schizophrenia spectrum diagnosis or people who meet the criteria for clozapine-resistant schizophrenia (CRS).

Prevalence rates and the personal, social and economic impact of schizophrenia

A first episode of psychosis typically occurs in young adults; a review of the literature on the age at onset of mental health disorders by Kessler et al. 20 found that, for those with a schizophrenia diagnosis, the age at onset is usually between 15 and 35 years of age. Reports of the lifetime prevalence in the literature vary from 0.12 to 1.6 per 100 persons. 21 In the UK, a systematic review22 commissioned by the Department of Health and Social Care reported that the incidence rate for all clinically relevant psychosis diagnoses is 31.72 per 100,000 persons per year, and for schizophrenia is 15 per 100,000 persons per year; the incidence of the latter being the same as the international incidence rate for schizophrenia reported by McGrath et al. 23

Although the prevalence of schizophrenia is relatively low, the personal, social and economic costs are considerable and the management of schizophrenia is among one of the largest health challenges globally as well as for the UK NHS. For example, Kirkbride et al. 22 report that, in 2009, the estimated economic costs for services and society attributable to broadly defined schizophrenia amounted to £8.8B. The majority of the costs were attributable to lost employment (47%) and service costs (40%). Using disability-adjusted life-years lost as a measure of the overall number of years lost as a result of ill health, disability or early death, schizophrenia was ranked eighth among mental health diagnoses and brain disorders in Europe in 2010. 24

Considerable health inequalities are reported for people who have a schizophrenia diagnosis, with increased risk of serious diseases, such as diabetes mellitus, cardiovascular disease (CVD), a human immunodeficiency virus (HIV) infection and respiratory problems. 25 The prevalence of CVD among those with a schizophrenia diagnosis is estimated to be 75%, compared with an estimated 50% in the general population. 26 Of great concern is the mortality risk associated with schizophrenia. A recent systematic review and meta-analysis27 of the literature on potential life lost and life expectancy in schizophrenia found that, on average, people with a diagnosis of schizophrenia die 14.5 years earlier than those in the general population. The same authors27 report that the average life expectancy for people with a schizophrenia diagnosis is 64.7 years (for men and women combined); among men only, the average life expectancy is 8 years shorter than in the general population. The estimated risk of suicide in the general population is 1%, in comparison with 10% in those with a schizophrenia diagnosis. 26 In addition to the health inequalities outlined above, people with a schizophrenia diagnosis face widespread public stigma and discrimination, representing further inequalities and a major challenge to recovery. The incidence of anticipated, experienced and internalised stigma is high among people with psychosis. 28,29 A large-scale survey of people with schizophrenia diagnoses across 27 different countries found that nearly half of the respondents felt at a disadvantage as a result of having a diagnosis of schizophrenia, because of stigma. 29 Nearly half of the sample reported interpersonal/social difficulties with making or keeping friends and around one-quarter identified feeling at a disadvantage in terms of finding and keeping work and in relation to their personal safety. 29

Given the significant personal, societal and economic costs associated with schizophrenia, the treatment and care provided for people who meet the criteria for schizophrenia spectrum diagnoses should be a priority for policy-makers, commissioners and service providers.

Treatment options for people with a schizophrenia diagnosis

In this section of the report, we will consider the evidence base for treatments for people with a schizophrenia diagnosis considering both pharmacological and psychological interventions. We will begin the section with a consideration of the efficacy of antipsychotic medication, with a particular emphasis on treatment options for people with a schizophrenia diagnosis who have a poor response to antipsychotic medication. We will outline the psychological interventions recommended by the National Institute for Health and Care Excellence (NICE), with a specific emphasis on cognitive–behavioural therapy (CBT), given that this is the intervention evaluated by the FOCUS trial. We will consider the efficacy of CBT in conjunction with antipsychotic medication with a particular emphasis on efficacy for CRS.

Summary

To summarise, there is clear evidence from the CBT trials that people with TRS and CRS can be engaged in CBT, and that CBT can have small to moderate effects on overall symptoms and may be particularly beneficial for auditory hallucinations. However, it has been highlighted in a 2014 meta-analysis69 that the large and methodologically robust trials of CBT for psychosis have not demonstrated a significant advantage of CBT for either symptoms or relapse, and to date there have been no large high-quality trials of CBT for people with CRS. Moreover, CBT trials have been criticised for poor reporting of adverse effects,124 and future trials should report adverse effects as an outcome. Klingberg et al. 125,126 have provided a useful template for assessing adverse effects that includes death caused by suicide, suicide attempt, suicidal crisis [as defined in the Calgary Depression Rating Scale for Schizophrenia (CDSS), item 8, rating 2] and severe symptomatic exacerbation, defined by the Clinical Global Impression (CGI) scale, which includes ratings of illness severity, changes in overall clinical status, and therapeutic effects. In addition, further research is needed to identify factors that predict a good outcome from CBT.

Rationale for the research/trial aims and objectives

The objectives of this RCT were to provide evidence of the clinical effectiveness and cost-effectiveness of CBT for people with CRS and to utilise baseline data from the RCT to develop a risk model that identifies factors that predict good outcome from CBT. Using the patient-level data available from the trial, the objectives for the economic evaluation were to:

• estimate the costs of health and social care in the intervention and TAU groups, and assess whether or not there were differences between the groups

• estimate the quality-adjusted life-years (QALYs) of participants in the intervention and TAU groups, and assess whether or not there were differences between groups

• assess whether or not any additional benefit is worth any additional cost.

The research objectives of this RCT were to test the following hypotheses:

• In people with a diagnosis of a schizophrenia spectrum disorder who have an inadequate response to or are unable to tolerate clozapine, CBT plus TAU will lead to improvement in psychotic symptoms, measured using a psychiatric interview (PANSS) over a 21-month follow-up period compared with TAU alone.

• Cognitive–behavioural therapy plus TAU will lead to improved quality of life and user-defined recovery compared with TAU alone.

• Cognitive–behavioural therapy plus TAU will lead to a reduction in affective symptoms and negative symptoms compared with TAU alone.

• Cognitive–behavioural therapy plus TAU will be cost-effective compared with TAU alone.

Trial design

The FOCUS trial was a parallel-group, randomised, outcome-blinded evaluation (PROBE) to evaluate the addition of a standardised CBT intervention to TAU for individuals who are unable to tolerate or have had an inadequate response to clozapine. The comparison group received TAU only. The trial was intended to be a definitive, pragmatic clinical effectiveness and cost-effectiveness trial. It was conducted over 4 years across five sites within the UK, with recruitment commencing on 1 January 2013 and ending on 1 June 2015. The follow-up phase for the trial ended in February 2017. A copy of the full ethics-approved trial protocol can be found on the project web page: www.journalslibrary.nihr.ac.uk/programmes/hta/1010102#/. In addition, the study protocol has been published in a peer-reviewed journal. 127

Role of funding source

The FOCUS trial was funded as a result of a National Institute for Health Research (NIHR) Health Technology Assessment-commissioned call. The call specified the design in PICO (population, intervention, comparator and outcome) terms, requiring that the population be patients with schizophrenia who had not responded to an adequate dose of clozapine or were unable to tolerate it, the intervention was CBT, the comparator was TAU and the outcome was psychiatric symptoms.

Approval

The National Research Ethics Service (NRES) Committee North West – Lancaster (reference 12/NW/0520) approved the FOCUS trial. Ethics approval was granted on 13 August 2012. The trial was also registered on the International Standard Randomised Controlled Trial Number (ISRCTN) clinical trial registry (reference ISRCTN99672552). The trial was registered on 29 November 2012, before recruitment was started in January 2013.

Trial sites

The study was conducted in secondary care mental health services (community mental health, residential rehabilitation and inpatient settings) at five UK centres. These were (1) Manchester, (2) Edinburgh, (3) Glasgow, (4) Newcastle upon Tyne and (5) Southampton.

Participants

A total of 487 participants were recruited across the five sites between 1 January 2013 and 1 June 2015. The Manchester site recruited 108 of the total participants, Southampton recruited 105, Edinburgh recruited 94, Newcastle upon Tyne recruited 92 and Glasgow recruited 88. Participants were recruited from a range of services and settings including community mental health teams (CMHTs), early intervention teams, recovery teams and inpatient services.

Data collection

In accordance with the approved protocol, potential participants were initially informed about the study by a member of their care team and, if they expressed interest, were asked to consent to being contacted by the FOCUS trial research team. If they did so, a member of the research team briefly described the study and sent the participant information sheet (PIS) by post. The individual was then given a minimum of 24 hours to consider the information. Following this, the RA arranged to meet the participant at a place of their choosing; in the majority of cases this was the participant’s own home. Some preferred to meet within a mental health service or, if there were any possible risk issues, a meeting at a NHS site would be arranged. Participants who were current inpatients were visited on the ward. The RA talked through the PIS with the individual and ensured that the information was understood by asking the participant to reflect it back to them. When both the RA and the participant were satisfied that all the information about the trial had been provided and understood, the participant was asked to sign the consent form. The RA then read through each point and the participant initialled the boxes provided if they agreed to the information. Both the RA and the participant signed their names underneath.

The RA would then commence the baseline assessment. In the majority of cases, this was conducted across two visits, but this was at the participant’s preference. On average, to complete all assessment measures in full would take approximately 2 hours. The assessment would begin with the PANSS interview and then move on to the self-report measures (outlined below). Each participant was also provided with a personalised crisis card at the baseline assessment. This included contact details for their care team and general practitioner (GP) as well as other helpline numbers, such as the Samaritans. Finally, participants were compensated with £10 for their time and contribution to the research process.

Face-to-face follow-up assessments were completed at 9 and 21 months. The participant was contacted by telephone and an appointment was arranged. Ongoing consent was confirmed with participants at each follow-up. The RA conducted a PANSS interview and asked the participant to complete the self-report measures at each of these time points. Participants were compensated £10 for their time and contribution to the research process at these time points.

Follow-up assessments were completed by telephone at 3, 6, 13 and 17 months. These telephone assessments focused only on obtaining health economics data – no clinical outcome data were collected. The participants were sent £5 gift vouchers in the post on completion of these follow-ups. ‘Keeping in touch’ cards were also posted to the participant on two occasions between these telephone calls.

Outcome measures

Economic assessment

Trial interventions

The intervention to be assessed was CBT. This was delivered by appropriately qualified psychological therapists over a 9-month period. Therapists were employed at NHS band 7, which is the starting point for most psychological and nurse therapists and is representative of CBT delivered within NHS services. Participants were offered up to 30 hours of CBT on an approximately weekly basis during the treatment window. The sessions were provided on an individual basis. Most therapy appointments were delivered in the participants’ own homes to help increase the acceptability and accessibility of the intervention.

The CBT was based on a specific cognitive model63 and was manualised. 104 For a more detailed outline of the intervention, see Pyle et al. 127 The specific CBT interventions used were dependent on the individual formulation, but had to be consistent with the intervention model. The range of permissible interventions was provided in published treatment manuals. 64,153 The therapy was flexible, but aimed to involve four phases: (1) assessment, engagement and formulation, (2) change strategies, (3) longitudinal aspects and (4) consolidation. Key milestones were included in each phase. The aims of CBT were to reduce distress (particularly that associated with psychotic symptoms) and to improve quality of life, often by changing the impact of psychotic experiences and beliefs. The CBT was delivered in a collaborative relationship between the participant and therapist and addressed the problems and goals that were agreed between the participant and therapist. Therefore, target of treatment could include positive symptoms of psychosis, social relational issues and issues of comorbidity including anxiety and depression. Key therapeutic principles included formulation, normalisation, collaboration and evaluation of the client’s appraisals of and responses to psychological phenomena.

There was an emphasis on encouraging participants to undertake between-session practice, as research evaluating components of CBT as mechanisms for change suggests that CBT is more effective if between-session tasks are used. 154

By the third session it was expected that there would be a shared list of problems and goals, and shared formulation. It was expected that, by session 12, there would be a shared longitudinal formulation. Milestones regarding formulation were important given the evidence to suggest that formulation is a core component of CBT. 154

Reliability of outcome measures

The reliability of all outcome measures (total scores and subscales) at baseline were assessed using Cronbach’s alpha reliability statistic. The reliability statistics can be seen in Table 3. The reliability of all measures was acceptable (as indicated by α ≥ 0.7) except for PANSS positive, PANSS excitement, PANSS emotional distress and ISMI scale stigma resistance.

Participant baseline characteristics

Trial recruitment

In total, 487 participants were recruited from five centres (Table 4): 242 were randomised to CBT and 245 were randomised to TAU. The referral pathway by the different service type for the participants randomised overall and by centre is shown in Appendix 1. The highest recruiter across all centres was the CMHTs, followed by the clozapine clinic. Participants were recruited to the trial between 1 January 2013 and 31 May 2015 and followed up to March 2017. The trajectory of recruitment from all centres is shown in Figure 1.

TABLE 4 - Recruitment by centre

Centre	Participants, n (%)
	Eligible (N = 565)	Ineligible (N = 47)	Declined (N = 31)	Randomised (N = 487)	Randomised to CBT (N = 242)	Randomised to TAU (N = 245)
Manchester	129 (22.8)	11 (23.4)	10 (32.3)	108 (22.2)	54 (22.3)	54 (22.0)
Southampton	121 (21.4)	10 (21.3)	6 (19.4)	105 (21.6)	52 (21.5)	53 (21.6)
Newcastle	109 (19.3)	8 (17.0)	9 (29.0)	92 (18.9)	46 (19.0)	46 (18.8)
Edinburgh	100 (17.7)	6 (12.8)	2 (6.5)	92 (18.9)	46 (19.0)	46 (18.8)
Glasgow	106 (18.8)	12 (25.5)	4 (12.9)	90 (18.5)	44 (18.2)	46 (18.8)

FIGURE 1.

Recruitment over time.

Participant flow

Figure 2 shows the Consolidated Standards of Reporting Trials (CONSORT) flow diagram for the trial. Of the patients, 898 were identified as potentially eligible for inclusion in the trial and were referred, and of these 565 were found to be eligible at the referral stage. Of the 78 patients excluded from the trial, 47 were found to be ineligible and 31 declined. Details of the reasons for patients being ineligible at the referral stage and before randomisation are shown in Appendix 1. All randomised participants completed the baseline questionnaires. In the CBT arm, 230 participants received treatment; further detail on the number of sessions attended is in Chapter 5. At 21 months, 425 participants (87.2%) were included in the primary analysis: there were 10 deaths, 36 participants withdrew from the trial and declined to provide outcome data and a further six were not assessed as they were not contactable at follow-up.

FIGURE 2.

The CONSORT flow diagram. Reprinted from Lancet Psychiatry, Volume 5, Morrison et al. , Cognitive behavioural therapy in clozapine-resistant schizophrenia (FOCUS): an assessor-blinded, randomised controlled trial, Pages 633–43, © 2018, with permission from Elsevier. 171

Reprinted from Lancet Psychiatry, Volume 5, Morrison et al., Cognitive behavioural therapy in clozapine-resistant schizophrenia (FOCUS): an assessor-blinded, randomised controlled trial, Pages 633–43, © 2018, with permission from Elsevier171

Primary outcome

The PANSS total and subscale scores at each time point are described in Table 7. Figure 3 shows the profile for the two treatment groups over the study period. Treatment effect estimates are also included in Table 7, based on 425 participants for whom at least one follow-up measurement was available. At 9 months, the total PANSS score was lower in the CBT arm (mean –2.40, 95% CI –4.79 to –0.02; p = 0.049), with a standardised ES of 0.16. The mean difference at 21 months was –0.89 (95% CI –3.32 to 1.55; p = 0.475), with a standardised ES of 0.06. At 9 months, the subscale PANSS positive score was lower in the CBT arm (mean –1.56, 95% CI –2.53 to –0.59; p = 0.002), with a standardised ES of 0.24; PANSS excitement was lower in the CBT arm (mean –1.18, 95% CI –1.85 to 0.50; p = 0.001), with a standardised ES of 0.28, and PANSS emotional distress was lower in the CBT arm (mean –1.08, 95% CI –2.02 to –0.13; p = 0.025), with a standardised ES of 0.17. For PANSS negative and PANSS disorganised, there was no evidence of a difference at either time point. Sensitivity analysis using MI gave similar results (see Appendix 2). Figure 4 shows the site difference and within-site differences for total PANSS score at 9 and 21 months. At 21 months, the Manchester site recorded a greater effect in total PANSS score in favour of CBT.

FIGURE 3.

Profile of PANSS total scores.

FIGURE 4.

Site differences for PANSS total scores. (a) 9 months; and (b) 21 months.

At the 9-month assessment, a total of 51 blind breaks had occurred. However, 23 of these cases were transferred to a new, independent assessor, meaning that 28 assessments were unblind at the 9-month assessment. At the 21-month assessment, the number of breaks that had occurred was 55. However, 35 of these cases were transferred to a new, independent assessor, meaning that 20 assessments were unblind at the 21-month assessment. Outcome analyses for PANSS were repeated excluding those participants for whom a blind break had occurred; the results are presented in Table 8.

The percentage of improvement in PANSS total is shown in Table 9. At 9 months, 16 participants (6.6%) in the CBT arm and 11 (4.5%) in the TAU arm had > 50% improvement; the number needed to treat (NNT) was 42. At 21 months, 28 participants (11.6%) in the CBT arm and 14 (5.7%) in the TAU arm had > 50% improvement, and the NNT was 15.

Table 10 shows the effect of time on PANSS total for the sample as a whole. The analysis was adjusted for randomised treatment, age, sex and centre. Table 10 shows a reduction in PANSS total score of 5.26 at 9 months compared with baseline and a reduction in PANSS total score of 9.1 at 21 months compared with baseline.

Secondary outcomes

The PSYRATS subscales at 9 and 21 months are shown in Table 11. At 9 months there was a mean difference in favour of CBT for auditory hallucinations (–2.56, 95% CI –4.87 to –0.26; p = 0.029) and physical voices (–0.58, 95% CI –1.11 to –0.04; p = 0.034). Both subscales were similar in both groups at 21 months. At 21 months the mean difference between groups on emotional unusual beliefs was –0.53 (95% CI –1.05 to –0.00; p = 0.049). For the remainder of the PSYRATS subscales, there was no evidence of a difference between the two treatments at 9 or 21 months. Other secondary outcomes were similar with the exceptions of QPR and CGI (Table 12).

As indicated in Table 13, chi-squared analysis did not indicate any significant difference between the groups regarding access to education, employment or training at either 9 or 21 months.

Compliance with treatment

The median number of CBT sessions attended was 23, and 213 out of 242 participants (88%) attended at least six sessions, which was the minimum number of sessions needed to be classified as having received CBT (Table 14). Treatment effects from a CACE analysis, adjusting for compliance at 9 months and the primary and secondary outcomes, are shown in Appendix 2. We repeated this analysis using the actual number of sessions attended; results are presented in Table 15 for 9 months and Table 16 for 21 months. For every extra session attended, the PANSS total score resulted in a difference of –0.12 (95% CI –0.24 to 0.00; p = 0.059) at 9 months, which suggests that attending more CBT sessions was beneficial in the short term.

Session record data were analysed to determine if therapy milestones were achieved. The percentage of participants allocated to CBT with whom therapy milestones were achieved is shown in Table 17.

Fidelity to the CBT model was evaluated using 57 audio-recordings of therapy sessions. Table 18 provides descriptive statistics for the total fidelity ratings.

Subgroup analyses

Results from the subgroup analysis are reported in Figures 5 and 6 for PANSS total. Results from the subgroup analysis for the secondary outcomes QPR, PSP and PSYRATS – auditory hallucinations are in Appendix 2. There was no evidence that the treatment effect was moderated by any of the specified subgroups. Sensitivity analysis for the subgroup LNS including participants who refused at least one question is shown in Appendix 2. Overall, there was no evidence of a statistical difference for any of the subgroups on PANSS total at either time point.

FIGURE 5.

Comparison of CBT and TAU by subgroups. (a) Age, age of onset and sex at 9 months; (b) age, age of onset and sex at 21 months; (c) DI (years), DUP (years), antipsychotic and clozapine daily dose at 9 months; (d) DI (years), DUP (years), antipsychotic and clozapine daily dose at 21 months; (e) difficulty with abstract thinking and conceptual disorganisation at 9 months; (f) difficulty with abstract thinking and conceptual disorganisation at 21 months; (g) childhood trauma at 9 months; (h) childhood trauma at 21 months; (i) AUDIT, DAST and PAM-SR attachment avoidance at 9 months; (j) AUDIT, DAST and PAM-SR attachment avoidance at 21 months; (k) BCSS at 9 months; and (l) BCSS at 21 months.

FIGURE 6.

Comparison of CBT and TAU by psychosis subgroup. (a) 9 months; and (b) 21 months.

For subgroup analyses of QPR (see Appendix 2), there was evidence of a significant interaction effect at 9 months for the BCSS subscale ‘negative others’, which scored < 7.2, with those subscales that scored ≥ 7.2 (interaction effect –4.96, 95% CI –8.94 to –0.98; p = 0.015). All other subgroups showed no evidence of a difference. Appendix 2 shows the subgroup analysis for PSP. There was evidence of a significant interaction effect for participants with a DI of ≥ 31 years with participants with a DI of 0–15 years at 9 months (interaction effect –9.03, 95% CI –16.62 to –1.98; p = 0.013). For the CTQ physical abuse subscale, there was evidence of a difference at 9 months for participants scoring ≥ 8 with those scoring ≤ 7 (interaction effect 6.29, 95% CI 0.80 to 11.78; p = 0.025).

In the subgroup analyses for PSYRATS – auditory hallucinations (see Appendix 2), there was evidence of a difference for participants with a DI of between 16 and 30 years and those with a DI of between 0 and 15 years at 9 months (interaction effect –7.33, 95% CI –12.41 to –2.25; p = 0.005). For AUDIT, these was evidence of a difference for participants with a score of ≤ 12 and for those with a score of ≤ 12 (interaction) (interaction effect –9.76, 95% CI –18.40 to –1.12; p = 0.027). Furthermore, there was evidence of a difference for PAM-SR attachment avoidance at 9 months for those who scored 17–24 compared with those who scored 1–9 in favour of TAU (interaction effect 8.56, 95% CI 0.73 to 16.30; p = 0.032).

Adverse events and potential unwanted side effects of trial participation

In total, three participants experienced an AE that was deemed to be related to the trial or unclear if related; these AEs were reported to the National Research Ethics Committee (Table 19). Two of the affected participants were in the CBT arm (unclear if related or not) and one was in the TAU arm (related); one AE was categorised as life-threatening and resulted in self-harm, one was an involuntary hospitalisation to a psychiatric hospital and one was self-harm that required treatment at an accident and emergency department.

There were 107 participants (44.2%) in the CBT arm and 104 (42.4%) in the TAU arm who reported at least one AE or adverse effect; there was no significant difference between the groups in the number of people with at least one AE or adverse effect (p = 0.58) (see Table 19). The main reasons were voluntary hospitalisation to a psychiatric hospital, self-harm, CGI severity of > 6 and > 25% deterioration on the PANSS. In the CBT group, 22 of the 27 reported self-harm events were from the same participant. There were six deaths in the CBT arm and four in the TAU arm; all were deemed to be unrelated to the study.

At 9 months, 22 participants (9%) in the CBT arm and 28 (11%) in the TAU arm had a deterioration in PANSS total score of > 25%. The number of deteriorations at > 50% and > 75% was similar in both groups at 9 and 21 months.

The frequency of participants responding ‘quite a lot’ or ‘very much’ to each of the items on the bespoke adverse effects measure developed for the FOCUS trial can be found in Appendix 2. There were no significant differences between the groups for any of the 27 items in the adverse effects measure.

Internalised stigma of mental illness

Levels of ISMI subscales for the whole sample at baseline are reported in Table 20. For the ISMI alienation scale, 201 participants (41%) had a severe level, and 225 participants (46%) had moderate levels of stigma resistance.

Figure 7 shows the profile for the ISMI subscales for the two treatment groups over the study period and Table 21 shows the treatment effects. At 9 months, ISMI discrimination experience was lower in the CBT arm than the TAU arm (–0.13, 95% CI –0.25 to –0.01; p = 0.029). For the subgroup analysis on the PANSS total, only stigma resistance at 9 months showed evidence of a significant interaction effect (Figure 8).

FIGURE 7.

Profile of ISMI subscales. (a) ISMI alienation; (b) ISMI sterotype endorsement; (c) ISMI discrimination experience; (d) ISMI social withdrawal; (e) ISMI stigma resistance; and (f) ISMI total.

FIGURE 8.

The PANSS total subgroups. (a) Alienation, stereotype endorsement and discrimination experience at 9 months; (b) alienation, stereotype endorsement and discrimination experience at 21 months; (c) social withdrawal and stigma resistance at 9 months; and (d) social withdrawal and stigma resistance at 21 months.

Chapter overview

This chapter describes the results of the economic evaluation conducted as part of the FOCUS trial of CBT compared with TAU for people who cannot tolerate or have had an inadequate response to clozapine. The economic evaluation uses service use and health status data collected in the RCT to compare the costs and QALYs of CBT with those of TAU, and estimate the cost per QALY gained. The analysis used the viewpoint of NHS health and social care service providers (costs) and patients (health benefits) for the 21-month follow-up period of the trial. Full methods are provided in Chapter 2.

Missing cost and utility data

Table 22 summarises the number of participants with complete cost or utility data at baseline and at 9- and 21-month follow-up. Overall, complete cost and QALY data at 9-month follow-up were available for 126 out of 245 participants (51%) in the TAU arm, compared with 114 out of 242 participants (47%) in the CBT group. At 21-month follow-up, complete cost and QALY data were available for 93 out of 245 participants (38%) in the TAU arm, compared with 76 out of 242 participants (31%) in the CBT group.

Baseline clinical and demographic characteristics

Sociodemographic characteristics for the participants were reported in Table 5. The data suggested that the two groups were similar at baseline.

Table 23 reports key baseline demographic and clinical characteristics that are statistically significantly associated with either baseline utility or baseline cost data (Kendall’s τ-coefficient).

Stepwise linear regression (SPSS version 22), using all the characteristics in Table 23, was used to identify the key characteristics to include in the MI of missing data and as covariates for the primary and sensitivity analyses.

Table 24 shows the results of the regression analyses for the characteristics that were statistically associated with utility (p ≤ 0.10) and included as covariates in the analyses. QALYs are estimated from baseline and follow-up utilities. The EuroQol Visual Analogue Scale (EQ VAS) score was used to adjust for any differences in participants’ reported health status at baseline. The EQ VAS is included in the EQ-5D instrument as an alternative measure of self-reported health. None of the baseline characteristics identified in Table 24 was associated with baseline cost. The characteristics listed in Table 24 that were statistically significantly associated with cost were included in the MI of missing data and as covariates for the analyses. The baseline cost categories were also included in the MI models, whereas total baseline cost was included as a covariate in the regression models to estimate net costs. 159,160

Descriptive analysis of health status, utility, quality-adjusted life-years and cost for participants with complete cost and quality-adjusted life-year data

Descriptive analysis of utility, quality-adjusted life-years and cost for all participants, using multiple imputation data

Table 32 reports the utility values and QALYs for all participants, using the MI data. As with the complete-case analysis, utility increases from baseline to end of scheduled follow-up in both groups. The data in Table 32 indicate that the number of QALYs is similar in each group at 9 months. There appears to be a trend towards a difference in total QALYs between the two groups at 21 months. However, these data are not adjusted for any differences in the characteristics or utility of participants at baseline. Although the QALY estimates include baseline utility, this may not fully capture the impact of differences in baseline utility on utility at follow-up.

Primary analysis of incremental costs, quality-adjusted life-years and incremental cost-effectiveness ratio

Table 34 shows the net costs and QALYs of CBT compared with TAU at 21 months of follow-up for the primary and sensitivity analyses. Appendix 3 reports the full results of the regression analyses used to adjust for baseline covariates. A linear regression model was used to estimate net QALYs and alternative measures of health benefit and a generalised linear model with gamma log distribution was used for the costs, to account for the skewed distribution of costs towards zero. The data from each of the primary and sensitivity analyses were bootstrapped to generate 10,000 pairs of net cost and net QALY estimates.

The primary analysis indicates that CBT is associated with a net gain in QALYs and a net cost. The net QALY gain is statistically significant. In contrast, the net cost is associated with a high level of variability and is not statistically significant. The additional net cost associated with CBT is attributable to the costs of delivering the intervention and the higher costs of care overall. Excluding the cost of CBT, the net cost of care for the intervention was £2123 (SE £4595, 95% CI –£6883 to £11,129; p = 0.644); however, the difference in cost between CBT and TAU was not statistically significantly.

It is important to consider the joint uncertainty associated with the pairs of net cost and QALY estimates, because these are combined to estimate the ICER, which is the overall outcome measure for the economic evaluation. This uncertainty is illustrated in Figure 9 with a scatterplot of the 10,000 pairs of net cost and QALYs, from the bootstrapped data, in the form of a cost-effectiveness plane. The cost-effectiveness plane demonstrates that the majority of pairs of net cost and QALYs are to the right-hand side of the horizontal axis, indicating that the CBT intervention is associated with a net gain in QALYs and health benefit. However, the majority of pairs also lie in the top half of the vertical axis, indicating a net cost to CBT.

FIGURE 9.

Cost-effectiveness plane of 10,000 pairs of net costs and QALYs: bootstrapped MI data.

Table 35 reports the ICER, the likelihood that CBT is cost-effective at a WTPT of £15,000 and the estimated net monetary benefit statistic of CBT, when compared with TAU. There is no universally agreed monetary value to attach to QALYs. Therefore, the simulated net QALYs were revalued using a range of values that a decision-maker may be willing to pay to gain 1 QALY, ranging from £0 to £30,000. This was based on the range of willingness-to-pay values historically used in NICE decisions. 167,172 This approach takes into account uncertainty about the amount that decision-makers would be willing to pay to gain 1 additional QALY from the CBT intervention. The impact of different WTPT values on the likelihood that CBT is cost-effective is shown in the cost-effectiveness acceptability curve in Figure 10.

FIGURE 10.

Cost-effectiveness acceptability curve: primary analysis.

The primary analysis indicates that, although CBT is associated with higher health benefits than TAU in terms of QALYs, the additional costs mean that it is unlikely to be cost-effective if decision-makers are willing to pay £15,000 to gain 1 QALY. The probability that CBT is cost-effective (in this analysis of 10,000 net cost and QALY pair estimates) is 0.13, or 13% (see Table 35). If decision-makers are willing to pay £30,000 to gain 1 QALY, the chance that CBT is cost-effective is < 50%, at 17% (see Figure 10).

Sensitivity analyses of incremental costs, quality-adjusted life-years and incremental cost-effectiveness ratio

The cost-effectiveness acceptability curves in Figures 11 and 12 show the sensitivity analyses for the complete-case analysis, 9-month time horizon, QALYs estimated from crosswalk utility values (see Figure 11) and the alternative health benefit measures (see Figure 12). These indicate that CBT has a likelihood of being cost-effective of < 50%. This is the case for a WTPT of up to £30,000 per QALY gained (or, for the alternative health benefit measures, a WTPT of up to £30,000 per person with a clinically relevant improvement gained). The exception is the analysis using ≥ 25% improvement in QPR. In this case, CBT had a 54% chance of being cost-effective.

FIGURE 11.

Cost-effectiveness acceptability curve: sensitivity analysis 1.

FIGURE 12.

Cost-effectiveness acceptability curve: sensitivity analysis 2.

The complete-case analysis suggests that there is a 40% chance that CBT is cost-effective if decision-makers are willing to pay up to £15,000 to gain 1 QALY. This rises to an 89% chance of being cost-effective if decision-makers are willing to pay up to £30,000 to gain 1 QALY. However, the complete-case analysis consists of the subset of participants with complete data on costs and QALYs over the 21-month follow-up period. These participants did not use inpatient psychiatric hospital care over the 21-month follow-up period.

Summary

The FOCUS trial is, to our knowledge, the first definitive RCT to evaluate the long-term clinical effectiveness and cost-effectiveness of CBT in comparison with TAU for people who meet the criteria for CRS. In addition, we utilised baseline data to develop a risk model of factors that may predict a good outcome from CBT. The FOCUS trial was methodologically robust with a low risk of bias.

No effect on the primary outcome of PANSS total score was found at the 21-month assessment and the CIs around estimated treatment effect rule out the hypothesised ES. Many of the secondary outcomes were not found to be significant; however, there was a significant effect of CBT over TAU for the main secondary outcome of PANSS total at the end of treatment (9-month assessment) and an encouraging NNT for achieving a good outcome on the PANSS total score at 21 months. The results also indicated small but significant effects of CBT over TAU for self-rated recovery, emotional distress associated with delusions and CGI improvement at long-term follow-up, and at the end of treatment on PANSS total score, positive symptoms, emotional distress and auditory hallucinations. With 88% of those who were allocated to CBT having six sessions or more, it is clear that CBT is acceptable to the majority of those who are offered it. We did not find a difference between CBT and TAU in reportable SAEs or in the number of participants who had one or more AE. The number of reportable SAEs and the absence of any significant difference between the groups on non-reportable AEs suggest that CBT is a safe treatment for this population.

The results of the risk modelling did not reveal any factors that predict a good response to CBT. There was no evidence that the treatment effects of CBT varied over any of the subgroups investigated in the FOCUS trial population for PANSS total score or the PANSS subscales.

Overall, CBT was associated with a net cost and net QALY gain compared with TAU. The additional cost for the CBT group was the result of higher use of health and social care services by participant as well as the additional cost of the CBT intervention. There was a high level of variance in the costs of different services and total costs, and the differences in cost between the two groups were not statistically significant. This indicates uncertainty about whether the CBT group incurred higher costs than the TAU group or whether the difference found was attributable to chance. Nevertheless, the cost-effectiveness acceptability analysis indicated a low likelihood that CBT was cost-effective, in the primary and sensitivity analyses (probability of < 50%). There are a number of limitations that increase the uncertainty of the results, which are discussed in Strengths and limitations.

Findings in context

The results of the FOCUS trial showed no lasting effect of CBT for people who meet the criteria for CRS on total symptoms as measured by PANSS. However, there was a statistically significant impact on overall health for those who received CBT in comparison with those who received TAU. Clinically significant improvement on PANSS total score has been defined in meta-analyses of antipsychotic medication as ≥ 50% improvement in the PANSS total score (rescaled) from baseline;32 our NNT for a good improvement in PANSS total score was 15. A recent meta-analysis of all double-blind, placebo-controlled RCTs of antipsychotic medications, except clozapine, found a NNT of eight for ≥ 50% improvement on PANSS. 32 More specifically, for clozapine the NNT has been reported as eight. 47 However, the participants in the FOCUS trial had experienced a poor response to treatment with both standard antipsychotic medication and clozapine and, therefore, our NNT of 15 is encouraging, particularly as this was at long-term follow-up.

In addition, the effect of CBT at end of treatment on PANSS total, PANSS subscales (positive symptoms, emotional distress and excitement) and auditory hallucinations demonstrates that CBT can change symptoms of psychosis in the short term. The reasons for the short-term effect of CBT are unclear; however, given the long DI experienced by our sample, it is possible that a greater number of CBT sessions or longer duration of the treatment window may be required to yield a long-term benefit. It could also be argued that the effects seen at the end of treatment are a result of the non-specific aspects of therapy, such as the therapeutic relationship, rather than the active components of CBT. Therapeutic alliance has been shown to contribute to both improvement and deterioration in PANSS scores for CBT and other psychological therapies, such as supportive counselling. 173 However, not all FOCUS trial participants completed CBT over the full 9-month treatment window and, therefore, it is not possible to say with any confidence that the effects of CBT seen at the end of treatment were indicative that it works only while treatment is ongoing.

The ES of CBT in this trial is similar to those reported by meta-analyses of CBT for psychosis trials that are deemed to be methodically robust. 67,69 The authors of the most current meta-analysis have proposed that the field of CBT for psychosis research is lacking evidence from a large and methodologically robust trial. 69 Prior to the FOCUS trial, there had been no definitive trials of CBT for CRS. The FOCUS trial has clearly addressed this need for the CRS population and also addressed methodological limitations of previous trials. Jauhar et al. 69 reported an ES of 0.15 from trials at a low risk of masking bias and 0.62 from trials at a high risk of bias. For the FOCUS trial, the risk of this particular bias is low, with only 6.17% of assessments at 9 months (end of treatment) being conducted by an unmasked assessor and our ES at the end of treatment being 0.16, consistent with the pooled estimate from low-risk trials reported by Jauhar et al. 69

With regard to other outcomes, meta-analyses have shown very small ESs for trials with a low risk of bias from masking for positive symptoms. 67,69 Jauhar et al. 69 found, at the end of treatment, an ES of 0.08 for positive symptoms and concluded, therefore, that it is not justified to propose that CBT is effective for positive symptoms. However, the end-of-treatment results for the FOCUS trial indicate an ES of 0.24 and, although this is a small effect, this finding comes from a methodologically robust trial and is encouraging as it indicates that for positive symptoms, CBT can have small short-term benefits for people who meet criteria for CRS, while in receipt of treatment. This finding is particularly encouraging given that our participants have experienced positive symptoms with a poor response to antipsychotic medication and their DI, on average, was long. This finding also demonstrates, in line with psychological models of psychosis, that hallucinations and delusions can change in response to consideration of the person’s interpretation of these experiences and their associated behavioural responses.

Cognitive–behavioural models of psychosis and schizophrenia recognise the link between cognitions, emotions and behaviours; the CBT delivered in this trial permitted work on emotional distress and dysfunction, including prioritising depression and anxiety (if the participant prioritised this as part of their problem list or goals for therapy). Given the role of emotion in psychosis, it has been argued that CBT for psychosis trials should include emotional distress as a secondary outcome121 and the results of the FOCUS trial indicate that CBT can have small but significant effects at the end of treatment for this important secondary outcome.

There were also some small long-term treatment effects at the 21-month assessment, including for self-rated recovery and an encouraging NNT of 15 to achieve a good response (> 50% improvement on PANSS at 21 months). The primary health economics analysis also suggested that CBT is associated with higher health benefits than TAU in terms of QALYs. Service users advocate that outcomes used in research and clinical services should refocus to ones of personal recovery goals, rather than the reduction or absence of symptoms, which has traditionally been the key outcome advocated by clinicians. 118 UK policy places an emphasis on recovery-orientated outcomes and services for people who experience psychosis and schizophrenia30 and, for this reason, our finding that CBT can have a lasting effect on self-rated recovery is arguably more important for service users and services.

The primary health economics analysis uses the EQ-5D-5L and new value set rather than the earlier, interim, crosswalk value set. The crosswalk value set mapped utility values from the older EQ-5D-3L measure to the EQ-5D-5L. 170 The new set of utility values157 is intended to replace the crosswalk system. The data for the new value set were collected using a different methodology, at a different time and on a different sample to the original utility value set used for the EQ-5D-3L and the crosswalk value set. This means that differences in utility between the crosswalk and new value set may reflect differences in methods, time and sample rather than underlying preferences and/or changes in preferences over time. Thus, the crosswalk-derived values were used to facilitate comparison between utility values estimated using the older three-level version of the EQ-5D and the more recent five-level version.

Table 36 compares the UK population norms for the EQ-5D-3L utility values and those of the FOCUS trial participants using the EQ-5D-5L and crosswalk value set. This indicates that FOCUS trial participants have lower utility values across all age group, and, overall, than the sample of the general population used to generate the population norms. 174

The utility values for FOCUS trial participants were similar to those reported in other studies. Barton et al. 158 reported EQ-5D-3L baseline utility values of 0.657 (n = 32) to 0.693 (n = 36), whereas Crawford et al. 175 reported EQ-5D-3L utility values of 0.664 to 0.699 (n = 409). The pooled baseline mean utility for participants with severe schizophrenia requiring a change in management in the Cost Utility of the Latest Antipsychotic Drugs in Schizophrenia Study (CUtLASS) (unpublished analysis of utility data from the CUtLASS trial) was 0.628 (SE 0.017, 95% CI 0.595 to 0.661; n = 361). 159,160 Similarly, another large study,159 also using the EQ-5D-3L, looked at the use of antipsychotics in a population with schizophrenia and found that the baseline mean utility was 0.61 to 0.67 (SD 0.29 to 0.33; n = 118). Finally, a recent large study176 (n = 275) of schizophrenia reported higher values using the EQ-5D-5L, with reported baseline utility values between 0.74 and 0.76.

There is limited evidence about the relative cost-effectiveness of CBT in people with psychosis and schizophrenia. We identified three economic evaluations158,177,178 published since 2010 that used broadly similar methods, in a UK setting. Barton et al. 158 compared social recovery-orientated CBT with usual care (defined as active case management), over 9 months’ follow-up. The authors concluded that social recovery-orientated CBT was cost-effective in 54% of scenarios at a WTPT of £20,000 per QALY. 158 A UK evaluation of CBT combined with motivational care, using a societal perspective and a time frame of 18 months, concluded that the probability of the intervention being cost-effective was 69%, even if decision-makers are not willing to pay to gain a 1-point increase in the Global Assessment of Functioning. 177 McCrone et al. 178 evaluated an early intervention service for people with psychosis, which included low-dose medication regimens, CBT, family therapy and vocational rehabilitation. This study found that the intervention did not increase costs and was likely to be cost-effective (probability of 76%) even if decision-makers are not willing to pay to gain an additional person who makes a full or partial vocational recovery. 178

A fourth study179 compared cognitive remediation therapy plus usual care with usual care alone and found it likely to be cost-effective from a NHS and social care perspective but at a limited time horizon of 40 weeks.

Although using different time horizons and health benefit measures, the findings of these studies differ from the findings of this study, which indicates that CBT is not likely to be cost-effective. All identified studies had limitations, including missing data, small sample sizes and challenges controlling for other medications/treatments received outside the trial intervention. In addition, the participants in this study were people unable to tolerate, or with an inadequate response to, clozapine. This may mean that they are less likely to respond to CBT than people who are not treatment resistant or treatment intolerant. Although CBT was associated with a net gain in QALYs, the costs of CBT were not offset by reduced use of other health and social care services; the participants in the CBT group used more services and had higher costs than the TAU group. However, the greater use of health and social care services found in the CBT group may reflect a desirable outcome for this population who typically have a high level of unidentified and unmet need for health and social care. 180 There is evidence that the physical health needs of this population are severely neglected, often as a result of discrimination by health-care providers. 180,181 This can arise if health-care providers attribute physical sympyoms to a person’s mental health condition, often referred to as ‘diagnostic overshadowing’. 181 Moreover, the fear and shame associated with the public stigmatisation of psychosis and schizophrenia often results in feelings of disempowerment and avoidance of help-seeking for health-care needs. 182 The reduction of stigma and discrimination has been identified as a key target for improving the physical health care of people with a serious mental illness. 183 The early mortality associated with these unmet needs indicates an area in which improvements in health and social care are needed. It is suggested that one approach to making such improvements could be to utilise interventions that encourage service users with serious mental illness, such as CRS, to become empowered to seek support for their physical health care. 183 The findings of this trial indicate that participants who received CBT had greater use of the health-care services that people who meet the criteria for CRS have an equal right to access. Increasing the overall level of physical and mental health care for CRS patients may reduce the differences in costs between the TAU and CBT groups in this trial, potentially improving the relative cost-effectiveness of CBT.

Strengths and limitations

The FOCUS trial provides high-quality evidence regarding the clinical effectiveness and cost-effectiveness of CBT for this population, with a low risk of bias across a number of domains. Our trial was pre-registered and our a priori SAP was published online and in the public domain. Compared with other trials that have evaluated CBT, the FOCUS trial had many methodological strengths that reduced the potential for bias: randomisation was implemented via a web-based platform centrally and independently, resulting in concealed allocation; outcome assessors were blinded and there were few instances of unblinding; a high proportion of participants received their allocated intervention; and there was low attrition for the primary outcome. Furthermore, the FOCUS trial was adequately powered to detect a clinically meaningful difference at 21-month follow-up. Data from the CONSORT diagram (see Figure 2) indicate a representative sample of people who meet CRS criteria. Of people identified for the study, 72% agreed to meet with the research team and, of those, 94.5% agreed to take part in the study. Although there is a subgroup of people who met the CRS criteria who refused to meet with the study team, overall the majority of people referred were engaged. Moreover, the services we recruited from were typical of services for CRS, that is CMHTs, clozapine clinics, psychiatrists and inpatient settings.

The FOCUS trial had strong patient and public involvement (PPI) in the conception of the study design and oversight of the study throughout, with PPI representation on the trial management committee, the Trial Steering Committee and the Data Monitoring and Ethics Committee. The trial documentation was reviewed by two co-applicants, who provided PPI representation, interpreted the results and wrote the participant summary sheet.

The RAs and therapist received training before the trial commenced, or, if they were appointed during the trial, received training before they started to see participants. They received regular supervision throughout the lifetime of the trial to ensure that assessments were conducted reliably and CBT was delivered with fidelity to the model. The small treatment effects should be routinely replicable in the NHS, because almost all of our therapists were band 7, which is the lowest NHS band for psychological therapists. The frequency with which FOCUS trial therapists received supervision is likely to be greater for FOCUS trial therapists than in the NHS; however, it could be argued that for the CRS population, in whom there is a greater degree of complexity, more frequent supervision may be indicated. One limitation to the design of the study was the choice of TAU as the comparator. Although this was specified in the commissioned call by the funder as the comparator, we recognise that a control group that does not include non-specific therapeutic factors, such as contact time, does not allow us to exclude the possibility that any observed change was not attributable to the non-specific therapeutic aspects of CBT.

When placing the above findings in context, it is important to note that, although the effects at the end of treatment and long-term follow-up were statistically significant, the changes observed on the outcome measures are unlikely to reach a threshold of clinically significant change. The NNT of 15 to achieve a good response on our primary outcome, a 50% reduction in PANSS score, at 21 months is encouraging. NNT is reported for 25%, 50% and 75% change in PANSS score from baseline and it is widely accepted that a 50% improvement in PANSS score is a good outcome. 32 Clinicians consider NNT to be a useful way to interpret the findings of trials. 184 However, there are limitations to representing the outcome data for this trial as NNT; the amount of change on PANSS from baseline is dependent on the starting point, that is the baseline score, and the use of percentage change to generate the NNT ignores the absolute benefit. The NNT should be presented with CIs; however, when there is no treatment effect, generating an accurate CI is problematic. 185 In the case of the FOCUS trial, the treatment effect on PANSS at 21 months was limited. Therefore, although NNT has been reported here to aid interpretation by clinicians, who regularly use this statistic, its interpretation should be undertaken with caution, with the caveat that the most efficient and least biased way to analyse the trial data is using our primary analysis of a linear mixed model adjusting for baseline. It should also be noted that NNT is one of a number of statistical tests carried out, and multiple hypothesis testing may lead to type I error, inflating the chance of finding a positive result.

The economic analysis was subject to the same strengths and limitations discussed for the clinical evaluation of effectiveness in terms of trial design, trial sample and length of follow-up.

In line with NICE recommendations,156 the measure of health benefit used for the primary analysis was the QALY, estimated from the published EQ-5D utility values. This enables comparison between different disorders, which is useful for policy-makers and commissioners, who have to consider the distribution of limited budgets between different health-care services. However, the EQ-5D is a generic health status measure that may not be sufficiently sensitive to identify important clinical changes in participants’ mental health. The measure covers five domains (mobility, self-care, usual activity, pain and distress and anxiety and depression), but does not include specific symptoms that may be important to service users. This means that the measure could underestimate the benefits of a successful intervention. The descriptive analyses indicated that the EQ-5D health status measure and associated utility index correlated well with the clinical measures used in the trial (see Appendix 3). In addition, the sensitivity analyses using clinical measures of improvement in mental health symptoms (PANSS) and indicators of recovery (QPR) also found that CBT was not likely to be cost-effective.

As planned prior to the end of data collection, the new value set was used to estimate utility values from the EQ-5D-5L. More recently, however, NICE advised that the crosswalk mapping system, developed as an interim method of estimating utility values, should be used. 186 This is to allow time for further work to explore and understand differences between two approaches before the NICE methods guidance is updated. Accordingly, we re-estimated the results using the older crosswalk system as one of the sensitivity analyses. The results of this did not differ substantially from the primary analysis.

There were differences between the CBT and TAU groups in utility scores at baseline, although these differences were not statistically significant. The lower score in TAU participants may reflect the lower overall health status of people in the TAU group. This was accounted for, to some extent, by including the baseline utility value in the calculation of total QALYs between baseline and the end of follow-up. In addition, the EQ-5D thermometer or EQ VAS score at baseline was included as a covariate in the analyses to account for possible differences in overall health and utility between the groups.

The proportion of participants with complete-case data (34%) makes imputation of missing data essential but difficult. In addition, none of the participants for whom complete cost and QALY data were available at 21-month follow-up had psychiatric inpatient stays. In contrast, the available case data at each assessment indicated that this service was used by participants. This suggests that the group of participants with complete cost and QALY data may not be representative of the study sample as a whole. To strengthen the robustness of the results, available data were used at each assessment point to impute cost categories and EQ-5D domains and passively impute total costs and QALYs. The level of missing data for psychiatric inpatient stays was ≤ 20% at baseline and at 9- and 21-month follow-up, which, in part, reflects the use of hospital case note review for participants reporting any psychiatric hospital inpatient stay. Psychiatric inpatient stay was the key cost component of the total cost. In the available and multiple imputation data, it accounted for approximately 70% of the total cost.

With the exception of non-psychiatric hospital stay, the level of missing data for the other cost categories at baseline and 9-month follow-up was ≈30%. However, it was higher (37–44%) at 21-month follow-up. Constraints on researcher time meant that it was not feasible to review primary and community care or non-psychiatric hospital case notes. The level of missing utility data was lower: < 10% at baseline and < 30% at 21-month follow-up.

The greater number of complete data for each cost category means that our approach of imputing cost categories for each assessment point mitigates the overall impact of missing data to some extent (in terms of both bias and imprecision). Nevertheless, the large number of missing cost data at 21-month follow-up means that the results must be interpreted with caution. The extensive uncertainty around estimates from all analysis reflects this.

The primary analysis for the economic evaluation was limited to direct costs, from NHS and social care perspectives. In this population, effective interventions may also affect whether or not a person is employed and the subsequent earnings and benefits they receive. The trial design needed to balance complete and detailed assessment of outcomes and service use with minimisation of participant burden when possible. It was felt to be important to collect detailed service use data at 3-monthly intervals to minimise problems with participant recall. Accordingly, limited information was collected about paid and unpaid work and activities. This covered whether the participant was participating in paid or unpaid/voluntary employment, education or training, other productive activity or was unemployed. The data suggested that there were no statistically significant differences between groups in the proportion of participants engaged in any productive activity or participating in paid employment.

If CBT improves overall health and/or mental health symptoms and recovery, then this may reduce the need for informal carers (e.g. family members), potentially offsetting some of the costs of formal care and CBT. Experience in previous trials indicated a number of barriers to collecting data about the use of informal carers and the time spent by carers. These included incomplete reporting of whether or not participants had family members or friends who they considered to be informal carers. In addition, informed consent was required from participants to contact any informal carers identified, and from informal carers to collect this information. The data collected from informal carers were also incomplete in a number of cases. These factors meant that complete data were available only for a small proportion of participants. 159,160 Combined with the need to collect detailed formal care-use data, it was felt to be outside the scope of this trial to collect information about the use of informal care. This means that, if CBT reduced or increased the use of informal care, our estimates would underestimate or overestimate the relative cost-effectiveness of CBT.

The economic evaluation was limited to the choice of intervention and comparator included within the trial. Although this leads to a robust comparison, it may not reflect the full range of interventions that are available within a clinical setting. For example, other economic evaluations identified looked at a wider range of psychological therapies in a population with schizophrenia, including cognitive remediation therapy, group art therapy and body psychotherapy. 179,187,188 A modelling study, informed by a network meta-analysis of trial data, would have the potential to compare costs and outcomes across a wider range of interventions. However, in this treatment-resistant population there are likely to be limited data to inform the model structure and populate the model.

Although PANSS is commonly used as a primary outcome measure in treatment trials for people who meet criteria for a schizophrenia diagnosis (both pharmacological and psychological), it may not have been the most appropriate outcome for this trial for a number of reasons. Although PANSS has been rated favourably by service users,116 the same study also found that, overall, service users have a preference for self-rated measures. In addition, it has been argued that distinctions between pharmacological and psychological interventions should be made clearer in that psychological interventions, such as CBT, are not a quasi-neuroleptic. Therefore, outcome measures should reflect the target of the specific intervention. 121 For CBT this could be positive symptoms,67 but it may be more appropriate to consider the reduction of emotional distress given that this is a primary aim of CBT, or self-rated recovery, given the emphasis in CBT on working with the client’s own problem list and goals.

The analysis adjusting for the measure of working memory at baseline needs to be interpreted with caution for two reasons. First, the response rate at baseline was poor because the assessment was introduced 9 months into the recruitment window; a small proportion of data was expected to be missing given that the working-memory assessment was introduced 9 months into the recruitment window. Burden may also explain missing data for this measure. There were a relatively high number of measures in the assessment battery; more specifically, in relation to the working-memory measure, the participant was required to recall and reorder sequences of letters and numbers from a string length of two to seven, and this may have been too burdensome. Second, we proposed a sensitivity analysis using our own algorithm for scoring the measure that was not described by the author.

Although the RAs were blind to allocation, it is not possible to blind participants from the treatment allocation and mask the receipt of CBT when the control is TAU. It could be argued, therefore, that the treatment effect, because it was so small, might be an artefact of the design.

Our inclusion criteria for CRS had been employed in a previous study of clozapine augmentation with a second antipsychotic. 52 This did not include tests to determine, for each potential participant, whether or not there were any periods of less than full clozapine adherence, that an adequate plasma clozapine level (350 ng/ml) had been achieved consistently for an adequate time or substance use (that may have adversely influenced the effectiveness of the clozapine treatment). It could be argued that such tests were necessary to establish clozapine unresponsiveness; however, given the existing precedent for our inclusion criteria, it was considered unrealistic and impracticable to apply criteria relating to the adequacy of the clozapine trial for each patient being screened for eligibility, which involved checks and investigations that are not routinely done in clinical practice and which, for many patients, could prove to be difficult if not impossible to definitively establish whether or not they were met.

Implications for public health/treatment services

The results of the FOCUS trial suggest that CBT should not be offered routinely with the aim of achieving lasting symptom reduction in people who meet the criteria for CRS. However, the finding that CBT has short-term effects for symptoms and long-term effects for self-recovery, and had a NNT of 15 for good response on our primary outcome, suggests that services could consider offering CBT as a pragmatic individual trial. This may be particularly indicated when the service user’s goals relate to acutely distressing positive symptoms and voices or longer-term recovery-orientated goals. The economic evaluation indicates that participants in the CBT group had a net gain in overall health as measured by QALYs. The CBT group also used more services over the course of the trial. Whether this was attributable to chance or to the intervention is unclear. The economic analysis of the participants with complete cost and QALY data also indicates that there may be participants for whom CBT is more likely to be cost-effective. One feature of those with complete cost and QALY data was that they had lower costs in the 3 months prior to the baseline assessment. As outlined, it was not possible to identify any subgroups of participants who had a good outcome from CBT and, therefore, if offering a pragmatic trial of CBT, it should be considered for all service users who meet the criteria for CRS. In addition, the uptake of CBT by FOCUS trial participants provides an optimistic message for the ability to engage people whose experiences are considered ‘treatment resistant’.

A commonly used approach to the treatment of CRS is to augment clozapine with a second antipsychotic. 53 The treatment ES for augmentation with a second antipsychotic has been shown to be in the small but significant range. The ESs found in the FOCUS trial for CBT are also very similar to those found for pharmacological augmentation with a second antipsychotic. 55 However, the FOCUS trial provides a more rigorous test of the effects of a treatment for CRS, with a lower risk of bias than the few, small, short-duration, low-quality trials that contribute to the meta-analyses of augmentation with antipsychotics. In addition to the methodological limitations of the clozapine augmentation trials, the current evidence base cannot provide a clear indication from the literature to indicate which clozapine combination strategy is superior. 58 The adverse effect profile for CBT is also likely to be favourable when compared with the likely cardiovascular risks associated with multiple antipsychotic medications. Therefore, for service users who are reluctant to consider pharmacological augmentation because of the likely side effect burden of polypharmacy, the provision of CBT may be indicated.

A pragmatic trial of CBT may be more effective if offered earlier in the course of clozapine use, when treatment resistance is first established and when DI is shorter, given the personal and social functioning subgroup analysis, which indicated that participants with a shorter DI had greater benefits from CBT in this domain. This may be particularly important as entrenched symptoms may persist for longer and respond more slowly to CBT, which is similar to the use of medication. A person may need to attend more sessions of CBT to derive maximum benefit, given that results of the CACE analysis indicated a potential dose response: for every session of CBT attended, there was a 0.11 reduction in the PANSS total score.

The FOCUS trial recruited people who met the criteria for CRS and, for this reason, the small and short-term effect on symptoms does not mean that CBT will not have stronger effects for less severe or earlier psychosis populations. The NICE guideline for the treatment of psychosis and schizophrenia30 recommends that EIP services routinely offer CBT and family intervention for people with a first episode of psychosis, and meta-analysis of the effects of CBT for this population has demonstrated that it can reduce symptom severity. 189

Finally, it is important to note that the FOCUS trial has an optimistic message for people who meet the criteria for CRS: both the CBT and TAU groups recovered during the trial period to a degree that, on average, approaches clinical significance. Given the DI, poor response to medication and lack of social opportunities (such as employment) among FOCUS trial participants, the finding that, on average, each group had a 10-point improvement in PANSS total scores is a positive one.

Implications for future research

Clearly, this is a population of people who have significant unmet need in relation to their physical and mental health and, therefore, future research to develop and evaluate more effective interventions is required. This could include pharmacological (although issues of physical health and side effects are likely to be important) and other new interventions of either a psychological or social nature. Given the current criticism of CBT for psychosis trials, future RCTs of psychological or social interventions for this population should also demonstrate high methodological rigour and a low risk of bias, as this will help to identify populations most likely to benefit from this intervention.

More specifically, in relation to CBT for this population, approaches to analysis, such as trajectory analysis, may identify responders to CBT; however, it was not possible to use this analytical approach in the FOCUS trial because of the limited number of time points at which our data were collected. There is no research to date to indicate whether or not there are any active ingredients of CBT for the CRS population, and, in the FOCUS trial, a number of techniques were utilised within therapy. Therefore, from the FOCUS trial data, it should be explored if there are any specific therapeutic techniques that result in a good outcome from CBT. Previous research indicates that there is a greater treatment effect from CBT for people considered at risk of developing psychosis if therapy involves between-session tasks and formulation,154 and mediation analyses of the FOCUS trial data may reveal components of therapy as predictors of response.

The mean health and social care costs for the 3 months prior to the baseline assessment were lower in the analysis of participants with complete cost and QALY data (£686, 95% CI £572 to £801; n = 169) than the MI analysis of all participants (£985, 95% CI £863 to £1107; n = 487). This indicates that, for whatever reason, these participants were making less use of services. In addition, the analysis indicated that participants in the CBT group used more health and social care services than those in the TAU group, particularly towards the end of the follow-up period. Future research could explore the reasons for high and low service use and whether it indicates higher need for care, or whether CBT helps participants and/or health-care professionals better recognise the need for care.

The economic evaluation was limited to the choice of intervention and comparator included within the trial. Although this leads to a robust comparison, it may not reflect the full range of interventions that are available within a clinical setting. A modelling study, informed by a network meta-analysis of trial data, would have the potential to compare costs and outcomes across a wider range of interventions. This could include comparisons of TAU plus CBT with TAU plus other forms of counselling, supportive care or psychosocial therapy interventions or with TAU plus pharmacological augmentation. However, in this treatment-resistant population, there are likely to be limited data to inform the model structure and populate the model.

Conclusions

Cognitive–behavioural therapy for people who meet the criteria for CRS has small but significant improvements for symptoms of psychosis in the short term, but there is not a lasting effect in the long term (except in relation to self-rated recovery). CBT is a highly acceptable treatment with little evidence of adverse effects for this population. Finally, psychotic symptoms in CRS improved in both groups to a clinically significant extent over 21 months, suggesting that long-term recovery is possible.

Contributions of authors

Anthony P Morrison (Professor of Clinical Psychology) planned the study, contributed to the application for funding, made substantial contributions to the design of the trial protocol, the SAP, managed the trial as chief investigator, contributed to writing and critically read the manuscript.

Melissa Pyle (Research Trial Manager) made substantial contribution to the development of the trial protocol, as well as overall management of the trial and data management and wrote the first draft of the manuscript.

Andrew Gumley (Professor of Psychological Therapy) contributed to the application for funding, made a substantial contribution to the design of the trial and protocol and critically read the manuscript.

Matthias Schwannauer (Professor of Clinical Psychology) contributed to the application for funding, made substantial contribution to the design of the trial and protocol and critically read the manuscript.

Douglas Turkington (Professor of Psychosocial Psychiatry) contributed to the application for funding, made substantial contribution to the design of the trial and protocol and critically read the manuscript.

Graeme MacLennan (Director and Professor, The Centre for Healthcare Randomised Trials) made substantial contribution to the development of the trial design and protocol, as well as the SAP, conducted the analysis and critically read the manuscript.

John Norrie (Professor of Medical Statistics and Trial Methodology) contributed to the application for funding, made substantial contributions to the design of the trial and protocol, the SAP, and critically read the manuscript.

Jemma Hudson (Trial Statistician) made substantial contribution to the development of the trial design and protocol, as well as the SAP, conducted the analysis and critically read the manuscript.

Samantha Bowe (Clinical Psychologist) made substantial contribution to the development of the trial design and protocol, and critically read the manuscript.

Paul French (Mental Health Clinical Lead for Greater Manchester and East Cheshire Strategic Clinical Network, NHS England North Clinical Lead for EIP and Honorary Professor) contributed to the application for funding, made substantial contributions to the design of the trial and protocol, and the SAP, and critically read the manuscript.

Paul Hutton (Associate Professor of Therapeutic Interventions and Honorary Consultant Clinical Psychologist) made substantial contribution to the development of the trial design and protocol, and critically read the manuscript.

Rory Byrne (User-led Researcher) contributed to the application for funding, made substantial contribution to the design of the trial and protocol and critically read the manuscript.

Suzy Syrett (User-led Researcher) contributed to the application for funding, made substantial contribution to the design of the trial and protocol and critically read the manuscript.

Robert Dudley (Consultant Clinical Psychologist and Associate Clinical Lecturer) made substantial contribution to the trial protocol and critically read the manuscript.

Hamish J McLeod (Senior Lecturer in Mental Health and Wellbeing) made substantial contribution to the development of the trial design and protocol, and critically read the manuscript.

Helen Griffiths (Clinical Psychologist and Lecturer in Clinical Psychology) made substantial contribution to the development of the trial design and protocol, and critically read the manuscript.

Thomas RE Barnes (Professor of Clinical Psychiatry) contributed to the application for funding, made substantial contribution to the design of the trial and protocol and critically read the manuscript.

Linda Davies (Professor of Health Economics) contributed to the application for funding, made substantial contributions to the design of the trial and protocol, the health economic analysis plan, and critically read the manuscript.

Gemma Shields (Health Economist) made substantial contribution to the health economics SAP, conducted the health economics analysis, conducted the health economics section of the manuscript and critically read the manuscript as a whole.

Deborah Buck (Health Economist) made substantial contribution to the health economics SAP, conducted the health economics analysis, conducted the health economics section of the manuscript and critically read the manuscript as a whole.

Sarah Tully (Assistant Research Psychologist) contributed to the trial protocol, significantly contributed to writing the manuscript and critically read the manuscript.

David Kingdon (Professor of Mental Health Care Delivery) contributed to the application for funding, made substantial contribution to the design of the trial and protocol, and critically read the manuscript.

All authors read and approved the final manuscript.

Publications

Pyle M, Norrie J, Schwannauer M, Kingdon D, Gumley A, Tukington D, et al. Design and protocol for the Focusing on Clozapine Unresponsive Symptoms (FOCUS) trial: a randomised controlled trial. BMC Psychiatry 2016;16:1–12.

Morrison AP, Pyle M, Gumley A, Schwannauer M, Turkington D, MacLennan G, et al. Cognitive behavioural therapy in clozapine-resistant schizophrenia (FOCUS): an assessor-blinded, randomised controlled trial. Lancet Psychiatry 2018;5:633–43.

Data-sharing statement

All data requests should be submitted to the corresponding author for consideration. Access to anonymised data may be granted following review.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HTA programme or the Department of Health and Social Care.