Notes
Article history
The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 12/64/124. The contractual start date was in November 2013. The final report began editorial review in April 2018 and was accepted for publication in December 2018. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
Richard Grieve is a member of the National Institute for Health Research (NIHR) Health Technology Assessment Commissioning Board since January 2018. Kathryn M Rowan is a member of the NIHR Health Services and Delivery Research Board since September 2014.
Permissions
Copyright statement
© Queen’s Printer and Controller of HMSO 2019. This work was produced by Mouncey et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
2019 Queen’s Printer and Controller of HMSO
Chapter 1 Introduction
Background and rationale
Over 170,000 patients are admitted to adult, general critical care units in the NHS each year. 1 Meta-analyses of outcome data for survivors indicate a pooled prevalence of 25% for clinically important post-traumatic stress disorder (PTSD) during the first 6 months after discharge from critical care. 2 Similar figures for anxiety and depression are 40%3 and 34%,4 respectively. Patients who develop serious psychological morbidity are also at higher risk of further physical morbidities and mortality. 5–7
Experiencing acute stress and early memories of frightening critical care experiences (e.g. hallucinations, delusions and nightmares), while in the unit, are among the known risk factors for longer-term, post-critical care psychological morbidity including PTSD, depression and anxiety. 8–13
Research has estimated that ≈50% of patients experience acute stress, and up to two-thirds experience hallucinations and delusions, while in the critical care unit. 14,15 Acute stress, including symptoms of anxiety, low mood and panic, may be caused by a range of difficult, cumulative experiences that are common for patients in critical care units: fear of dying; invasive treatments, such as mechanical ventilation; pain and discomfort; inability to communicate; and terrifying hallucinatory delusions. 8,14,16,17 The aetiology of critical care hallucinations and delusions is unknown, but they have been linked to delirium, the provision and withdrawal of sedatives and other psychoactive drugs, effects of illness (such as sepsis), immobility, and sensory and sleep deprivation. 8,15,18 Hallucinations and delusions are known, from the psychosis literature, to be exacerbated by, and comorbid with, acute stress. Critical care unit hallucinations frequently have horrifying themes such as conspiracy to kill by staff, torture, poisoning, demons, extortion or organ theft;19 thus, a vicious cycle of stress, confusion and terror is common for many, though not all, critical care unit patients.
In 2000, the UK Department of Health and Social Care recognised that the critical care unit was an extremely distressing place for patients and that there was considerable need for psychological support for patients. 20 In 2009, the National Institute for Health and Care Excellence (NICE) recommended that all critically ill patients should be assessed for risk of non-physical (i.e. psychological) morbidity, and that those deemed to be at high risk of adverse outcomes, such as PTSD, should receive structured psychological support as part of an individualised rehabilitation plan, both during and after their critical care unit stay. 21 Separately, a NICE evidence update on PTSD22 suggested that, compared with no psychological intervention, an early (delivered immediately), brief (three sessions), trauma-focused psychological intervention has the potential to reduce the development of subsequent symptoms; the evidence update also called for further research. 22 However, little is currently done to alleviate patients’ stressful experiences in critical care units, because of a lack of strong evidence about what may help.
The modification of clinical risk factors for PTSD, such as duration of mechanical ventilation and sedation, has been discussed in the literature,23,24 but less invasive medical interventions or better drugs are currently not widely available. There is a lack of high-quality research evaluating psychological interventions in the critical care setting with a view to preventing the development of longer-term psychological morbidity. 25 To date, only a critical care unit diary intervention26 and a clinical psychology intervention27 have been shown to have an effect on longer-term psychological outcomes. Outside the critical care unit setting, cognitive–behavioural therapy (CBT) techniques have been found to be effective in reducing symptoms of stress in patients with mental or physical illness, mitigating hallucinations and delusions in mental health settings and in reducing PTSD symptoms. 21,22,28–39
Research suggests that post discharge (e.g. at 6 weeks40 or at outpatient follow-up clinics41) may be too late to provide psychological interventions for critical care unit survivors, and that earlier intervention could be more beneficial in preventing longer-term psychological morbidity. For example, a clinical psychology study27 indicated that considerably fewer individuals reported experiencing PTSD, depression or anxiety one year after their critical care unit stay when they had received interventions by practitioner psychologists while in the critical care unit. However, this study27 did not specify or standardise the interventions that patients received and was not a randomised clinical trial (RCT). The diary intervention was effective in reducing the proportion of patients with PTSD at 3 months in a RCT, but it retrospectively targets factual memory gaps rather than acute stress. Although diaries are written by nurses and/or relatives in the unit, it is not an early intervention because patients receive the diaries at varying time points after discharge.
Very few NHS critical care units have regular access to psychologists as part of their multidisciplinary critical care team (because of resource and other constraints). Acknowledging this and the evidence indicating that non-experts can be trained to deliver effective psychological interventions in other settings [i.e. studies have evaluated CBT techniques as effective when delivered by non-expert staff (including nurses) to patients with psychosis in mental health settings], we developed a preventive, complex psychological intervention to be led by specially trained nurses, who were selected by local critical care units. 30,31,42–44
To develop the preventive, complex psychological intervention, previously, members of the team, led by the lead adult critical care health psychologist (DW), followed stage one of the Medical Research Council (MRC) framework (see Appendix 1) to guide researchers to develop and evaluate complex interventions. 45 Stage one involved three steps, which were to:
-
Identify the evidence base for an intervention to reduce patients’ acute stress.
-
Develop a theoretical understanding of the likely process of change in reducing stress.
-
Model the process to progressively refine the intervention.
Evidence was reviewed to inform the team’s understanding of the determinants of acute stress in critical care patients, and the likely mechanisms of change involved in reducing acute stress. We carried out a systematic review of psychosocial interventions to reduce acute or long-term stress of critical care patients (step one). 46 The results46 indicated that interventions such as music, massage, relaxation and psychology sessions in critical care reduced acute or long-term stress in 12 out of 23 studies. To develop a theoretical understanding of the likely process of change, we conducted quantitative and qualitative research (Dr Dorothy Wade, University College London Hospitals NHS Foundation Trust, 4 July 2019, personal communication)14,19,47 with patients and nurses to learn about their experiences of managing patients’ stress, and applied existing health psychology theories of stress, health and coping, and clinical psychology theories of psychosis and post-traumatic stress (step two). 48–50
Based on steps one and two, we hypothesised that an early intervention, commencing while in the critical care unit and combining evidence-based psychological techniques from CBT for psychosis and trauma and coping strategies such as relaxation, meditation and music, could help alleviate acute stress and the development of longer-term psychological morbidity. We adapted techniques from CBT that have been found to be effective in other populations and settings in reducing acute stress, lessening the impact of hallucinations and delusions, and preventing post-traumatic stress, to the critical care setting. The intervention was modelled (step 3) with critical care patients by a health psychologist and senior nurses in one unit, in consultation with former patients and experts in health and clinical psychology, PTSD and psychosis. This was an iterative process to progressively refine the intervention.
The outcome of this process was the development of an early, nurse-led, preventive complex psychological intervention with the aim of alleviating acute stress and the development of longer-term psychological morbidity in critical care patients. The intervention comprised three elements:
-
creating a therapeutic environment in critical care
-
three stress support sessions for patients identified as acutely stressed
-
a relaxation and recovery programme for patients identified as acutely stressed.
Aim
The overall aims of the Psychological Outcomes following a nurse-led Preventative Psychological Intervention for critically ill patients (POPPI) study were to produce a standardised preventive, complex psychological intervention (the ‘POPPI intervention’) and, if deemed feasible, evaluate whether or not it is beneficial to critically ill patients and the NHS in terms of clinical effectiveness and cost-effectiveness.
Objectives
The POPPI study consisted of two phases: (1) a standardisation and feasibility phase and, if deemed feasible, (2) a cluster RCT. The overall objectives were to:
-
standardise the preventive, complex psychological intervention
-
develop an education package and support tools to deliver the intervention
-
test the feasibility and acceptability of the education package, support tools and delivery of the intervention to patients and staff (i.e. the intervention feasibility study)
-
refine the education package, support tools and intervention based on results of the intervention feasibility study
-
test the feasibility of the proposed procedures for the cluster RCT (i.e. the RCT procedures feasibility study)
-
evaluate, using a parallel-group cluster RCT design, the clinical effectiveness and cost-effectiveness of the intervention, including an integrated process evaluation (if deemed feasible).
Chapter 2 Standardisation of the POPPI intervention
Objectives
-
To standardise the proposed POPPI intervention.
-
To develop an education package to train staff to deliver the POPPI intervention.
-
To develop support tools and materials for staff and patients.
Oversight
The content of the POPPI intervention was informed, developed and refined by the lead adult critical care psychologist (DW), supported by two senior psychologists (CRB and JWei) and senior nurses (DS and JWel). An expert psychology advisory group (EPAG) was set up to oversee both the standardisation of the proposed intervention and the development of the education package and support tools.
The EPAG, chaired by Professor Daniel Freeman (a senior clinical psychologist and MRC Senior Clinical Fellow), met every 2 months and included expertise in clinical psychology, clinical education, critical care nursing and lived experience from former critical care patients. The former patients were chosen from a patient and family advisory group, set up in 2012 at University College London Hospitals NHS Foundation Trust (UCLH). Members brought clinical experience in psychosis and PTSD from University College London and from the Psychological Interventions Clinic for outpatients with Psychosis service, at the internationally renowned centre for training in CBT for psychosis at the Maudsley Hospital and Institute of Psychiatry, Psychology & Neuroscience at King’s College London, as well as experience in clinical and medical education, from Queen Mary University of London.
Standardisation of the intervention
Even though parts of the POPPI intervention had been delivered on a local level, it was vital to standardise the content of the proposed intervention to ensure that it could be delivered consistently across a number of different hospitals and by non-experts. To do this, we established whether or not any modification to the content of the proposed intervention was needed by reviewing stage one of the MRC framework (see Chapter 1, Background and rationale for details). 45
Building on our previous work,16,47 we updated the evidence base on critical care-related psychological risk factors, outcomes and interventions. Two authors (DW and JWei) completed a systematic review of psychosocial interventions in critical care. 51 We then completed step three (modelling process and outcomes), which the clinical research team had begun prior to the POPPI study.
This work confirmed that the key psychological outcomes should remain as PTSD and depression, as they were the most serious common psychological outcomes of critical care. As research in identifying modifiable critical care risk factors is more extensive for PTSD than for depression, we confirmed that the target of the intervention should remain as reducing the development of PTSD symptoms, which would be chosen as the primary outcome of the cluster RCT (if deemed feasible).
Creating a therapeutic environment in critical care
The aim of the first element of the POPPI intervention was to make all staff aware of the psychological distress endured by many critical care patients and to help staff learn ways to reduce stress in the environment and improve communication with distressed and fearful patients. When standardising the content of this element, the following key aspects were deemed to be required:
-
Increasing awareness and understanding by staff of acute stress and poor psychological outcomes experienced by critical care patients.
-
Identifying and reducing stressors in the critical care unit such as loud noise, unnatural light, pain, sleep deprivation and psychoactive drug effects.
-
Improving communication between staff, families and distressed patients, particularly those who are delirious or experiencing hallucinations and/or delusions.
-
Promoting a sense of hope and optimism during the psychological and physical recovery period.
Three stress support sessions for patients identified as acutely stressed
Building on the existing evidence and drawing on theory and techniques from CBT for psychosis and PTSD, the second element of the POPPI intervention was designed to be three, one-to-one, stress support sessions delivered to high-risk (acutely stressed) patients in hospital by a specially trained, self-selected, critical care nurse (‘POPPI nurse’). Acute stress was detected if a patient scored ≥ 7 on the Intensive care Psychological Assessment Tool (IPAT), previously developed and validated at UCLH. 52 The three sessions included techniques adapted from CBT that were deemed appropriate for patients early in recovery, such as emotional expression, normalisation, cognitive reappraisal, psychoeducation and ‘homework’ tasks. Exposure to traumatic memories (one element of trauma-focused CBT) was not included as it was deemed inappropriate; for example, during the stress support sessions, patients are encouraged to express their thoughts and feelings about their critical care experiences if they wish, but not to undertake ‘reliving’ of their critical care stay, particularly as their trauma may be continuing.
It was planned that each session would last ≈30 minutes and that all three sessions would be delivered within a week (for the pragmatic reason that many patients might be discharged from hospital around that time) by the same trained POPPI nurse. Because of the nature of the sessions, they would start when the patient was awake and alert, either in the critical care unit or following discharge to the hospital ward.
The main aim of the stress support sessions was for nurses to develop a trusting relationship with patients, so patients could discuss concerns that they might feel embarrassed or worried about communicating, and to reduce emotional distress. The objectives included establishing a collaborative relationship focused on reducing distress; managing patient concerns, including hallucinations and delusions; psychological education (psychoeducation) to reduce distressing interpretations of unusual experiences, reduce stigma and encourage open communication; and provision of active coping strategies. 48 Each session had five component parts. Patients were also asked to rate their stress levels on a ‘stress thermometer’ (a simple tool commonly used by psychotherapists to rate stress levels during therapy on a scale of 0–10) at the beginning and end of each session, to help nurses monitor how patients were responding as the sessions progressed.
Under the guidance of the lead adult critical care psychologist (DW) and supervision from the EPAG, the content of this element was standardised by two senior critical care nurses (DS and JWel) modelling the process of the stress support sessions with patients, after receiving training in delivering the sessions and close support from the health psychologist (DW) and a clinical psychologist from the EPAG. This step was important to ensure that the stress support sessions could be delivered to patients by specially trained non-experts such as experienced critical care nurses. Testing and revision of the elements of the stress support sessions continued prior to confirmation of the final content and process.
Relaxation and recovery programme for patients identified as acutely stressed
The third element of the POPPI intervention was a relaxation and recovery programme for high-risk (acutely stressed) patients. The programme was split into two parts. The first, starting within the first stress support session and to be used during the patient’s hospital stay, was designed to:
-
provide meaningful activity and distraction
-
help people practise new coping strategies to reduce stress and improve sleep
-
learn from other patients’ experiences, between and following the stress support sessions.
Content was aimed at patients receiving the stress support sessions, who were to be given access to calming classical and ambient music; relaxation, breathing and visualisation exercises including ‘the safe place’ exercise; meditation exercises; nature sounds and videos; and patient recovery stories.
The second part focused on providing patients with information on making a good psychological recovery after a critical care stay and building on the support patients received during the stress support sessions, focusing on well-being and coping strategies. Patients would be given information about what to expect in the early days (e.g. leaving critical care, returning home, emotions after critical care, relationships), tips for psychological well-being, advice on coping with difficulties (e.g. worries, panic, low mood, memories), information on sources of psychological support, further information for family and friends and a personal action plan to promote well-being and recovery.
Development of the education package
Creating an education package to train critical care staff to deliver each element of the POPPI intervention was a major focus of phase one of the POPPI study. We adopted a ‘blended-learning’ approach, combining online training (e-learning) and face-to-face teaching. Interactive and engaging online training courses are recommended by educationalists for the provision of knowledge to a large number of staff, whereas face-to-face training is suitable for adults acquiring new skills by practising and receiving feedback. 53 The education package was created by the clinical research team with support from the medical educationalist and other members of the EPAG, with specialist support from an online training designer and a medical film-maker from UCLH.
Creating a therapeutic environment in critical care
With guidance from the EPAG, it was decided that the most efficient way to ensure that the key learning aspects of this element reached as many of the critical care staff as possible was to create the POPPI online training course (titled ‘Improving communication and psychological care in the ICU’). The course was co-designed with experts to give a balance of concise, readable text with graphics and other visual or audio aids. The online training course was divided into four sections (see Standardisation of the intervention) with test-yourself questions at the end of each section (with informative feedback). Videos formed a key part of the course and included former patients talking about their experiences of critical care and nurses modelling good communication techniques with simulated patients. There was an assessment at the end of the training; on passing (i.e. a score of ≥ 80%), staff received certificates.
Three stress support sessions for patients identified as acutely stressed
To enable training to be conducted for three, self-selected, ‘POPPI’ nurses from multiple sites simultaneously, we created a central, face-to-face, 3-day training course. This course was devised under the guidance of the medical educationalist on the EPAG, with a major focus on delivery of the stress support sessions and the relaxation and recovery programme. In addition to the stress support sessions, key psychological principles underlying the intervention were taught and all three elements of the intervention were covered extensively during the 3-day training course.
Owing to the complexity of delivery of the stress support sessions, we decided it was necessary to spend a significant amount of available time devoted to skills practice in delivering the sessions. This practice would involve the training team (a psychologist, nurses and patient representatives) observing trainees delivering the sessions and offering feedback. We decided to have two patient representatives (former critical care patients) talking about their experiences of critical care each day; use games and exercises to enhance learning; and show and analyse a video of a sample stress support session delivered by a clinical psychologist specialist in psychosis.
We also designed the course to educate the POPPI nurses in their wider role in their units, including encouraging staff to undertake the online training and create and promote a therapeutic environment in the critical care unit; providing bedside teaching; and overseeing delivery of the whole intervention.
Finally, we planned an additional training day of feedback and assessment, aimed at supporting the trained POPPI nurses once they had delivered sessions to at least one patient. The day would include a focus group with the nurses on their experience of delivering all three elements of the intervention, and a competency assessment, in which each nurse delivered the second stress support session to a simulated patient (actor), observed by two trainers who completed an assessment checklist. Nurses would be required to pass the assessment. Anyone not passing the assessment would receive further training prior to sign-off of competency.
Debriefing and support for the POPPI nurses
A clinical supervision structure with one of the trainers (psychologist or senior nurse) was set up to allow regular ‘debriefing and support’ telephone calls. The first debriefing and support call was aimed to be made during or soon after the three stress support session nurses delivered to their first patients. Subsequent calls would be scheduled every 2 months, or at nurses’ request. The focus of these calls would be on enhancing nurses’ skills and discussing patient cases. E-mails could be exchanged between nurses and trainers about issues arising from stress support sessions at any time.
Relaxation and recovery programme for patients identified as acutely stressed
As the third element, the relaxation and recovery programme to reduce stress and improve sleep, is introduced to patients by POPPI nurses during the stress support sessions, we planned that education for element three would take place during the 3-day training course. This was to cover how to use the relaxation and recovery programme, theories of relaxation and mindfulness, and creating a patient action plan.
Development of support tools and materials
Creating a therapeutic environment in critical care
Materials to aid and encourage completion of the online training course were produced. Flyers, cards and posters advertising the online training were created to be displayed and left in communal staff areas.
Three stress support sessions for patients identified as acutely stressed
Associated training materials were developed for the POPPI nurses, including an intervention manual, a set of slides for the 3-day training course and a training folder. All were designed for readability and clarity, with short sections and clear signposting. All materials were designed using the same palette of fonts and colours, and included photographs, graphics and colourful diagrams. The writing of the stress support session manual was led by the lead adult critical care health psychologist (DW) with input and guidance from clinical psychologists and senior nurses, under the supervision of the EPAG. The manual comprised 32 pages, with sections on the POPPI nurse role; creating a therapeutic environment; screening with the IPAT; the relaxation and recovery programme; background and introduction to the stress support sessions; a two-page summary and one-page diagram of each session; and appendices of notes, scripts, language and examples to help nurses deliver the stress support sessions.
The manual was designed to be given to the POPPI nurses on the 3-day training course along with a training folder consisting of course hand-outs, games and exercises to practise stress support skills such as ‘guided discovery’ and ‘psychological education’, example patient scenarios for the skills practice sessions, laminated stress thermometers, summaries of each stress support session, and structured reflective note sheets for the nurses.
Relaxation and recovery programme for patients identified as acutely stressed
To give patients access to the relaxation and recovery programme, we developed an application (hereafter referred to as an app), which was loaded onto a tablet computer for use in hospital between stress support sessions, and a digital versatile disc (DVD) and patient booklet to use following the sessions and at home after hospital discharge.
The functionality of the app was developed by a web designer from UCLH, working closely with two psychologists who devised and procured the content. It had a green nature-scene background and large coloured buttons for patients to use for easy navigation between different sections. App contents included a ‘safe-place’ visualisation and relaxation exercise, muscle and breathing relaxation exercises; a body scan and other mindfulness meditations; relaxing classical music from Bach to Vivaldi and calming, modern ambient music; and restful nature sounds and videos. The app also included a section of former critical care patients’ recovery stories, to help to normalise emotional reactions and unusual psychological experiences in critical care, and to encourage hope and optimism for recovery. There were five stories from patients of differing age, sex and ethnicity, illustrating a range of critical care experiences. These were edited from patient interviews conducted by a psychologist and filmed by the medical film-maker.
The DVD included a shorter selection of relaxation exercises and music from the app, and longer versions of the patient recovery stories, assuming longer concentration spans as patients get closer to hospital discharge or go home.
The Getting Well, Staying Well patient booklet was developed to provide patients with useful information on making a good psychological recovery after a critical care stay. The booklet was designed to build on the support patients had already received during the stress support sessions. Getting Well, Staying Well was designed as a readable guide focusing on psychological well-being and positive coping strategies to help patients better deal with the challenges of recovery.
Research has shown the effectiveness of CBT-based self-help books for common mental health problems and also found that guided self-help, in which a professional is involved in supporting and offering guidance on how to use the self-help materials, makes self-help more effective than the provision of information alone. In line with this, the patient booklet involved guidance from the POPPI nurses during the stress support sessions on the use of the booklet and DVD.
Improving Access to Psychological Therapies guidelines54 recommend that self-help resources should follow three principles: (1) communicate a normalising recovery-focused message, (2) help patients understand their difficulties in a timely and understandable way and (3) facilitate knowledge and acquisition of evidence-based interventions to enhance self-efficacy and promote self-management.
The Getting Well, Staying Well booklet has the following contents:
-
personal stay-well plan
-
information about what to expect after critical care in the early days
-
seven tips for psychological well-being
-
advice on coping with difficulties (e.g. worries, panic, low mood, memories)
-
further information on sources of psychological support
-
information about the relaxation and recovery programme (DVD)
-
information for family and friends.
The personal stay-well plan would be used to address the potential challenges ahead and create an individual psychological recovery plan based on advice and information in the rest of the booklet.
Conclusion
Building on previous work by this team, the proposed intervention was standardised to allow consistent delivery across different hospitals, by non-experts. To facilitate this, an education package and associated support tools were developed (Table 1). The key assumptions and theory underpinning the POPPI intervention are diagrammatically represented in Figure 1. The intervention and education package were ready for testing the feasibility of delivery and acceptability.
| Patient intervention | Who receives? | Where? | Proposed mechanisms of change | Training methods | Who is trained? |
|---|---|---|---|---|---|
| Element one: creating a therapeutic environment in critical care | All critical care patients | In the critical care unit |
|
|
All clinical critical care staff |
| Element two: three stress support sessions for patients identified as acutely stressed | Patients identified by IPAT as acutely stressed (score of ≥ 7 points, range 0–20) | In the critical care unit; on wards following discharge from critical care |
|
|
Three POPPI nurses per critical care unit, selected by units with reference to suitability criteria |
| Element three: relaxation and recovery programme on app, DVD and booklet (i.e. music, relaxation, meditation, patient recovery videos and self-help information) | Patients identified as acutely stressed and receiving stress support sessions | In the critical care unit, on wards (via tablet computer) and at home (via DVD and self-help booklet) |
|
Included in training for element two | POPPI nurses |
FIGURE 1.
The POPPI intervention logic model. HRQoL, health-related quality of life.
Chapter 3 Feasibility, piloting and refinement
Objectives
-
To test the feasibility and acceptability of the education package, support tools and delivery of the intervention to patients and staff.
-
To test the feasibility of the proposed procedures for the cluster RCT (including confirmation of the recruitment and retention rates).
-
To refine the education package, support tools, intervention and cluster RCT procedures based on results of the feasibility studies.
The scope of the two feasibility studies is presented in Figure 2.
FIGURE 2.
Scope of the POPPI feasibility studies.
Methods
Research governance
A joint protocol for two feasibility studies was prepared and submitted to a Research Ethics Committee (REC) for approval. Applications for adoption onto the National Institute for Health Research (NIHR) Clinical Research Network (CRN) portfolio and the International Standard Randomised Controlled Trial Number (ISRCTN) registry were made.
Recruitment
Sites and POPPI nurses
Two NHS adult, general critical care units (one located in a teaching hospital and one in a district general hospital) were invited to participate in the intervention feasibility study. Two different units (also one in a teaching hospital and one in a district general hospital) were invited to participate in the RCT procedures feasibility study. All four sites were asked to identify principal investigators (PIs).
For the two intervention feasibility study sites, three self-selected POPPI nurses were identified locally to participate in the education package and to support and deliver the POPPI intervention, using the following criteria:
-
expert practitioner in critical care
-
excellent communicator with good interpersonal skills
-
interested in improving critical care patients’ psychological outcomes
-
willing and able to support the rest of the team in delivery of all elements of the POPPI intervention
-
available to attend a 3-day training course and 1 day for feedback and assessment.
Within both feasibility studies, all sites obtained local research and development (R&D) approval and were visited by the trial team for a site initiation visit, prior to the commencement of patient screening and recruitment. Regular contact was kept between the trial team and the local research teams throughout the recruitment periods for each study. In addition, in the intervention feasibility study, the expert trainers (health psychologist and senior nurses) provided the POPPI nurses, both in person and over the telephone, ongoing one-to-one debriefing and support on the stress support sessions.
Patients
Identical screening procedures were used in both feasibility studies. On admission to the critical care unit, all patients were added to a screening and enrolment log. Once a patient had stayed 48 hours in the unit, they were screened by the local research team for the following criteria:
-
aged ≥ 18 years
-
received 48 hours of level 2 or 3 care
-
English speaking
-
no pre-existing –
-
chronic cognitive impairment (e.g. dementia)
-
psychotic illnesses
-
chronic PTSD
-
-
not previously recruited into the intervention feasibility study/RCT procedures feasibility study (as relevant).
If the patient met all of the above criteria, daily screening of the following criteria commenced:
-
current Richmond Agitation–Sedation Scale (RASS) score of between + 1 and −1
-
current Glasgow Coma Scale score of 15
-
not terminally ill or receiving end-of-life care
-
currently able to communicate orally
-
able to give informed consent [e.g. not deemed delirious by the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU)].
If any of the daily screening criteria were not met, the patient would be rescreened each day until either fully meeting the criteria or being discharged from the critical care unit. Once the patient met all the daily screening criteria, they would be approached for informed consent in the unit. The final screening status of the patient, including reasons why patients were not recruited, were recorded on the screening and enrolment log.
To maintain the profile of the feasibility studies, posters were displayed in staff areas and at other relevant locations within the units and pocket cards, which summarised the eligibility criteria, were distributed to unit staff.
The informed consent process included provision of information, both written [in the form of the prepared information leaflet and patient information sheet (PIS)] and verbally in conversation with authorised members of the local research team. Eligible patients were encouraged to ask questions and given the opportunity to discuss the study with both family and/or friends. After the local research team member was satisfied that the patient had read and understood the information, they were invited to sign the consent form.
Delivery of the POPPI intervention: intervention feasibility study
At intervention feasibility study sites, consented patients were assessed using the IPAT. If the patient was assessed as acutely stressed (scoring ≥ 7 points), they were offered three stress support sessions, which were to be delivered by the specially trained POPPI nurse, alongside the relaxation and recovery programme.
Feasibility assessment
To deliver the education package
Assessment of feasibility to deliver all elements of the education package encompassed the following activities.
POPPI online training
The POPPI online training was evaluated by critical care staff members completing the course as follows:
-
an assessment of key learning (knowledge assessment)
-
feedback on the course.
Three-day training course
The 3-day course on the intervention was evaluated by each of the six POPPI nurses (three nurses across two sites) as follows:
-
a pre and post assessment of self-efficacy
-
self-assessment of key learning objectives (knowledge assessment)
-
feedback on the course and course materials.
One day for feedback and assessment
The 1 day for feedback and assessment on delivering the intervention was evaluated by the six POPPI nurses and expert trainers as follows:
-
an assessment of self-efficacy
-
an expert trainer assessment of efficacy (skills assessment)
-
a focus group discussion.
To deliver the POPPI intervention
Assessment of feasibility to deliver the three elements of the intervention encompassed the following activities.
Creating a therapeutic environment in critical care
Feasibility to deliver the first element of the intervention (creating a therapeutic environment in critical care) via the POPPI online training was assessed as follows:
-
the proportion of critical care unit staff members undertaking the course
-
the proportion of staff completing and passing the end-of-course assessment.
Three stress support sessions for patients identified as acutely stressed
Feasibility to deliver the second element of the intervention (three stress support sessions for patients identified as acutely stressed) was assessed as follows:
-
the proportion of eligible and consenting patients assessed with the IPAT
-
the number of stress support sessions delivered
-
the timing of delivery
-
the duration of each stress support session
-
feedback from the one-to-one debriefing and support sessions with the expert trainers.
In addition, the acceptability of the three stress support sessions was measured as follows:
-
by patient-rated stress thermometer scores (collected before and after every stress support session); and
-
by patient-completed satisfaction questionnaires (for those receiving at least two stress support sessions and recruited after the one day for feedback and assessment).
Relaxation and recovery programme for patients identified as acutely stressed
Feasibility to deliver the third element of the intervention [the relaxation and recovery programme (for use on tablet computer by patients between stress support sessions and on DVD at home after discharge with patient booklet)] was assessed by the POPPI nurse and patient feedback.
Final focus group
At the end of the intervention feasibility study, two focus groups were held with the POPPI nurses at each site to discuss their experiences.
Patient follow-up: randomised clinical trial procedures feasibility study
At RCT procedures feasibility study sites, consented patients were sent a follow-up questionnaire at 5 months post recruitment by the Intensive Care National Audit & Research Centre (ICNARC) Clinical Trials Unit (CTU) via its indicated, preferred route (via post or e-mail – indicated at consent). (Note that the intervention was not delivered in this study.) The questionnaire pack included the proposed instruments to be used in the cluster RCT: the EuroQol-5 Dimensions, five-level version (EQ-5D-5L);56 the PTSD Symptom Scale – Self-Report questionnaire (PSS-SR);57 the Center for Epidemiologic Studies Depression Scale (short form) (CES-D-10);58 and a health services questionnaire (providing information for the proposed integrated economic evaluation in the cluster RCT). For questionnaire packs sent via post, a pen and a self-addressed, stamped envelope were included. After 2 weeks, non-responders were telephoned to check receipt and to provide an option of responding over the telephone.
Both the response rate and the completeness of the primary outcome measure, the PSS-SR, were assessed – the latter through detailed inspection for missing data across the 17 items that make up the scale.
Prior to follow-up commencing, the primary outcome questionnaire was changed from the Post-traumatic Diagnostic Scale to the PSS-SR because of issues regarding copyright and requirements for formatting. We also changed the feasibility follow-up time period from 1 to 5 months to allow for closer replication of the cluster RCT (follow-up at 6 months). This ensured that we would have increased confidence over the expected follow-up rates when planning the cluster RCT.
We conducted a small, methodological substudy to elicit whether or not the burden of questionnaires (i.e. number of questionnaires) influenced the response rate by randomly allocating patients to receive either a full questionnaire pack (i.e. PSS-SR, CES-D-10, EQ-5D-5L and health services questionnaire) or solely the single PSS-SR required for the primary outcome.
Results
Research governance
The joint protocol for the two feasibility studies was submitted to the National Research Ethics Service (NRES) Committee South Central B – Oxford (reference number 14/SC/0149) on 25 February 2014. Favourable ethics opinion was granted on 23 April 2014 following response to requested minor modifications. Two further minor modifications occurred in May and September 2014 (these related to the timing and questionnaires used as part of the patient follow-up in the RCT procedures pilot study). The feasibility studies were adopted onto the NIHR CRN portfolio (16479) and registered on the ISRCTN Registry (ISRCTN61088114).
Recruitment to the intervention feasibility study
Sites and POPPI nurses
The two adult, general critical care units (sites) recruited were University College Hospital, London (a teaching hospital) and Watford General Hospital (a district general hospital). Both sites identified a PI and three POPPI nurses. Soon after delivery of the 3-day training course for the POPPI nurses, R&D approval at both sites was achieved, site initiation visits occurred and site activation and the start of patient screening commenced. In total, the sites were screened for 5.5 months.
Patients
Of 1246 patients screened, 315 were deemed potentially eligible, of which 99 eligible patients consented to participate in the intervention feasibility study; the Consolidated Standards of Reporting Trials (CONSORT) flow diagram can be seen in Figure 3. All patients were approached for informed consent while still in the critical care unit. Two patients provided written consent after discharge from the unit.
FIGURE 3.
Intervention feasibility study CONSORT flow diagram.
Relative to the day of admission, the median number of days for gaining consent was 5 [interquartile range (IQR) 4–7 days]. With an inclusion criterion of ‘received ≥ 48 hours of level 2 or 3 care’, of the 99 consented patients, 45 (45.5%) were consented within the next 48 hours (i.e. within 96 hours of admission) and 78 (78.8%) were consented within their first week in the critical care unit.
Feasibility assessment
To deliver the education package
POPPI online training
All POPPI nurses completed the online training prior to or soon after the 3-day training course. Local POPPI teams enumerated all clinical critical care unit staff and provided e-mail addresses for each staff member to the POPPI trial team, as registration for the POPPI online training. On the day following the 3-day course, each staff member was sent an e-mail invitation to undertake the online training. Uptake was monitored and staff were regularly prompted in person and via e-mail about completion.
In total, 283 staff members across both sites completed the POPPI online training. Five key learning objectives were self-assessed and each explored how much knowledge the individual undertaking felt that they had acquired during the online training course and was rated on a five-point scale from ‘not at all’ to ‘a lot’.
Feedback on the POPPI online training was provided by 260 (91.9%) critical care unit staff members who completed it. Four aspects were rated on a five-point scale: (1) stimulating to boring, (2) useful to useless, (3) well designed to not well designed and (4) just right (in length) to too long. Those completing the feedback were also asked to indicate any preference for their most and least liked aspect about the POPPI online training.
The POPPI online training was rated as stimulating/very stimulating by 72.7%, useful/very useful by 85.8%, well designed by 84.0% and just right (in length) by 80.0%. With regard to preferences about aspects of the course, factual information was the most liked aspect of the course (41.8% rated this as their most liked aspect), followed by videos from patients (34.8%), videos of staff–patient interactions (12.5%) and quizzes/assessments (10.2%). With regard to the least liked aspect of the course, most (69.3%) indicated there was no aspect that they least liked.
Overall, comments in response to open questions around content were positive and suggestions for improvement included ideas for additional content; additional clarification around the part dealing with how to assess patients with the IPAT; a clearer description of all the elements of the POPPI intervention and, specifically, the stress support sessions to be delivered by the POPPI nurses; clarity on where to access post-discharge information available for patients; graded, tougher assessments at the end of the online training course; and promotion of wider and repeated teaching opportunities (in the form of seminars and encouraging repeated use of the POPPI online training). Feedback was also provided on technical issues, emphasising the need to make access to the course as simple as possible and the absence of sound cards/speakers in NHS computers making the videos redundant and thus frustrating users (the use of subtitles or transcripts was suggested).
Other feedback provided to the POPPI trial team included the need for key messages, the need for key messages relevant to practice, a preference for practical rather than theoretical knowledge, the need for autogeneration of a certification following course completion and the need for improved design (with more colours, more graphics and larger but less text).
Three-day training course
All six POPPI nurses attended the 3-day course on the intervention. The 3-day training course was held from 20 to 22 October 2014 in central London. At the 3-day course, each POPPI nurse received training materials in both paper and electronic [on a USB (Universal Serial Bus) stick] format. At the end of the course, each POPPI nurse returned to their site with a tablet computer containing the relaxation and recovery programme for use by patients.
Pre and post assessment of self-efficacy was completed by all six POPPI nurses. Seven aspects were assessed; each explored how confident the POPPI nurse felt about each aspect and was rated on a five-point scale from ‘not at all confident’ to ‘very confident’. Although the sample was small, improvement in self-efficacy, in terms of increased confidence, was seen across six of the seven aspects assessed.
Self-assessment of key learning objectives was completed by all six POPPI nurses. Eight key learning objectives were assessed; each explored how much knowledge the POPPI nurse felt that they had acquired during the 3-day training course and was rated on a five-point scale from ‘nothing’ to ‘a lot’. Although the sample was small, acquisition of knowledge for all eight key learning objectives was indicated.
Feedback on the course was provided by all six POPPI nurses. Five aspects were rated on a five-point scale from stimulating to boring; useful to useless; relevant to irrelevant; well conducted to poorly conducted; and motivating to not motivating. All six POPPI nurses rated the 3-day training course as stimulating/very stimulating, useful/very useful, relevant/very relevant, well conducted/very well conducted and motivating/very motivating.
In response to open questions, positive feedback on the course included involvement of ex-patients (hearing their and other patients’ stories), incorporating practical sessions with one-to-one role plays, the openness in discussing shared experiences and knowledge, relevant course materials and the training team’s knowledge of theory and techniques. Suggestions for improvement included the challenge of role playing, the potential to use actors (rather than each other) in role plays, the challenge of the course content and materials (greater insistence on pre-course preparation was suggested), and some refocusing of course timings to the more challenging elements.
One day for feedback and assessment
All six POPPI nurses attended the 1 day for feedback and assessment, held on 13 January 2015 (for POPPI nurses from Watford General Hospital) and 15 January 2015 (for POPPI nurses from University College Hospital, London) in central London.
Assessments of self-efficacy were completed by all six POPPI nurses. As before, for the 3-day training course, seven aspects were assessed, exploring how confident the POPPI nurse felt about each, and were rated on a five-point scale from ‘not at all confident’ to ‘very confident’. For all six POPPI nurses, levels of confidence in each aspect were either sustained or improved.
Two expert trainers rated each of the six POPPI nurses conducting the second of the three stress support sessions (using an actor and conducted in a private room). Five of the six nurses passed their skills development assessment. During the assessment, feedback and support were provided. Feedback provided to the POPPI nurses was around continuous revision (through continuing use of the POPPI nurse training manual), the mode of delivery (less hesitancy) and the timing of delivery (increasing the time taken). Support from the expert trainers reinforced the value of practice and their confidence in their newly acquired skills.
One POPPI nurse, who did not pass on the day (predominantly as a result of nervousness around the notion of assessment) was referred for further training and support from the expert trainers. This occurred on-site and the POPPI nurse passed on a subsequent visit from the expert trainers on 22 January 2015.
At the 1-day focus group, reflections on the 3-day training course indicated that earlier engagement with the theory would have helped and would have made the course less hard and tiring. The POPPI nurses indicated that, during skills practice, they preferred the approach of dividing each stress support session into essential steps and then mastering each of these, in turn, prior to putting them all together to deliver the complete session. Demonstrations (by the expert trainers at the course or by videos of expert trainers) and the use of actors in role plays were suggested improvements. Clarity was requested over when to approach patients in the critical care unit, namely that the 48-hour mark was just the point at which to begin daily screening and that determining when a patient was ‘ready’ could be difficult. They said the consensus was that patients welcomed the opportunity to talk. One suggested approach was to leave the information leaflet on the first visit. All found the IPAT straightforward to use. All POPPI nurses said that training in delivery of the stress support sessions had made them much more aware of what patients might be experiencing and made it easier to start conversations with patients. It was felt that methods to achieve greater awareness by other staff of the POPPI study/POPPI nurse role, particularly relating to the stress support sessions, would help. The POPPI nurses felt that engagement with the study by other key staff in the unit (those at a senior level and those with responsibility for research, etc.) would be key to successful delivery of the POPPI cluster RCT. Issues raised about the tablet computer and DVD were around the music playing continuously (rather than as a track at a time) and directing patients to patients’ stories that were relevant to them. Regular teleconferences on wider study progress with other key staff, as well as individual debriefing and support sessions for the POPPI nurses, were seen as important. The POPPI nurses felt that the content of the online training course needed to be reinforced by others (team leaders, educators, etc.) in the unit. It was felt that creation of the therapeutic environment should be done by others, leaving more time for the POPPI nurses to deliver the stress support sessions.
To deliver the POPPI intervention
Creating a therapeutic environment in critical care
Uptake of the POPPI online training on creating a therapeutic environment was monitored closely. Overall, uptake for the 338 enumerated staff members across the two critical care units was 283 (84%). By job role, uptake was 92% for doctors, 83% for nurses and 76% for other allied health professionals. Overall uptake for each of the two critical care units was 81% and 87%, with uptake by doctors, nurses and allied health professionals following a similar pattern in both units.
Of those staff members taking the POPPI online training, 280 (98.9%) attempted the six-question assessment at the end of the course. Of these, 277 (97.9%) passed the assessment (achieving a score of ≥ 80%) and 3 (1.1%) failed the assessment and opted not to retake it.
Of those staff members passing the assessment, 128 (45.2%) passed first time and 138 (48.8%) passed second time. For 11 (3.9%) staff members, three or more attempts were required to pass. All staff members who passed the assessment received a certificate.
Three stress support sessions for patients identified as acutely stressed
All 99 (100%) eligible, consented and recruited patients were screened for acute stress with the IPAT. Of these, 40 (40.4%) were assessed as acutely stressed and being at a high risk of psychological morbidity (IPAT score of ≥ 7 points). The median score for acutely stressed patients was 10 (IQR 9–12). The median score for all patients was 5 (IQR 3–9) and for low stress patients was 3 (IQR 2–3).
For patients identified as acutely stressed, the median length of stay from consent to critical care unit discharge was 1 (IQR 0–4) day and to hospital discharge was 10 (IQR 6–26) days.
Of the 40 eligible patients identified as acutely stressed, 18 (45.0%) received all three stress support sessions, 12 (30.0%) received two, four (10.0%) received one and six (15.0%) received none of the stress support sessions (Figure 4).
FIGURE 4.
Number of stress support sessions received by patients (n = 40). SSS, stress support session.
Of the six patients who received no stress support session, one declined and one was discharged from hospital prior to delivery. The remaining four participants received no session(s) because of unavailability of a POPPI nurse to deliver the session(s) (see Site issues and resolution).
Of the four patients who received one stress support session, the second session was scheduled for two of them, but then postponed because of the patient’s conditions (one patient was too tired and was then discharged from hospital before the second session was rescheduled and the other participant deteriorated clinically and was no longer eligible). Of the other two patients, one was discharged from hospital before the second session occurred, and the second did not receive the second session, because they were busy when approached at the scheduled time. Later, the POPPI nurse was unavailable to deliver the session (see Site issues and resolution).
Of the 12 patients who received two stress support sessions, five were discharged from hospital before the third session could be conducted; two declined; three patients received the second and third sessions combined, in anticipation of hospital discharge; and two did not receive the third session because of unavailability of a POPPI nurse (see Site issues and resolution). Five of the 12 patients were provided with the relaxation and recovery DVD and the Getting Well, Staying Well booklet, due to be given during session three, prior to leaving the hospital.
Relative to the day of admission, the median day for gaining consent was 5 (IQR 4–7) and the median day for delivery of the first stress support session was 9 (IQR 7–12). Of the 34 patients who received at least one stress support session, the time from consent to first session was ≤ 2 days for 20 patients (58.8%) and, cumulatively, ≤ 4 days for 27 patients (79.4%). The first stress support session was delivered in the critical care unit for 15 (45.5%) of the patients. Of the 18 patients who received all three stress support sessions, 13 (72.2%) received them within 1 week, as promoted by the POPPI study. All stress support sessions were delivered to patients by the same trained POPPI nurse (i.e. one of the three POPPI nurses assigned to each patient).
The median duration for each stress support session was 35 (IQR 33–40) minutes for the first session, 30 (IQR 25–35) minutes for the second and 30 (IQR 30–40) minutes for the third session.
Ongoing debriefing and support indicated that the POPPI nurses found delivery of the stress support sessions rewarding and challenging. Issues requiring resolution arose around engaging patients, ensuring that patients understood what the sessions did (and did not) encompass, identifying when to refer patients on for further support and knowing how to handle patients revealing very serious matters unrelated to the critical care unit stay. At each debriefing and support session with POPPI nurses, the expert trainers encouraged the value of ongoing revision, reinforced the importance of seeking support from the expert trainers, acknowledged the particular challenge of delivering the content of the second stress support session, stressed the value of listening to the patient and confirmed the importance of sharing concerns about the patient with their clinical team.
It became clear that, as time progressed, the POPPI nurses at one of the two sites were unable to commit the necessary time to the intervention feasibility study for a number of reasons: one reason was personal (compassionate leave), one reason was professional (attendance/involvement in another course/activity, confounded by the unavoidable delay in commencing the intervention feasibility study) and one nurse never really engaged. This clearly affected their availability for consent (see Figure 3) (patients were not consented if stress support sessions could not be offered) and ability to deliver the stress support sessions (see Figure 4).
To address this, the intervention feasibility study was put on hold at one site while four new POPPI nurses were rapidly identified and trained (i.e. attending the 3-day training course and the 1 day for feedback and assessment). The results for their pre and post assessment of self-efficacy and self-assessment of key learning objectives are included, along with those of the initial six nurses, in Table 2.
| Element of intervention | Content/delivery of element | Feasibility and acceptability indicators: quantitative and qualitative | Feasibility and acceptability results | Refinement of the intervention post-feasibility study |
|---|---|---|---|---|
| Element one: creating a therapeutic environment in critical care | Content of online training course | Training course ratings by all staff [percentage with score of 4 or 5 (range 0–5) or ‘good’] | Stimulating, 73%; useful, 86%; well designed, 84%; right length, 80% (n = 260, but missing data for some items) | Online training course shortened and made more visually appealing, with more practical advice on reducing stressors in critical care units and clearer presentation of key messages |
| Favourite parts of course: all staff | Factual information, 42%; patient stories, 35%; communication videos, 13%; tests, 10% (n = 260) | |||
| Nurse qualitative feedback | Staff were positive and suggested minor improvements | |||
| Delivery of online training and creating a therapeutic environment | Staff taking course (target: 80%) | n = 283 (84%) | Provision of training, display materials and slide-sets for seminars/workshops for local education teams to support and motivate staff in creating a therapeutic environment | |
| Staff passing final test (i.e. score of > 80%) | n = 277 (98%) | |||
| Staff learning scores [percentage with a score of 4 or 5 (range 0–5) or ‘good’] | 74% (n = 259) | |||
| Nurse qualitative feedback | POPPI nurses lacked time, because of workload, to support staff in creating therapeutic environment | |||
| Element two: three stress support sessions for patients identified as acutely stressed | Content of screening | Previously validated52 | ||
| Delivery of screening | Consenting patients screened | n = 127 (100%) | ||
| Identified as acutely stressed | n = 51 (40%) | |||
| Content of stress support sessions | Median (IQR) difference in patient stress thermometer scores (range 0–10) from the start of session 1 to the end of session 3 | −3 (−5 to −1) (n = 25 patients who had all three sessions) | Content of stress support sessions clarified for POPPI nurses and patients by reorganising sessions from five components each to three common components in all sessions and three individual components per session. Manual became more tightly focused on stress support sessions (rather than the whole intervention), with clearer signposting to and between sections | |
| Patient satisfaction with stress support sessions [percentage with score of 4 or 5 (range 0–5) or ‘good’] | Overall 93%; helped express fears 93%; nurse understanding 100%; nurse normalised fears 100%; fewer stressful thoughts 87%; fewer stressful feelings 80%; number/duration of sessions 80% (n = 15, missing data some items) | |||
| Patient qualitative feedback | Stress support from nurses was very helpful | |||
| Nurse qualitative feedback | Rewarding, but challenging to explain to patients | |||
| Delivery of stress support sessions | Number of stress support sessions patients had | 25 patients (49%) had three sessions; 14 (28%) had two sessions; five (10%) had one session; and seven (14%) had none |
|
|
| Median duration of sessions (minutes) |
|
|||
| Nurse qualitative feedback | Hard to fit sessions in with ordinary duties, especially if patients postponed. Patients missed session 3 if they were discharged to home early | |||
| Content of POPPI nurse face-to-face training course (3 days and a feedback/assessment day) | Nurse feedback: post-course questionnaire [percentage with score of 4 or 5 (range 0–5) or ‘good’] | Stimulating, 100%; useful, 100%; relevant, 90%; well conducted, 100%; motivating, 100% (n = 10) |
|
|
| Nurse self-efficacy (in delivering psychological support): pre/post course and feedback-day questionnaires [percentage with score of 4 or 5 (range 0–5) or ‘good’] | Big increase in self-efficacy from pre to post course; maintained at follow-up. Across items, 30% of scores (21/70) were good pre course; 73% were good scores (51/70) post course and on feedback day | |||
| Nurse course learning: an eight-item post-course knowledge questionnaire [percentage with score of 4 or 5 (range 0–5) or ‘good’] | 87% of scores (69/79) were ‘good’ for learning on acute stress; screening; aims of stress support sessions, normalising, psychoeducation, communication style, stressful thinking, checking out fears and coping | |||
| Trainer assessment of nurse competence using six-item checklist (score range 0–12; pass = 6) |
100% passed (9 on first attempt, 1 on second) Median score of passes was 9 (IQR 9–10) |
|||
| Nurse qualitative feedback | Course highly valued but tiring. Skills practice stressful. Competence assessment on follow-up day stressful | |||
| Delivery of 3-day training course | Number of required trainees attending | 10 (100%)a | Some modules dropped from the course; pre-course booklet on psychological principles provided | |
| Content: debriefing and support by trainers | Nurse qualitative feedback |
|
Assessment to be one-to-one confirmation of skills, as part of ongoing debriefing and support | |
| Delivery: debriefing and support | Nurse qualitative feedback | Nurse debriefing and support should start earlier | First debriefing call after the first POPPI patient | |
| Element three: relaxation and recovery programme for patients identified as acutely stressed | Content of relaxation and recovery programme | Patient satisfaction with content of programme [percentage with score of 4 or 5 (range 0–5) or ‘good’] | Content on tablet computer app, 71%; useful post-ICU coping ideas, 67% (15 patients, some missing data) |
Content and design of the relaxation app were improved Balance of contents of DVD improved, and calming classical music tracks added Layout and readability of the patient booklet improved |
| Nurse-reported qualitative patient feedback |
|
|||
| Delivery of relaxation and recovery programme | Patients receiving tablet in session 1 | 40 (90%) |
|
|
| Patients receiving DVD or booklet | 27 (61%) | |||
| Nurse-reported qualitative patient feedback | Some liked the tablet, others found it hard to use, some preferred the DVD or the patient booklet |
All four nurses passed their skills development assessment by two expert trainers. The feasibility study recommenced, and a further 28 eligible patients were recruited prior to the feasibility study end. Of these 28 eligible patients who consented to participate, 27 (96%) consented while still in the critical care unit. Relative to the day of admission, the median day for gaining consent was 5 (IQR 4–7). With an inclusion criterion of ‘received ≥ 48 hours of level 3 or 2 care’, of the 28, 12 (42.9%) were consented within the next 48 hours (i.e. within 96 hours of admission) and 22 (78.6%) were consented within their first week in the critical care unit.
All 28 (100%) eligible patients were screened for acute stress with the IPAT. Of these, 11 (39.3%) were identified as acutely stressed (IPAT score of ≥ 7 points). The median IPAT score for these patients was 9 (IQR 8–13). For these 11 patients, the median stay in the critical care unit from consent to unit discharge was 1 (IQR 1–2) day and to hospital discharge was 10 (IQR 5–20) days.
Of the 11 eligible patients assessed as being acutely stressed, seven (64%) received all three stress support sessions, two (18%) received two stress support sessions, one (9%) received one stress support session and one (9%) received none.
The one patient who received no stress support sessions withdrew consent. The one patient receiving only one session became confused and agitated and future sessions were deemed inappropriate. The two patients receiving two sessions were discharged from hospital before the third session could be conducted.
Of the 10 patients receiving the first stress support session, the time from consent to the first session was ≤ 2 days for all. For five patients (50%), the first session was delivered in the critical care unit. Relative to the day of admission, the median day for gaining consent was 5 (IQR 4–7) and the median day for delivery of the first stress support session was 11 (IQR 9–12). For the seven patients who received all three stress support sessions, all were delivered in 1 week by the same POPPI nurse, as promoted by the POPPI study.
The median duration for each stress support session was 30 (IQR 24–30) minutes for the first session, 30 (IQR 30–35) minutes for the second session and 25 (IQR 25–30) minutes for the third session.
Further feasibility indicators, based on the combined data for the 51 patients identified as acutely stressed and the 10 trained POPPI nurses, are presented in Table 2.
Patient acceptability of the three one-to-one stress support sessions are based on the combined data for the 51 patients identified as acutely stressed in the intervention feasibility study.
Stress thermometer scores were collected at the beginning and end of each stress support session. Stress thermometer scores ranged from 10 (very stressed) to 0 (very calm). The mean scores at the start and end of each stress support session were:
-
stress support session 1 – start, 5.4 [standard deviation (SD) 2.6]; end, 5.0 (SD 2.7)
-
stress support session 2 – start, 4.7 (SD 2.8); end, 3.6 (SD 2.4)
-
stress support session 3 – start, 3.5 (SD 2.7); end, 2.7 (SD 2.7).
On average, each stress support session resulted in a small negative mean difference, –0.4 (SD 1.7), –1.1 (SD 1.8) and –0.8 (SD 1.4), respectively, indicating a small reduction in immediate acute stress. In addition, patients’ stress thermometer-rated acute stress also showed a negative mean difference [–2.6 (SD 3.3)] from the start of the first stress support session to the end of the third session for patients receiving all three sessions.
Following the 1-day course for feedback and assessment, when possible, patients recruited and receiving at least two stress support sessions were asked, independently of the POPPI nurses, to rate their satisfaction with the stress support sessions. Ten aspects were assessed, with seven rated on a five-point scale from ‘strongly agree’ to ‘strongly disagree’ and three rated on three different five-point scales: from ‘definitely too few’ to ‘definitely too many’, from ‘much too short’ to much too long’ and from ‘not at all useful’ to ‘very useful’.
Of 23 patients receiving at least two stress support sessions, 15 (65.2%) were given satisfaction questionnaires prior to hospital discharge and all 15 (100%) completed them (Figure 5).
FIGURE 5.
Acceptability of the stress support sessions by patients (n = 15).
In general, patients reported satisfaction with the stress support sessions and the majority felt that the number and duration were about right (with a minority preferring a greater duration) and, overall, that the support provided was useful. Some patients reported that the tablet computer was not easy to use.
A free-text box on the patient satisfaction questionnaires elicited comments:
Every hospital should have it.
It seemed a life-saver and really helpful to know the organisation is treating the subject so strongly.
I am glad to hear that notice has been taken of the many vital warning alarms in ICU [intensive care unit] . . . which increase patient stress.
I hope that POPPI will eventually be used in all ICUs [intensive care units]. Thank you to xxx for the support.
I feel the programme is such a good idea.
Stressful not knowing what’s going on.
Too many people coming in and out.
Comments on the intervention:
Really enjoyed the sessions.
. . . proves that I am normal but under the influence of morphine!
xxx was excellent and had explanations for what seemed a very dark world.
I have been given strategies that I will use in everyday life.
I had no idea how stressed I was feeling until I started talking to xxx about my experience.
Nurse was very good – felt less stressed about leaving hospital than last time.
Found the tape/music very relaxing.
Tablet computer difficult to use – didn’t always pick up commands.
Could have done with one more session.
Relaxation and recovery programme
The POPPI nurses reported variation in the use of the elements of the relaxation and recovery programme by patients. For example, some patients related best to the patient stories, some to the relaxation exercises, some to the meditation and some to the music. Some patients were apprehensive about using the tablet computer and required both encouragement and support in its use. Despite this support, some found it confusing/difficult to use, owing to a combination of unfamiliarity with the technology and reduced dexterity. Patients were pleased to be able to take home the DVD with the same relaxation and recovery programme for their ongoing use.
Final focus group
Following recruitment of the last patients, two final focus groups were held at the sites: one on 5 May 2015 with the POPPI nurses from University College Hospital, London, and the other on 8 May 2015 with the POPPI nurses from Watford General Hospital. The focus groups discussed the experiences of the POPPI nurses delivering the intervention.
The POPPI nurse role
All POPPI nurses indicated that they found the role enjoyable and beneficial (personally) yet also challenging. They indicated that it improved their communication skills and that they got a real boost (‘blown away by’) from delivering a session. Their main challenges were fitting the role around their clinical demands and inflexible rotas. As improvements, they suggested that better information on their role should be provided upfront, that the theory on psychological techniques should be provided ahead of the 3-day training course, that the concept of a ‘POPPI shift’ should be included into rotas and that, potentially, one POPPI nurse could be identified as the main/lead POPPI nurse in a unit and ensure peer support.
Most indicated that they received the right amount of ongoing debriefing and support from the training team, and that it was useful for reflection (particularly on harder cases). They found that reassurance in these debriefing sessions led to greater confidence. They felt that debriefing was best done as soon as possible, with some nurses suggesting that the first call could be booked at the 3-day training course, but several nurses also agreeing that it was best conducted after three full stress support sessions had been delivered by the POPPI nurse. They agreed that such debriefing and support sessions were difficult to arrange around shifts and that the expert trainers and the POPPI nurses would need to be flexible to make these happen. They agreed that one-to-one sessions with an expert trainer were important, but also thought joint and peer debriefing and support would be helpful too.
Education package
Three-day training course
All POPPI nurses indicated that the 3-day training course was very useful and that they felt privileged to be part of it. They liked the involvement of former patients and preferred when the learning was incremental (i.e. showing a video but using stop/start to discuss detail, learning the content of the stress support sessions in steps before putting them all together, etc.). All found the course hard work and tiring, the notion of assessment stressful, and felt that the skills practice would be better if they involved actors. The provision of scripts and a glossary were welcomed.
All POPPI nurses felt that the POPPI nurse training manual provided at the 3-day training course was invaluable as a backup but felt that some reordering around stress support sessions with relevant appendices would be welcome.
One day for feedback and assessment
The use of an actor on this day was welcomed (and they were pleased that their feedback had been listened to). All nurses found the skills assessment daunting, in anticipation, but knew that it was important. All indicated that the more that the assessment could be seen as ongoing debriefing and support, in a safe environment, the better – as all were seeking greater reassurance.
Online training course
The POPPI nurses felt that the online training did raise awareness and encouraged staff to think differently. They liked the fact that it was short (30 minutes duration) and that certificates were received following completion. With regard to content, they felt that the impact of critical care on long-term outcomes should be emphasised even more, and that some ‘raw’ patient stories should be used to engage the course-taker.
Intervention
Creating a therapeutic environment in critical care
The challenges of this element of the intervention included motivating others to do the online training, relieving staff to enable them to complete the online training, the difficulties of using Internet Explorer, version 6 (Microsoft Corporation, Redmond, WA, USA), and the absence of sound cards in NHS computers. Suggestions to improve uptake of the online training were that it should be incorporated into team study days (possibly mandated) and be available as a mobile phone app. For those who were less accustomed to using computers or electronic devices, training could be provided in a booklet or some form of buddying could be put in place.
With regard to microteaching, the POPPI nurses indicated that this had been very informal and that it should be more formalised and include involvement from others responsible for education within the unit.
Routine screening for acute stress with the Intensive Care Psychological Assessment Tool
Whether done by the POPPI nurses or by the research team, most felt that some dedicated training time would be good (and possibly to a wider group than just the POPPI nurses). Some felt that the IPAT questions on hallucinations, delusions and intrusive memories were more difficult to pose and that these should be covered more fully in the online training course, and in any dedicated training.
Three stress support sessions for patients identified as acutely stressed
All POPPI nurses indicated that the vast majority of patients seemed grateful for the stress support session(s) and enjoyed someone sitting and talking with them. All indicated that the language/approach required for the stress support sessions was very different from being a critical care nurse in clinical practice. It was hard to adjust to the practice of getting a patient to help themselves rather than them doing everything for the patient. Nevertheless, they appreciated the focus on patient empowerment. All complained that time, rotas and annual leave were a challenge to timely delivery and that delivery was more stressful when they were busy. Delivering the sessions was more stressful at the start because of their lack of confidence with the seemingly alien clinical approach. Suggested improvements included directing more focus at the second session, seen as the most challenging, and indicating during teaching that it was acceptable to deviate from the script using one’s own language. Two practical improvements noted were, first, the POPPI nurse should check the IPAT individual item scores (not just the overall score) and, second, in the situation in which hospital discharge was clearly close, that the second and third stress support sessions could, and should, be delivered together.
Relaxation and recovery programme for patients identified as acutely stressed
The POPPI nurses indicated that the tablet computers were used less than they imagined they would be. However, patients who did use them found them very beneficial – particularly if the selection of possible options (e.g. music, nature sounds, patient stories) was tailored to the individual patient by the POPPI nurse. All felt that wider knowledge by other hospital staff of the tablet computers’ existence would have increased use (e.g. bedside nurses could play a role in promoting their use). Challenges to their use appeared to be a result of dexterity (either because of age or current, post-intensive care health state). Some technological aspects (e.g. ease of increasing volume, size of icons, touch sensitivity) were also raised as challenges. The prospect of providing some support materials in other formats was also raised.
Randomised clinical trial procedures feasibility study
Sites and patients
Two sites and two PIs were recruited at the adult, general critical care units at Bristol Royal Infirmary (a teaching hospital) and at Medway Maritime Hospital (a district general hospital). Time from R&D approval to activation and start of screening took a total of 5 and 6 days at each site, respectively. Screening, consenting, recruitment and data collection commenced on 11 June 2014 at Bristol Royal Infirmary and 16 June 2014 at Medway Maritime Hospital, and both sites recruited patients for the required 2 months.
In total, 435 patients were screened between 11 June and 17 August 2014. Of these, 275 (63.2%) did not fulfil eligibility criteria or were deemed unable to give consent. Of the 160 eligible patients, 86 (53.8%) were consented (Figure 6).
FIGURE 6.
The RCT procedures feasibility study CONSORT flow diagram.
The estimated total patient recruitment was 44 patients, based on available data from the Case Mix Programme (CMP) – the national clinical audit for adult critical care co-ordinated by ICNARC – translating, on average, to 22 patients per site or 11 patients per month per site. The two sites recruited a total of 86 patients: 39 were recruited by the unit at Bristol Royal Infirmary and 47 by the unit at Medway Maritime Hospital (both operating as 19-bed, adult, general critical care units).
From the screening and enrolment logs, the reasons that eligible patients were not recruited were as follows:
-
declined (n = 21)
-
missed for recruitment (n = 48)
-
it was deemed that research/questionnaire burden was too much for patients (n = 5).
The median number of days from critical care unit admission to consent was 3 (IQR 3–5) and from critical care unit admission to hospital discharge was 11 (IQR 6–28).
Follow-up
Of the 86 recruited patients, nine (10.5%) died by 5 months. Of the 77 patients surviving to 5 months, 62 (80.5%) responded – 47 by post, eight by e-mail and seven by telephone. Nine (11.7%) patients declined to complete the questionnaire and six (7.8%) were lost to follow-up (Figure 7).
FIGURE 7.
Follow-up flow diagram.
In the small, methodological substudy to determine if the burden of questionnaires influenced the response rate, overall response was 81.6% (31/38) for the questionnaire pack and 79.5% (31/39) for the single questionnaire.
Overall completeness of the primary outcome measure, the PSS-SR, was very good. For a total of 1054 fields (62 responses with 17 items each), only 24 (2.3%) had missing data. We undertook a psychometric evaluation of the results (Table 3).
| Measure | Floor, n (%) | Ceiling, n (%) | Mean (SD) | Median (IQR) | Range | Cronbach’s alpha (95% CI) |
|---|---|---|---|---|---|---|
| PSS-SR total score | 16 (25.8) | 0 (0) | 6.1 (7.5) | 3 (0–8) | (0–30) | 0.90 (0.84 to 0.95) |
Refinements
Selection of the POPPI nurses
We decided it was important to identify four, rather than three, POPPI nurses (three plus one reserve) allowing for subsequent education of a further POPPI nurse, as required. Updated person specification and selection criteria for POPPI nurses (using feedback from the POPPI intervention feasibility study) would be provided to sites to ensure selection and engagement of the best-placed individuals.
Following feedback from the intervention feasibility study, it became evident that it was important to engage earlier with the POPPI nurses prior to the 3-day training course. At this point, we would provide the POPPI nurses with accessible learning materials on the theory and practice of essential psychological techniques underlying the intervention for the POPPI cluster RCT, particularly with respect to the stress support sessions, in an easily digestible form. Additionally prior to the course, the POPPI nurses would undertake the POPPI online training and familiarise themselves with their POPPI nurse training manuals, the DVD with the relaxation and recovery programme and other teaching/trial materials supplied for local, bedside teaching and support. However, all materials would be used only after the 3-day training course. The POPPI trial team would engage those responsible for research and for education within the sites to support the POPPI nurses by identifying and enhancing opportunities for local teaching (e.g. bedside, seminars, workshops) and wider support of the POPPI cluster RCT.
Earlier engagement with the POPPI nurses and with those responsible for research and education within the sites was in response to feedback from the intervention feasibility study about the need for more background and preparation prior to the 3-day training course and the need for additional support for local bedside teaching and support within the unit for creating the therapeutic environment.
A site initiation meeting would also be used to engage the wider team prior to the intervention period. The POPPI trial team would use this visit to each of the intervention sites to cover the activities required of the site as they transition into the intervention period. This would be presented both to the POPPI nurses and to the local research and education teams (including the site PIs – one senior nurse and one senior doctor from the unit). This site initiation meeting would also cover delivery of creating a therapeutic environment in critical care (via the POPPI online training for all critical care unit staff, supplemented by other teaching and educational activities) and introduce the concept of routine screening for acute stress with the IPAT.
The early engagement with the POPPI nurses and the site initiation meeting from the POPPI trial team (covering delivery of two of the elements of the intervention) would reduce the burden of content delivery on the 3-day training course – allowing the course to focus on the practical delivery of the three stress support sessions.
Refinements to the education package
POPPI online training
Technical issues around problems balancing ease of access with security of access (i.e. only to staff at intervention group sites) were resolved early in the intervention feasibility study. The POPPI online training was updated following feedback; updates included a greater emphasis on key messages – both overall and directly for practice (a shift of emphasis from theory to practice); techniques to make it more engaging (e.g. less text, more graphics); and additional content (e.g. on delirium, sources of information post discharge for patients). Reminders of key messages would be sent to those completing the course. We added subtitles and transcripts for audio on video files. The revised, final, online training course lasted 30 minutes and comprised the following five sections:
-
understanding the stresses of intensive care patients
-
reducing stress and fear in patients
-
communicating with distressed patients
-
inspiring patients with confidence and hope
-
summary and assessment.
Three-day training course
The 3-day training course was focused even more strongly on delivery of the three stress support sessions. Teaching of theory would be delivered through quick revision sessions (following earlier engagement and provision of pre-course information). The number of skills practice opportunities (i.e. role playing of stress support sessions) was increased and actors were employed to assist in these. Greater focus and time were given to the more challenging elements of the stress support sessions and skills practice, with individual steps in each stress support session being mastered first, with feedback, prior to practical delivery of a whole session. Case studies and exercises would now cover a wider spectrum of patient situations and include stress support sessions that do not necessarily go as planned. Other content was also revised based on feedback. The same expert trainers would deliver the training course, including the former critical care patients, whose contribution was very popular. Course documentation has been tailored to the newly focused, practical content of the course.
Between the 3-day training course and the 1 day for feedback and assessment, debriefing and support would be available to the POPPI nurses – initially after the first patient to whom they deliver stress support sessions, and then at regular intervals thereafter, as the transition period evolved into the full intervention period.
The refined, slimmed-down, programme for the 3-day training course was as follows.
-
Day 1:
-
Understanding critical care patients’ stress.
-
Learning the skills needed to deliver stress support sessions.
-
Overview of the stress support sessions.
-
Observing and practising session 1.
-
-
Day 2:
-
More about unusual experiences in critical care.
-
Learning further stress support skills.
-
Observing and practising stress support session 2.
-
-
Day 3:
-
Revision of key points from stress support session 2.
-
Observing and practising stress support session 3.
-
Using the patient booklet to create personal action plans.
-
Debriefing and support arrangements.
-
The three common components in every session were starting, building rapport and finishing. Starting a session included introductions, outlining aims, measuring stress and explaining confidentiality, note-taking and risk-reporting. Building rapport involved focusing throughout all the sessions on creating a relationship with the patient based on openness, honesty, warmth and listening. Finishing a session involved summarising, getting patient feedback, agreeing practice, remeasuring stress, agreeing wording for notes and arranging the next session.
We simplified the presentation of each session to consist of three individual components, rather than five, in the support tools. The individual components of the first stress support session were normalise reactions (discuss common psychological reactions and their causes in critical care), encourage communication (help patients open up about worries and concerns) and teach coping strategies (encourage patients to gather more information to address their concerns and use the relaxation programme) – acronym NET. The second session comprised stress reactions (encourage further talk about worries and fears, normalise concerns and notice stressful thoughts), explain stressful thinking (explain how unrealistic fears can create extra stress and identify one stressful thought that the patient agrees to work on) and teach ‘check out my fear’ (a technique for patients to find out if their fear is realistic or not) – acronym SET. The third session included summarise and review (key messages from sessions and persisting issues), action plan (create a personal plan to cope with challenges ahead, using the patient booklet and building on the stress sessions) and future expectations (encourage realistic optimism and hope about progress made and recovery) – acronym SAFe.
Documentation and support tools
All POPPI cluster RCT documentation and support tools for the education package were revisited. Specifically, the POPPI nurse training manual was split into an easy introduction to the theory (supplied in a booklet ahead of the 3-day training course) and a training manual focusing on practical delivery of the three stress support sessions, aligned to the 3-day training course. This resulted in improved linkage to appendices and greater focus on practical delivery in the training manual. The relaxation and recovery programme was improved and tablet computers with better touch sensitivity were identified for use and donated by Samsung Electronics (UK) Ltd (Chertsey, UK). If the materials were provided via DVD during stress support session 2, then more patients could take away the DVD regardless of whether or not they received the full three sessions. The Getting Well, Staying Well booklet will also be provided at the same time.
One day for feedback and assessment
The 1 day for feedback and assessment may be best delivered at sites, rather than centrally; feedback indicates that, if delivered locally, it will more likely to be seen by the POPPI nurses as a continuation of debriefing and support – while retaining an assessment of skills. In this way, assessment will become part of the debriefing and support process and assessment will be a one-to-one, face-to-face confirmation of skills session with the expert trainers. From experience, the feedback element of the day is best focused on local barriers (and solutions) to being a POPPI nurse, with feedback elicited in a focus group setting (with topic guides prepared from knowledge acquired in the POPPI intervention feasibility study).
Review of assumptions underlying the power calculation for the cluster randomised clinical trial
All assumptions underlying the sample size calculation and anticipated recruitment rate were reviewed in the light of the results of the two feasibility studies and the selection of sites for the POPPI cluster RCT (Table 4).
| Assumption | Source for revised assumption | ||
|---|---|---|---|
| Original | Revised | ||
| Number of potentially eligible patients (LOS of > 48 hours; level 3 care), harmonic mean per site per year | 240 | 255 | CMP data for actual sites |
| Number of potentially eligible patients (LOS of > 48 hours; level 3 care ), arithmetic mean per site per year | 306 | 306 | CMP data for actual sites |
| Potentially eligible patients who met trial eligibility and able to consent (%) | 60 | 55 |
|
| Of those, patients approached for consent (%) | 80 | 70 |
|
| Of those, patients recruited (%) | 90 | 75 | RCT procedures feasibility study: 74.8% |
| Number of participants recruited, harmonic mean per site per year | 103 | 72 | |
| Number of patients recruited, arithmetic mean per site per year | 132 | 87 | |
| Recruited patients who were alive at 6 months (%) | 90 | 90 | RCT procedures feasibility study: 89.5% |
| Of those, patients who complete follow-up (%) | 80 | 80 | RCT procedures feasibility study: 80.5% |
| Number of patients followed up, harmonic mean per site per year | 76 | 52 | |
| Number of patients followed up, arithmetic mean per site per year | 96 | 63 | |
| Number of patients followed up, harmonic mean per site per 5 months | 32 | 22 | |
| PSS-SR (mean) | 14 | 6 | RCT procedures feasibility study: 6.1 |
| PSS-SR (SD) | 12 | 7.5 | RCT procedures feasibility study: 7.5 |
| Between-site CoV | 0.5 | 0.5 | Conservative assumption |
| Between-site SD (mean PSS-SR × CV) | 7 | 3 | |
| ICC | 0.254 | 0.138 | |
| RCI for PSS-SR | 8 | 8.6 | Recalculated with observed data |
| Patients who were recruited but declined IPAT/stress support sessions during the intervention period (% of participants recruited) | – | 16 | Intervention feasibility study: 16.2% |
| Patients assessed as being at high risk during the intervention period (% of recruited participants who did not decline) | 50 | 40 | Intervention feasibility study: 40.2% |
| Minimal detectable difference (RCI × % that do not decline × % at high risk) | 4 | 2.9 | |
| Target power (%) | 90 | 90 | |
| Type I error rate (alpha) | 0.05 | 0.05 | |
| Anticipated actual power from proposed design (%) | 91.6 | 91.5 | |
| Total (anticipated) number of patients recruited | 2904 | 1914 | |
| Mean number of patients recruited per site per month | 11 | 7.25 | |
| Number of patients assessed using IPAT | |||
| Total | 792 | 438 | |
| Per site per month | 11 | 6.1 | |
| Number of patients receiving stress support sessions | |||
| Total | 396 | 175 | |
| Per site per month | 5.5 | 2.4 | |
Based on the most recent 12 months’ data from the CMP, the throughput of potentially eligible patients (i.e. with a length of stay in the critical care unit of ≥ 48 hours and receiving some level 3 care) for the 24 selected sites was extremely similar to the average values from all units in the CMP used in the original calculations. This enabled us to concentrate on those patients who would most likely have the potential to benefit from the intervention (i.e. those receiving level 3 care, rather than level 2/level 3).
Of these potentially eligible patients, the percentages who met trial criteria and were able to give informed consent (i.e. regained capacity and were free of delirium) were 61.2% and 48.2% in the RCT procedures feasibility study and intervention feasibility study, respectively, compared with an original assumption of 60%. Therefore, this assumption was lowered to a more conservative 55% (average of the rates from the two feasibility studies).
Of those that met trial eligibility and were able to consent, 71.4% and 73.2% were approached for consent in the two feasibility studies, respectively. We therefore lowered the assumption for this percentage from 80% to 70%.
Of those approached for consent in the RCT procedures feasibility study, 74.8% gave informed consent and were recruited to the study. We therefore lowered the assumption for the percentage of patients approached that will give informed consent from 90% to 75%. We did not use the consent rate from the intervention feasibility study to validate this assumption, as recruitment to the intervention feasibility study also required the patient to consent to assessment with the IPAT and, if identified as acutely stressed, receiving stress support sessions, which will not be the case for the cluster RCT.
The percentages of patients who were alive at 5 months and who returned a completed follow-up questionnaire in the RCT procedures feasibility study were extremely similar to the original assumptions (89.5% vs. 90% alive, 80.5% vs. 80% completing follow-up). We therefore retained the original assumptions. From the returned questionnaires, both the mean and the standard deviation (SD) of the PSS-SR were considerably lower than those assumed in the original sample size calculation (mean 6.1 vs. 14, SD 7.5 vs. 12). We therefore revised the assumptions based on these data, which we also used to recalculate the Reliable Change Index (RCI) for the PSS-SR.
Patients who declined assessment with the IPAT were excluded from the intervention feasibility study. However, in the cluster RCT, such patients would be recruited (provided they consented to complete the follow-up questionnaire), but they would have minimal opportunity to benefit from all aspects of the intervention. Using the screening and enrolment log from one of the two feasibility sites (with detailed reasons for non-recruitment available), we estimated that if these patients had been recruited to the study but declined the assessment with the IPAT and, if stressed, stress support sessions; this would correspond to 16.2% of recruited patients. No allowance for this was made in the original sample size calculation, but we have reduced the minimal detectable difference accordingly.
In the original sample size calculation, it was assumed that 50% of recruited patients would be screened and assessed as acutely stressed. In the intervention feasibility study, this figure was 40.2%. We therefore reduced the minimal detectable difference accordingly. Consequently, for the revised sample size calculation, we wished to be able to detect a minimal difference in the mean PSS-SR of 2.9 points.
Results of review of assumptions
Based on the revised assumptions above, and with the original proposed design of 12 intervention and 12 control sites recruiting for a 5-month baseline period, a 1-month transition and a 5-month intervention period, the anticipated power from the proposed design remained at 91.5% (exceeding the target power of 90%). However, the total anticipated number of patients recruited was lower at 1914 (7.25 per site per month), as were the anticipated numbers assessed using the IPAT and receiving stress support sessions.
As a consequence of this review, there was no need to change the proposed design of the POPPI cluster RCT.
Confirmation of the pre-trial power calculation
The power calculation was completed using the approach of Hussey and Hughes59 to achieve 90% power to detect a reduction from 6 points to 3.1 points (p < 0.05) in the mean PSS-SR score at 6 months, and was based on the following assumptions:
-
Mean (6) and SD (7.5) of the PSS-SR score taken from patients in the feasibility study.
-
Estimated intracluster correlation coefficient (ICC) of 0.138 – between-site coefficient of variation 0.5, corresponding to between-site SD of 3 (this is a conservative estimate as no multicentre data were available). 60 (The inclusion of a baseline recruitment period means that the sample size calculation is less sensitive to the degree of clustering. 59)
-
Treatment effect of a reduction of 2.9 points on the PSS-SR score based on a RCI for the PSS-SR of 8.6 points61 being observed in 40% of eligible patients in the intervention periods assessed as being at a high risk of psychological morbidity using the IPAT, with 16% of recruiting patients declining the intervention.
-
Harmonic mean of the number of patients completing follow-up (52 per site per annum – corresponding to 22 in a 5-month period) based on data from the CMP.
With the design and the above assumptions, the estimated total number of patients recruited (based on CMP data) for the cluster RCT would be 1914 patients from the 24 sites. It was anticipated that 438 patients would be assessed for acute stress, using the IPAT, of which 175 (40%) would be assessed as being at a high risk of psychological morbidity and receive the stress support sessions.
Conclusion
The feasibility to deliver the education package and the intervention, and the procedures for the cluster RCT, were successfully tested. The feasibility studies identified a number of issues that were translated into refinements and collated in Refinements and Review of assumptions underlying the power calculation for the cluster randomised clinical trial. These included the following areas: POPPI nurse selection, the education package, and the support tools and materials. We also used the additional knowledge and results to further inform the design of the cluster RCT, including reviewing the assumptions of the sample size calculation.
Chapter 4 Cluster randomised clinical trial methods
Introduction
Following completion of phase one of the POPPI study (standardisation and refinement of the POPPI intervention, and two successfully conducted feasibility studies), the POPPI intervention was ready for evaluation in a cluster RCT. 62 At the end of phase one, approval to proceed to phase two (the cluster RCT) was granted by the NIHR Health Services and Delivery Research (HSDR) programme. This chapter describes the cluster RCT methods.
Aims and objectives
The POPPI cluster RCT aimed to evaluate the clinical effectiveness and cost-effectiveness of the POPPI intervention in reducing the development of patient-reported PTSD symptom severity at 6 months post recruitment. The specific objectives were to:
-
evaluate the effect of the POPPI intervention on patient-reported PTSD symptom severity and other psychological outcomes and quality of life at 6 months
-
estimate, in an integrated economic evaluation, the cost-effectiveness of the POPPI intervention
-
assess, in an integrated process evaluation, the fidelity, dose and reach of the implementation of the POPPI intervention, and identify important contextual factors that affected its delivery.
Trial design
The POPPI cluster RCT was a multicentre, parallel-group cluster RCT, with a baseline (pre-intervention period) and staggered roll out of the intervention. Both an integrated process evaluation and an economic evaluation were embedded. The methods for the process evaluation and economic evaluation are described in Chapters 6 and 8, respectively.
The cluster RCT was nested in the CMP, the national clinical audit of adult, general critical care units in England, Wales and Northern Ireland, and co-ordinated by ICNARC. 63 Nesting the POPPI intervention in the CMP ensured an efficient design (with respect to participating units and data collection) and facilitated efficient management of the trial, including monitoring recruitment.
Setting
The setting for the cluster RCT was NHS adult, general critical care units in England, Wales and Northern Ireland.
Intervention
The POPPI intervention comprised three elements:
-
creating a therapeutic environment in critical care
-
three stress support sessions for patients identified as acutely stressed
-
a relaxation and recovery programme for patients identified as acutely stressed.
Each element was facilitated by an education package, including an online training course to train all staff in a unit to create a therapeutic environment, and a face-to-face, 3-day course to train self-selected critical care nurses (‘POPPI nurses’) to deliver the stress support sessions and the relaxation and recovery programme to patients identified as acutely stressed (see Chapters 2 and 3 for more details).
Participants
There were two levels of participation: site level and patient level. Patients were nested in sites.
Sites
The cluster RCT aimed to recruit a representative sample of 24 NHS adult, general critical care units across England, Wales and Northern Ireland. Adult, general critical care units were defined as intensive care units or combined intensive care/high-dependency units. Standalone high-dependency units and specialist critical care units (e.g. neurosciences, cardiothoracic) were not eligible for participation in the cluster RCT.
Patients
Patients admitted to participating critical care units were eligible according to the following criteria.
Inclusion criteria
-
Aged ≥ 18 years.
-
A duration of > 48 hours in the critical care unit.
-
Receipt of level 3 critical care (for any period of time) greater than 48 hours in the critical care unit.
-
A score of between + 1 and –1 on the RASS. 64
-
A Glasgow Coma Scale65 score of 15.
-
English-speaking.
-
Able to communicate orally.
Exclusion criteria
-
Pre-existing chronic cognitive impairment, such as dementia.
-
Pre-existing psychotic illness, such as schizophrenia.
-
Pre-existing PTSD.
-
Receiving end-of-life care.
-
Previously recruited to the POPPI study.
Outcomes
The primary clinical effectiveness outcome was the development of patient-reported PTSD symptom severity measured used the PSS-SR57 at 6 months. The primary cost-effectiveness outcomes were incremental costs [cost-effectiveness analysis (CEA)], quality-adjusted life-years (QALYs) and net monetary benefit at 6 months.
The secondary outcomes were:
-
days alive and free from sedation to day 30
-
duration of critical care unit stay
-
a PSS-SR score of > 18 points at 6 months66
-
anxiety and depression at six months, measured using the Hospital Anxiety and Depression Scale (HADS)67
-
health-related quality of life (HRQoL) at 6 months, measured using the EQ-5D-5L questionnaire. 56
Procedures: site level
Recruitment of sites
A call for expressions of interest was sent via e-mail to all adult, general critical care units actively participating in the CMP by the ICNARC CTU. In addition, advertisements were also placed on ICNARC’s website and Twitter feed (Twitter, Inc., San Francisco, CA, USA).
Sites were eligible to take part in the cluster RCT if able to commit to the following:
-
show that recruitment to target, timely data collection, and delivery of the POPPI intervention were feasible, via completion of a site feasibility questionnaire
-
dedicate adequate resources to carry out the POPPI intervention
-
agree to adhere to randomisation into either the control or the intervention group
-
identify two joint PIs to lead the POPPI intervention at the site (a lead nurse and a lead doctor)
-
agree, when possible, to recruit all eligible patients to the POPPI intervention and to maintain a screening and enrolment log to include reasons why eligible patients were not recruited
-
agree to use the CAM-ICU68 for assessing delirium and the RASS64 for assessing sedation status for the duration of the cluster RCT
-
continue active participation in the CMP.
Sites that took part in the POPPI intervention feasibility study (International Standard Randomised Controlled Trial Number ISRCTN61088114) were not eligible for selection for the cluster RCT. Stand-alone high-dependency units and specialist critical care units were excluded, along with critical care units that offered formal psychological support (diaries in the critical care unit were permitted as they were not deemed an early intervention).
Site initiation
Site initiation visits, facilitated by the trial manager (PRM), were held at each participating site prior to the commencement of patient screening. The purpose of these visits was to present the background/rationale of POPPI and to train the local teams in the trial procedures (e.g. screening, recruitment and data collection). Local PIs and research team members were required to attend the site visit. Procedures relating to delivery of the POPPI intervention were not discussed at these visits, given that these procedures would not be relevant to sites randomised to the control group.
Investigator site file
An investigator site file was provided to all participating sites at the site initiation visits. This contained all essential documents for the conduct of the trial and included the approved trial protocol, all relevant approvals (e.g. favourable ethics opinion letter), a signed copy of the clinical trial site agreement, the delegation log, copies of the approved PIS, patient consent forms and all standard operating procedures (e.g. for screening participants, for obtaining informed consent and for data collection). The site PIs and other delegated individuals were responsible for maintaining the investigator site file.
Randomisation
The 24 sites were randomly assigned to either the intervention group (n = 12) or the control group (n = 12) by the trial statistician at the ICNARC CTU, using a restricted randomisation approach to ensure balance across the groups in terms of geographical location, hospital teaching status and size of unit. 69 To allow for a staggered roll-out of the POPPI intervention, the 24 sites were allocated geographically into three groups of eight. Simulations of alternative ways to balance on size of unit were performed and compared:
-
balancing on teaching status and number of beds
-
balancing on teaching status and number of level 3 admissions
-
balancing on teaching status, number of beds and number of level 3 admissions.
The best combination on the above three factors (balance on teaching status and number of level 3 admissions) was used to perform the final random allocation.
For each group of eight sites, the individual sites were randomised (4 : 4) in their second month of recruitment (during the baseline period). It was necessary to randomise on a site (cluster) rather than individual level to avoid contamination of usual care, as it would not be possible to restrict parts of the POPPI intervention (e.g. the POPPI online training) to individual patients.
Sites randomised to the intervention group were then required to identify three self-selected POPPI nurses based on the following criteria:
-
registered nurse with at least 3 years critical care clinical experience
-
effective communicator with patients, families, colleagues and collaborators
-
able to work flexibly
-
interested in improving psychological care of patients
-
organised and able to manage a busy schedule.
The PIs were provided with the person specification, and decisions on POPPI nurse selection were made locally, as would be the case in usual practice.
Timeline
The 24 sites opened to recruitment in three groups of eight sites at 2-month intervals and recruited patients over a 17-month period (Figure 8). Control group sites delivered usual care for the duration of the recruitment period. Intervention group sites delivered usual care from months 1–5 (baseline period). Usual care was defined as patients receiving psychological support or treatment at the discretion of the treating clinician(s) following standard practice at their site (participating units did not offer formal psychological support to patients).
FIGURE 8.
The POPPI cluster RCT schedule. Reproduced from Wulff et al. 70 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. Includes minor additions and formatting changes to the original. Also reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
At month six, intervention group sites underwent a one-month transition period, during which the education package was rolled out and the intervention group sites transitioned from delivering usual care to delivering the POPPI intervention (Figure 9). The intervention was then delivered until the end of the recruitment period (intervention period).
FIGURE 9.
Intervention site timeline during the transition period. Reproduced from Richards-Belle et al. 62 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. Includes minor additions and formatting changes to the original.
Delivery of the POPPI intervention at the site level
Figure 9 shows the timeline for intervention group sites during the 1-month transition period.
Intervention site initiation visits
Prior to the start of the transition period, a second site initiation visit, facilitated by the trial manager (PRM) and lead clinical investigator (DW), was held locally at each intervention site. These visits were held with the local PIs, research team, the POPPI nurses and individuals responsible for education/training. At these visits, the procedures and logistics for delivery of the POPPI intervention were discussed. Training was provided on how to screen patients using the IPAT. 52 A particular focus was on identifying strategies for ensuring adherence to the intervention, including plans for ensuring that all clinical critical care staff completed the POPPI online training course and that the POPPI nurses were given sufficient time to deliver stress support sessions with patients. Changes to the recruitment and data collection procedures were also outlined.
Ahead of the 3-day training course for the POPPI nurses, the nurses were given a short pre-course theory booklet (entitled POPPI Nurse: Essential Psychological Theory and Techniques) to read before attending the course. This booklet contained an overview of the POPPI nurse role as well as the psychological theory underlying, and key techniques (e.g. normalising, psychological education) used in, the stress support sessions. A nurse training manual focusing on the three stress support sessions, and early access to the POPPI online training was also provided.
Three-day training course for the POPPI nurses
At the beginning of the transition period, all POPPI nurses attended a face-to-face, 3-day course. This was a central training course to educate the POPPI nurses in the content and delivery of the stress support sessions and relaxation and recovery programme to patients identified as acutely stressed. The course had a strong focus on skills practice. See Chapter 2 for full details of the content of the 3-day training course and associated materials. The course was repeated for each group of sites and was delivered by a psychologist, two senior nurses and a research assistant, with patient representatives in attendance to discuss their experiences of critical care and to provide feedback to nurses.
Debriefing and support for the POPPI nurses
All POPPI nurses were allocated a clinical trainer from the POPPI training team to provide debriefing and support following the 3-day training course. See Chapter 2 for full details of the content of debriefing and support. Logistically, the first debriefing and support session was to be carried out once a POPPI nurse had delivered stress support sessions to their first patient. Once all POPPI nurses at the site had delivered stress support sessions to at least one patient each, the trainer then visited the POPPI nurses in their units to offer further support. During the visit, the POPPI nurses also underwent a skills development assessment (competency testing), to ensure that they met the required standards for delivering the sessions. If necessary, further support and training was offered, prior to the delivery of further sessions with patients. The POPPI nurses continued to receive debriefing and support either via telephone calls or site visits.
In addition, monthly teleconferences were held with the POPPI training team and the POPPI nurses to facilitate peer support and share ideas across intervention group sites.
Creating a therapeutic environment
This element of the POPPI intervention was delivered at the site (cluster) level. Each intervention group site aimed to create and promote a therapeutic environment by reducing sources of stress in the unit and improving staff–patient communication. This element of the intervention was facilitated by the POPPI online training, entitled ‘Key skills in psychological care’, for all clinical critical care staff to complete. See Chapter 2 for full details on the content of the POPPI online training. The POPPI online training took ≈30 minutes to complete; it comprised five sections and included an end-of-course assessment. Local research teams enumerated all clinical critical care staff at the start of the transition period, and then monthly thereafter to ensure that new staff members were registered for the POPPI online training.
In addition, local site teams taught and modelled good communication skills and psychological care at the bedside and ensured that the POPPI intervention materials were clearly displayed (e.g. posters displaying the key messages from the POPPI online training) and distributed (e.g. pocket cards, badges) throughout the unit. Staff were encouraged to implement their own ideas for creating a more healing environment.
Transition period
Following the intervention site initiation visit and 3-day training course for POPPI nurses, the POPPI nurses and local education/research teams commenced delivery of the POPPI intervention.
During the transition period, each POPPI nurse aimed to deliver stress support sessions to at least one recruited patient, identified (using the IPAT) as being acutely stressed and at a high risk of psychological morbidity. In parallel, the POPPI nurses and local education/research teams encouraged staff in their unit to create a therapeutic environment by ensuring that all clinical critical care staff completed the POPPI online training and through other educational activities (e.g. seminars and short presentations, bedside teaching and display of materials reinforcing key messages from the POPPI online training). At the end of this transition period, the POPPI nurses underwent the skills development assessment. Following the transition period, the POPPI intervention was delivered until the end of the recruitment period.
Site management
Communication
The trial manager (PRM) and assistant trial manager (ARB), supported by the data manager (NH) and research administrator, maintained close contact with the research teams at participating sites by telephone and e-mail throughout the trial.
Teleconferences were held, initially every month, then every 2 months, with participating sites. The teleconferences acted as a forum to discuss local barriers and challenges and share best practice across sites. Summary notes and key discussion points from the teleconferences were distributed after the call.
Reports summarising screening and recruitment activity were regularly sent to participating sites as an additional prompt to identify and discuss potential issues being faced locally in the recruitment of eligible patients.
Site monitoring visits
To ensure that all sites were visited by the trial team at least once during the recruitment period, on-site routine monitoring visits were conducted at all control group sites (intervention group sites were visited for a second time ahead of the transition period; see Intervention site initiation visits). During the monitoring visit, the investigator site file was checked for completeness, patient consent forms were checked for all recruited patients and source data verification was conducted on a random sample of patient case report forms. After the visit, a report was provided by the trial monitor to the PIs summarising the documents that had been reviewed and the actions required by the site team. The site PIs were responsible for addressing the actions and reporting back to the ICNARC CTU.
Maintenance and motivation
Throughout the trial recruitment period, a weekly e-mail was sent to site teams providing an update on recruitment and a newsletter was sent quarterly. These e-mails and newsletters provided an opportunity to clarify any queries related to trial conduct, to share ideas for maximising recruitment, and to provide general updates on trial progress. In addition, to ensure awareness was maintained at participating sites, pocket cards summarising the eligibility criteria were distributed to staff members and posters were displayed in relevant areas of the unit.
Support
A 24/7 telephone support service was available to site teams for advice on screening and recruitment of patients and on delivery of the POPPI intervention.
NHS support costs
Trials in critical care are challenging and expensive to conduct. Resources are needed to enable screening and recruitment 7 days per week. In addition, because of critical illness and the treatments that are typically administered to patients in critical care units, it is essential that staff members seeking consent are experienced in assessing a patient’s mental capacity and are able to effectively communicate information about research to the patient during a stressful situation. To this end, resources equivalent to 0.24 whole-time equivalent of a band 6 research nurse for 14 months at participating sites were successfully agreed as NHS support costs at the grant application stage.
Procedures: patient level
The trial procedures for recruitment and follow-up of patients are summarised in Figure 10.
FIGURE 10.
Overview of patient flow through the POPPI cluster RCT. ICU, intensive care unit. Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
Screening
On admission to the critical care unit, all patients were added to a screening and enrolment log. Once the patient had stayed 48 hours in the critical care unit, they were screened by the local critical care research team for the following ‘stable’ criteria (i.e. those criteria unlikely to change after this time point):
-
aged ≥ 18 years
-
receipt of level 3 care during the first 48 hours in the critical care unit
-
English-speaking
-
no pre-existing –
-
chronic cognitive impairment (e.g. dementia)
-
psychotic illness (e.g. schizophrenia)
-
chronic PTSD
-
-
not previously recruited to the POPPI study.
If the patient met all the above criteria, daily screening of the following ‘transient’ criteria (i.e. those which could fluctuate) commenced:
-
current RASS64 score between +1 and –1
-
current Glasgow Coma Scale65 score of 15
-
not receiving end-of-life care
-
currently able to communicate orally
-
able to give informed consent (e.g. not deemed delirious by the CAM-ICU). 68
If any of the daily screening criteria were not met, the patient would be rescreened each day until either fully meeting the criteria or being discharged from the critical care unit. Once a patient met all daily screening criteria (in addition to the stable criteria), they were approached for informed consent in the unit. To ensure full and transparent reporting for the trial, the final screening status of the patient, including reasons why patients were not recruited (e.g. patient declining the invitation to take part, the patient being excluded by the treating clinician, logistical reasons), was recorded. No patient identifiers were submitted to the ICNARC CTU on the screening and enrolment log.
For local teams to become familiar with screening for eligibility, in the 2 weeks prior to the planned start date of recruitment, a screening ‘run-in’ period was implemented during which teams practised the procedures for screening (i.e. applying the eligibility criteria to patients and completing the screening and enrolment log). Eligible patients were not recruited/approached for consent during the run-in period.
Informed consent
During the recruitment period, patients who met the eligibility criteria in the critical care unit were approached and provided with written and verbal information about the POPPI intervention by an authorised member of the local research team. See Report Supplementary Materials 1 and 2 for the PIS used during the usual care period and intervention period, respectively. Potential participants were given the opportunity to ask questions and time to discuss the trial with family or friends before making their decision. After the person seeking consent was satisfied that the information had been understood and questions had been answered, they invited potential participants to sign the consent form. In providing informed consent, participants were agreeing for the trial team to have access to their medical records for data collection and to receive a follow-up questionnaire at 6 months. In addition, participants recruited at intervention group sites from month 6 onwards (see Figure 8) were offered the option to provide consent to receive an assessment with the IPAT and subsequent stress support sessions and relaxation and recovery programme (if applicable). To minimise selection bias between the intervention group and control group sites, it was possible for a patient at an intervention group site to decline participation in the intervention but still provide consent to receive the 6-month follow-up questionnaire.
On entry into the cluster RCT, the participant’s general practitioner (GP) was informed, by letter, of their recruitment into the POPPI trial.
Delivery of the POPPI intervention at the patient level
Figure 11 shows the timeline for a patient recruited at an intervention group site from month 6 onwards.
FIGURE 11.
Patient timeline during the intervention period. Reproduced from Richards-Belle et al. 62 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. Includes minor additions and formatting changes to the original.
The Intensive care Psychological Assessment Tool assessment
Consenting, eligible patients at intervention group sites, during the intervention period, were assessed using the IPAT52 by a trained staff member as soon as possible, but within 48 hours of consent being provided. A patient was deemed acutely stressed and at a high risk of psychological morbidity if they scored ≥ 7 on the IPAT and was then referred, as soon as possible, to a POPPI nurse to receive the three stress support sessions and relaxation and recovery programme. Patients who scored < 7 on the IPAT continued to receive usual care as determined by the treating clinician(s). If a patient scored 5 or 6 on the IPAT, they were reassessed daily, for a maximum of 3 days, until either they left the critical care unit or the score dropped below 5.
Three stress support sessions for patients identified as acutely stressed
The three stress support sessions were to be delivered by the same POPPI nurse ideally within 1 week, with the first stress support session starting as soon as possible within 48 hours following the IPAT assessment. Each session lasted ≈30 minutes and, ideally, at least the first session was delivered in the critical care unit, although sessions could be delivered elsewhere in the hospital (e.g. general ward) if the patient moved. If the patient showed signs of distress or fatigue during the session, then the session was stopped and a new visit was arranged for a more appropriate time. The State–Trait Anxiety Inventory – Short Form (STAI-6)72 was used to assess a patient’s anxiety prior to session 1 (at baseline) and at the end of session 3. If a patient showed serious signs of distress at the end of their three sessions, their medical team was informed. See Chapter 2 for full details of the content of the stress support sessions.
After the transition period, a sample of consented, acutely stressed patients were asked to consent to their stress support sessions being audio-recorded. If a patient agreed, they were asked to sign the audio-recording consent form. This was optional and did not preclude patient participation in the trial or delivery of the stress support sessions. Audio-recordings would be reviewed by the training team to monitor fidelity of the stress support sessions delivered, and would be destroyed at the end of the trial. If a patient withdrew consent for use of their session to be audio-recorded, then the audio file would be deleted and no longer used.
Follow-up
All participants surviving to 6 months were sent a follow-up questionnaire by the ICNARC CTU at 6 months. The questionnaire contained the PSS-SR57 (primary outcome), HADS67 and EQ-5D-5L56 questionnaires and a health services questionnaire (see Report Supplementary Material 3).
The follow-up process started at 157 days post-recruitment for the 6-month follow-up to allow for the administrative processes. Participating sites were contacted monthly to confirm survival status to help ensure that questionnaires were not received by relatives or friends of deceased patients. Patients indicated as having died since leaving hospital were logged and the follow-up process ended.
Questionnaire packs were sent to participants by either post or e-mail (as requested by the participant) and a self-addressed stamped envelope and pen were included in all postal questionnaires for ease of return. Participants who requested the questionnaire by e-mail were sent an electronic version of the questionnaire – which could either be completed using Microsoft Word (Microsoft Corporation, Redmond, WA, USA) and e-mailed back to the ICNARC CTU, or printed, completed on paper and posted back to the ICNARC CTU. The cover of the questionnaire included a ‘I do not wish to complete this questionnaire’ tick box for participants to indicate if they no longer wished to complete the questionnaire.
Non-responders were telephoned 3 weeks later to check whether or not they had received the questionnaire and were given the option to complete the questionnaire over the telephone, if preferable, with a trained member of the trial team. Telephone calls were made at various times from Monday to Friday between 08:30 and 20:00 to maximise the chances of contacting the patient.
Follow-up ended on receipt of a completed (or blank) questionnaire or a questionnaire with a ticked ‘I do not wish to complete this questionnaire’ box; on notification to the ICNARC CTU by telephone or e-mail that the participant wished to withdraw from the trial; or if there was no response to telephone follow-up. For questionnaire packs returned indicating that the recipient was not known at the address, the contact details for the patient were checked with the recruiting hospital and/or GP.
For patients who were identified as being a hospital inpatient or resident in a nursing home or rehabilitation centre, the relevant institution was contacted to establish the status of the patient and the most appropriate way to proceed with follow-up. If the patient had mental capacity to consent but required assistance in reading and/or completing the questionnaire, then health-care professionals usually assisted the patient.
If a completed questionnaire received at the ICNARC CTU indicated the presence of signs of serious stress (i.e. a score of ≥ 18 on the PSS-SR), anxiety or depression (i.e. scores of > 7 on the relevant subscale of HADS), then a referral letter from the lead clinical investigator (DW) was sent to the patient’s GP or the local PIs at the recruiting site.
Safety monitoring
As the treatments considered in this cluster RCT were psychological interventions, adverse events were not expected.
Data collection
A secure, dedicated electronic case report form, hosted by ICNARC, was set up to enable trial data to be entered by staff at sites (see Report Supplementary Materials 4 and 5). The electronic case report form was only accessible to authorised users and access was approved centrally by the assistant trial manager or data manager after cross-checking the site delegation log. Each individual created a unique username and password and had access to data for patients recruited at their site only.
The data set for the POPPI intervention included the minimum data required to confirm patient eligibility, described the patient population, monitored and described delivery of the intervention (at intervention group sites only), assessed primary and secondary outcomes, and enabled linkage to the CMP and NHS Digital. Table 5 shows the patient data collection schedule, which is described below.
| Data collected | Baseline (at point of recruitment) | End of critical care unit stay | Intervention group sites: during transition and intervention period | 6 months post recruitment | ||
|---|---|---|---|---|---|---|
| Before stress support session 1 | During sessions | After stress support session 3 | ||||
| Collected in hospital | ||||||
| Patient details | ✓ | |||||
| Clinical/baseline data | ✓ | |||||
| Critical care unit stay | ✓ | |||||
| IPAT assessment | ✓ | |||||
| stress support session delivery | ✓ | |||||
| STAI-6 | ✓ | ✓ | ||||
| Collected via follow-up questionnaires sent to patients | ||||||
| PSS-SR | ✓ | |||||
| HADS | ✓ | |||||
| EQ-5D-5L | ✓ | |||||
| Health service and resource use | ✓ | |||||
Eligibility
For all patients, data were collected to confirm that the patient met all of the inclusion criteria and none of the exclusion criteria.
Baseline
The following data were collected for all patients at baseline (i.e. at the point of recruitment) to enable follow-up and to describe the patient population:
-
full name and address of the patient and their GP
-
NHS number
-
date of birth
-
sex
-
last physiology [temperature, systolic blood pressure, pulse rate, respiratory rate, peripheral oxygen saturation (SpO2) and fraction of inspired oxygen (FiO2)] results prior to consent
-
HRQoL health thermometer score56 [scores relate to a participant’s health on the day of assessment and range from 0 (‘the worst health you can imagine’) to 100 (‘the best health you can imagine’)] and
-
STAI-673 anxiety score (scores range from 20 to 80, with higher scores reflecting greater anxiety).
Intervention period
At intervention group sites during the intervention period only, further data were collected to monitor adherence to the intervention and to describe and cost the delivery of the intervention (see Report Supplementary Material 6). These data included:
-
date/time of the IPAT52 assessments
-
The IPAT52 scores
-
delivery of the three stress support sessions (i.e. dates, location, duration)
-
delivery of the relaxation and recovery programme (whether or not patients were given the tablet computer, whether or not patients reported use of the tablet computer as well as whether or not they were given the relaxation and recovery DVD and the Getting Well, Staying Well booklet to take home)
-
STAI-673 anxiety score, following session 3.
Patients were also given the opportunity to feed back their experiences of the stress support sessions via e-mail or an online form.
During the critical care unit stay
Throughout the critical care unit stay, the following data were collected daily for all patients:
-
presence of delirium as defined by the CAM-ICU68
-
receipt of sedatives, anxiolytics, anaesthetics, sleep medications, antipsychotics, analgesics, antidepressants and vasoactive agents
-
receipt of mechanical ventilation.
At critical care unit discharge
At the time of discharge from the critical care unit the following data were collected: date and time of discharge from, or death in, the critical care unit.
At acute hospital discharge
At the time of discharge from the acute hospital, the following data were collected: date of discharge from, or death in, the acute hospital.
Data linkage with the Case Mix Programme
Linkage of patient trial data to the CMP database provided information on the baseline characteristics of patients and subsequent admission(s) to a critical care unit following discharge from the original critical care unit.
Data for the CMP are collected by trained data collectors to precise rules and definitions. The data then undergo extensive local and central validation for completeness, illogicalities and inconsistencies prior to pooling.
Data linkage with NHS Digital
Following the signing of a data-sharing agreement between the sponsor (ICNARC) and NHS Digital, to confirm survival status, the POPPI trial data were linked to national death registrations in the Medical Research Information Service Database Administration Service, held by NHS Digital.
Data management
To ensure data completeness and accuracy, data entered by sites into the electronic case report form were regularly monitored and checked. Ongoing data entry and validation at sites were closely monitored by the data manager (NH) and any concerns were raised with the site teams.
Two levels of data validation were built into the electronic case report form. The first level was to prevent obviously erroneous data from being entered, for example entering a date of birth that occurred after the date of consent. The second level involved checks for data completeness and any unusual data entered, for example a physiological variable, such as respiratory rate, that was outside the pre-defined range. Site staff could generate data validation reports, listing all outstanding data queries, at any time via the electronic case report form. The site PIs were responsible for ensuring that all data queries were resolved.
Data received from completed 6-month follow-up questionnaires were entered centrally into a secure database at the ICNARC CTU following a standard operating procedure. All identifiable information, such as names (e.g. of patients, family members or hospital staff members), was removed. All queries relating to data entry were reviewed by two members of the trial team (ARB/NH) and any disagreement was reviewed and discussed with a third member of the trial team (PRM). To ensure accuracy, all questionnaire data entered into the database were cross-checked by a second member of the ICNARC CTU team. Any errors found were logged and corrected on the database.
Following the start of the transition period at intervention group sites, data relating to the delivery of the POPPI intervention was monitored closely to ensure protocol adherence. Reports on uptake of the POPPI online training were initially provided to sites on a weekly/biweekly basis, and then monthly, to ensure that new starters and leavers were identified and to allow continued identification of those staff members who had yet to complete the POPPI online training.
Power calculation
Pre-trial power calculation
The power calculation was completed using the approach of Hussey and Hughes59 to achieve 90% power to detect a reduction from 6 points to 3.1 points (p < 0.05) in the mean PSS-SR score at 6 months, and was based on the following assumptions:
-
Mean (6) and SD (7.5) of the PSS-SR score taken from patients in the feasibility study.
-
Estimated ICC of 0.138 – between-site coefficient of variation 0.5, corresponding to between-site SD of 3 (this is a conservative estimate as no multicentre data were available). 60 Note that the inclusion of a baseline recruitment period means that the sample size calculation is less sensitive to the degree of clustering. 59
-
Treatment effect of a reduction of 2.9 points on the PSS-SR score, based on a RCI for the PSS-SR of 8.6 points,61 being observed in 40% of eligible patients in the intervention periods assessed as being acutely stressed and at a high risk of psychological morbidity using the IPAT, with 16% of recruiting patients declining the intervention.
-
Harmonic mean of the number of patients completing follow-up (52 patients per site per annum – corresponding to 22 patients in a 5-month period), based on data from the CMP.
With the design and the above assumptions, the estimated total number of patients recruited (based on the CMP data) for the cluster RCT was 1914 patients from the 24 sites. It was anticipated that 438 patients would be assessed using the IPAT, of which 175 (40%) would be assessed as being acutely stressed and at a high risk of psychological morbidity, and therefore receive the stress support sessions.
Final review of assumptions in pre-trial power calculation
During recruitment, in consultation with the Trial Steering Committee (TSC) and Data Monitoring and Ethics Committee (DMEC), a review of assumptions underlying the pre-trial power calculation was conducted once outcome data were available for patients recruited during the 5-month baseline period in both intervention and control group sites. This review, undertaken using data available on 9 August 2016, identified the following re-estimations of the assumptions:
-
Mean (10.3) and SD (10.8) of the PSS-SR.
-
ICC of 0.087 [95% confidence interval (CI) 0 to 0.192] (with mean, SD and ICC estimated using all available data from a previous observational study,52 the feasibility study and the baseline period of the cluster RCT).
-
Treatment effect of a reduction of 4.2 points on the PSS-SR – estimated by retaining the same effect size as a multiple of the within-site SD.
-
Harmonic mean of the number of patients completing follow-up (30.7 per site per annum – corresponding to 12.8 in a 5-month period), estimated using observed data from the baseline period.
This review established that the planned design had an anticipated 78% power under the observed parameter estimates (allowing for uncertainty in the between-site variation, between 73% and 85% power).
Consequently, the decision was taken to extend recruitment in all sites to the end of planned recruitment for the final group of eight sites (corresponding to an harmonic mean of 16.5 patients completing follow-up per site during the intervention period, allowing for the variation from 5 months’ to 9 months’ duration across sites, see Figure 8). With this extension to recruitment, the planned design had an anticipated 85% power (allowing for uncertainty in the between-site variation, between 79% and 91% power). It was anticipated that, with this extension to recruitment, the estimated total number of patients recruited would be 1378. Recruitment continued to be monitored to ensure that ≥ 1378 patients were recruited. A final decision to extend recruitment by an additional 2 months in all sites was taken to ensure that this minimum number was achieved. A protocol amendment was implemented to reflect this review of assumptions and the extension to the recruitment period.
Statistical methods
Analysis principles
All analyses were based on the intention-to-treat (ITT) principle. Patients were analysed according to the treatment group they were randomised to, irrespective of whether or not the allocated treatment was received. The final analyses were conducted according to the published statistical analysis plan70 and using Stata®/SE Version 14.2 (StataCorp LP, College Station, TX, USA). Multiple imputation was performed in R version 3.3.2 (The R Foundation for Statistical Computing, Vienna, Austria).
Interim analysis
As the duration of follow-up for the primary outcome (6 months) was long relative to the duration of recruitment (intervention period between 7 and 11 months), no interim analysis of effectiveness was planned.
Methods for withdrawals and missing data
All patients who provided informed consent are accounted for in the trial report. Mortality at 6 months was anticipated to be 10% and loss to follow-up for the primary outcome was anticipated to be 20% among survivors. Loss to follow-up for mortality at 6 months was anticipated to be < 1%. Patients who withdrew from the trial, those who died before 6 months and those lost to follow-up for mortality were excluded from the analysis of 6-month psychological outcomes. Patients recruited during the transition period were also excluded from the analysis. All other recruited patients were included in the primary analysis, with outcomes imputed if missing.
Loss to follow-up is reported by treatment group.
Multiple imputation was used to complete missing baseline and resource use covariates, and non- and partial responses for the PSS-SR,57 the HADS67 and the EQ-5D-5L,56 under the assumption that responses were missing at random (MAR) conditional on the observed data. 74 Two-level imputation (patients nested in sites) was implemented using the ‘mice’ package in R. 75 The overall score on each measure was imputed, not individual item responses. The imputation model included the following covariates:
-
site-level covariates (* denotes covariates used to balance treatment allocation) –
-
teaching status of hospital (teaching, non-teaching)*
-
number of beds in the critical care unit (linear)
-
number of critical care unit admissions receiving level 3 care staying ≥ 48 hours during the pre-trial period,1 April 2014 to 31 March 2015 (linear)*
-
allocated treatment group (intervention, control)
-
-
patient-level covariates –
-
time period (baseline, intervention) and interaction with treatment group
-
age in years (linear)
-
sex (female, male)
-
ethnicity (white, non-white)
-
quintile of the Index of Multiple Deprivation (IMD) 201576 (categorical)
-
documented (i.e. in medical records) pre-existing anxiety and/or depression prior to hospital admission (anxiety, depression, both, none)
-
planned admission to the critical care unit following elective/scheduled surgery (yes, no)
-
ICNARC Physiology Score77 from the first 24 hours following admission to the critical care unit (linear)
-
last National Early Warning Score (NEWS)78 prior to consent (linear)
-
HRQoL at time of consent, assessed as health thermometer score from 0 to 100 (linear)
-
STAI-673 at time of consent, scored from 6 to 24 (linear)
-
duration of stay in the critical care unit in days (linear)
-
number of days of delirium, as assessed by the CAM-ICU68, in the critical care unit (linear)
-
number of days receiving sedatives/anxiolytics/anaesthetics in the critical care unit (linear)
-
number of days receiving sleep medications in the critical care unit (linear)
-
receipt of benzodiazepines in the critical care unit (yes, no)
-
number of days receiving antipsychotics in the critical care unit (linear)
-
number of days receiving analgesics in the critical care unit (linear)
-
number of days receiving antidepressants in the critical care unit (linear)
-
number of days receiving vasoactive agents in the critical care unit (linear)
-
number of days receiving mechanical ventilation in the critical care unit (linear)
-
duration of stay in hospital following discharge from the critical care unit (linear)
-
adherence to intervention (binary)
-
PSS-SR57 score at 6 months (linear)
-
HADS67 anxiety score at 6 months (linear)
-
HADS67 depression score at 6 months (linear)
-
EQ-5D-5L56 score at 6 months (linear)
-
health services questionnaire costs at 6 months (linear)
-
Fifty imputed data sets were generated using predictive mean matching. The random number seed was set prior to sampling to ensure reproducibility of results.
To evaluate the results of the primary clinical evaluation under the assumption of missing completely at random (MCAR), the analyses were repeated using complete-case data (i.e. only those patients returning a completed questionnaire).
Multiple comparisons and multiplicity
No adjustment was made to account for multiple end points or multiple subgroups; p < 0.05 was taken to represent a statistically significant result. The results of subgroup analyses were interpreted, taking into account the number of significant findings that would have been expected by chance alone. 79
Statistical analyses
Screening and recruitment
The flow of clusters through the trial is displayed in a Consolidated Standards of Reporting Trials (CONSORT) diagram (see Figure 12), based on the CONSORT extension for cluster-randomised trials. 80
Descriptive statistics were calculated using data from screening and enrolment logs to describe the screening, recruitment and follow-up of patients through the trial. Patient data are summarised, as follows, across treatment groups and time periods:
-
total patients admitted
-
total patients not meeting stable criteria –
-
stayed in critical care unit < 48 hours
-
no level 3 care during first 48 hours in critical care unit
-
aged < 18 years
-
not English-speaking
-
previously recruited to the POPPI intervention
-
pre-existing chronic cognitive impairment
-
pre-existing psychotic illness
-
pre-existing chronic PTSD
-
-
total patients not meeting transient criteria –
-
Glasgow Coma Scale score of < 15
-
RASS64 score not between +1 and –1
-
receiving end-of-life care
-
not able to communicate orally
-
unable to consent
-
-
total patients for whom eligibility was unknown
-
total eligible patients –
-
missed
-
other reasons not enrolled
-
approached for informed consent
-
-
total patients approached –
-
consent declined
-
recruited.
-
Demographic and baseline characteristics
Baseline demographic and clinical data were summarised for the ITT population, for each of the two treatment groups in each of the two time periods. Continuous variables are summarised as mean (SD) and median (IQR) whereas categorical variables are summarised as number (%). There was no statistical testing for any of the summary measures while comparing the baseline variables between the treatment groups. The following baseline variables were compared between the two treatment groups:
-
age in years
-
sex (female, male)
-
ethnicity (white, mixed, Asian, black, other, not stated)
-
quintile of IMD 201576 (1 = least deprived, 5 = most deprived)
-
documented (i.e. in medical records) pre-existing anxiety/depression (anxiety, depression, both, none)
-
planned admission to the critical care unit following elective/scheduled surgery (yes, no)
-
ICNARC Physiology Score77 from the first 24 hours following admission to the critical care unit
-
Acute Physiology and Chronic Health Evaluation (APACHE) II score81 from the first 24 hours following admission to the critical care unit
-
duration of stay in the critical care unit prior to consent
-
number of days experiencing delirium, as defined by the CAM-ICU,68 in the critical care unit prior to consent
-
last NEWS78 prior to consent
-
STAI-673 at time of consent
-
HRQoL at time of consent (health thermometer score).
Delivery of the POPPI intervention
Uptake of the POPPI intervention online training is reported for intervention sites over time, as the percentage of the enumerated critical care unit staff who had completed the training course by month against a target of > 80% completion.
Co-interventions received in the critical care unit were summarised for the ITT population, for each of the two treatment groups in each of the two time periods. Interventions received were summarised as the number (%) of patients receiving the intervention, the median (IQR) number of days over which the intervention was received (among those receiving the intervention) and the mean (SD) number of days over which the intervention was received (for all patients, including those who did not receive the intervention). There was no statistical testing for any of the summary measures while comparing the intervention variables between the treatment groups. The following categories of interventions are compared between the two treatment groups:
-
sedatives/anxiolytics/anaesthetics
-
sleep medications
-
benzodiazepines (note that benzodiazepines are included as either sedatives/anxiolytics/anaesthetics or sleep medications, as appropriate)
-
antipsychotics
-
analgesics
-
antidepressants
-
vasoactive agents
-
mechanical ventilation.
Delivery of the POPPI intervention at a patient level is summarised for patients in the intervention group during the intervention period. The following are reported for all patients:
-
number (%) of patients consenting to assessment using the IPAT
-
among those consenting, the number (%) of patients assessed using the IPAT
-
number (%) of patients with an IPAT score of ≥ 7
-
median (IQR) IPAT score.
The following are reported for patients with an IPAT score of ≥ 7:
-
number (%) of patients by number of stress support sessions received (0, 1, 2, 3)
-
number (%) of patients by location of delivery of the stress support sessions
-
reasons for not receiving all three stress support sessions
-
number of patients receiving a tablet computer (% of those receiving stress support session 1)
-
number of patients reporting using the tablet computer (% of those receiving tablet computer)
-
number of patients receiving the relaxation and recovery DVD and the Getting Well, Staying Well booklet (% of those receiving stress support session 2)
-
mean (SD) and median (IQR) STAI-6 at time of consent and after stress support session 3 for patients receiving three stress support sessions and with STAI-6 reported at both time points.
Clinical effectiveness analysis: primary outcome
The primary analysis for the clinical evaluation examines if there is a significant difference in the mean PSS-SR57 score at 6 months between patients recruited to the intervention group and those recruited to the control group. This was performed using a generalised linear mixed model (GLMM) at the individual patient level (patients nested within sites and within treatment group/time period), including the following terms:
-
Fixed effects at the site level (* denotes covariates used to balance treatment allocation) –
-
teaching status of hospital (teaching, non-teaching)*
-
number of beds in the critical care unit (linear)
-
number of critical care unit admissions receiving level 3 care staying ≥ 48 hours during the pre-trial period,1 April 2014 to 31 March 2015 (linear)*
-
allocated treatment group (intervention, control)
-
-
Fixed effects at the patient level –
-
time period (baseline, intervention) and interaction with treatment group
-
age in years (restricted cubic splines, 4 knots)
-
sex (female, male)
-
ethnicity (white, non-white)
-
quintile of IMD 201576 (categorical)
-
documented pre-existing anxiety and/or depression prior to hospital admission (anxiety, depression, both, none)
-
planned admission to the critical care unit following elective/scheduled surgery (yes, no)
-
ICNARC Physiology Score77 from the first 24 hours following admission to the critical care unit (restricted cubic splines, 4 knots)
-
-
Random effects (intercepts) at the site level.
The identity link (i.e. linear regression) was used as the link function for the model and a robust variance estimation82 was used to estimate the standard errors of the covariates, as it adjusts for possible deviations from the model’s assumptions. Rubin’s rules were used to combine estimates from the multiply imputed data sets. The coefficients with their 95% CIs and p-values are presented for the fixed-effect covariates whereas only the coefficients with their 95% CIs are reported for the random-effect variables. The primary effect estimate is the interaction (difference in difference) between treatment group and time period.
A secondary analysis used structural mean models with an instrumental variable of randomised allocated treatment to estimate the efficacy (adherence-adjusted causal effect) of the stress support sessions among those patients consenting to the IPAT assessment and stress support sessions, assessed as being acutely stressed and at a high risk of psychological morbidity (IPAT score of ≥ 7 points) and receiving at least two stress support sessions. 83
As a post hoc secondary analysis of the primary outcome, the model was refitted including an additional site-level covariate of the natural logarithm of the standardised mortality ratio (ratio of observed deaths to predicted deaths from the ICNARCH-2015 risk prediction model)84 from the period April 2014 to March 2015.
Clinical effectiveness analysis: secondary outcomes
Analyses of the secondary outcomes were also performed using GLMMs (like the primary outcome analysis), with identity link (i.e. linear regression) for continuous secondary outcomes (reported as difference in means with 95% CIs and p-values) and logit link (i.e. logistic regression) for binary secondary outcomes (reported as odds ratios with 95% CIs and p-values). Robust variance estimation methods were used to estimate the standard errors of the covariates in both the mixed linear and the logistic regression models.
Subgroup analyses
There were planned subgroup and interaction analyses for the cluster RCT. The a priori identified subgroups that were used for the subgroup analyses are as follows:
-
age
-
quartiles
-
-
sex
-
male versus female
-
-
socioeconomic status – quintile of IMD 201576
-
1 (least deprived) versus 2 versus 3 versus 4 versus 5 (most deprived)
-
-
duration of delirium
-
patients with no CAM-ICU defined delirium versus patients with CAM-ICU defined delirium less than the overall median duration of delirium versus patients with CAM-ICU defined delirium greater than or equal to the overall median duration of delirium
-
-
STAI-673
-
quartiles
-
-
Surgical status
-
emergency/urgent surgery versus elective/scheduled surgery versus non-surgical
-
-
predicted PSS-SR (heterogeneity of treatment effect), from a prediction model for the primary outcome derived using data from patients receiving usual care and adjusted for a priori important covariates (age, sex, socioeconomic status, duration of delirium, STAI-6, surgical status)85
-
quintiles
-
-
site implementation score (from process evaluation)
-
tertiles.
-
The evaluation of the subgroup effect on the primary outcome was carried out using a test of interaction obtained from the GLMM. 79 The GLMM contained a main effect term denoting the specific subgroup of interest, main effect terms for treatment group and time period, and all second- and third-level interaction terms. The test of whether or not the treatment effect varied across subgroups was based on the third-level interactions.
Governance, management and oversight
Ethics approval and local permissions
The POPPI cluster RCT was sponsored by ICNARC and co-ordinated by the ICNARC CTU. An ethics application was submitted to the National Research Ethics Committee South Central – Oxford B REC on 27 April 2015 and the POPPI cluster RCT received a favourable ethics opinion on 15 May 2015 (reference number: 15/SC/0287). Local NHS permissions were obtained from each participating NHS trust. A clinical trial site agreement, based on the model agreement for non-commercial research in the health service, was signed by each participating NHS trust and the sponsor (ICNARC).
Trial registration
To ensure transparency, the cluster RCT was prospectively registered with the ISRCTN registry on 15 July 2015. Registration was confirmed on 16 July 2015 (ISRCTN53448131). The NIHR CRN Portfolio details high-quality clinical research studies that are eligible for support from the NIHR CRN in England. The trial was adopted onto the NIHR CRN Portfolio on 30 April 2015 and issued the NIHR CRN Portfolio number 18940. In addition, the POPPI cluster RCT was adopted onto the Health and Care Research Wales Clinical Research Portfolio on 25 November 2015.
Trial management
The trial manager (PRM) was responsible for day-to-day management of the cluster RCT with support from the assistant trial manager (ARB). In addition, the day-to- day trial team comprised the research administrator, data manger (NH) and trial statistician (JWu). The Trial Management Group (TMG), chaired by the trial manager (PRM), was responsible for overseeing day-to-day management of the trial and comprised the chief investigator (KMR), lead clinical investigator (DW), methodological co-investigators (DAH, RG, SH and ZS), clinical/psychology co-investigators (CRB, DH, DS, JWei, JWel and MM) and day-to-day trial team (ARB, JWu and NH). The TMG met regularly throughout the cluster RCT to monitor conduct and progress and to ensure adherence to the protocol.
Patient and public involvement
The involvement of former critical care patients was crucial to the successful conduct of the cluster RCT. A group of six former patients were members of the POPPI training team and attended the 3-day training courses for the POPPI nurses to talk about their experiences of critical care and to observe and provide feedback to trainees. In addition, three patient and public representatives were members of the TSC and provided input into the conduct of the trial.
Oversight committees
Following NIHR guidelines, a TSC, with a majority of independent members, was convened to oversee the trial on behalf of the funder (NIHR) and the sponsor (ICNARC). The TSC met at least annually during the trial and comprised an independent chairperson, lay members (representing patient and public perspectives), independent clinicians and researchers (specialising in critical care and psychology), the chief investigator (KMR) and the lead clinical investigator (DW) representing the TMG.
Furthermore, an independent DMEC was convened to monitor trial data and ensure the safety of trial participants. The DMEC met at least annually during the trial, comprised two expert clinicians specialising in critical care medicine and was chaired by an experienced statistician.
Substantial amendments to the cluster randomised clinical trial
Following receipt of initial favourable opinion for the trial from the REC on 15 May 2015, four substantial amendments were submitted and received favourable opinion. These were as follows:
-
Amendment 1 (approved on 11 March 2016) – following completion of the POPPI feasibility phase, the assumptions underlying the power calculation were reviewed and revised, following ratification from the DMEC (see Chapter 3 for full details of this review). To understand any potential effect of the intervention on anxiety, the patient follow-up questionnaires were revised and the HADS67 replaced the CES-D-1058 to allow both depression and anxiety to be measured. A witness signature line was added to the patient consent forms to formalise the procedure by which a patient is willing to provide informed consent but unable to physically sign the consent form (e.g. as a result of reduced dexterity). A poster was included for sites to display in relatives’ rooms. A GP reporting form was added to the GP letter to allow GPs to inform the ICNARC CTU of any patient significant psychological difficulties that they may be aware of.
-
Amendment 2 (approved on 1 February 2017) – following review of outcome data from the baseline (pre-intervention) period, the assumptions underlying the power calculation were reviewed and revised, following ratification from the DMEC and TSC.
-
Amendment 3 (approved on 29 March 2017) – at the recommendation of the TSC and following approval from the funder (NIHR), a £5.00 gift voucher was included for participants receiving their 6-month follow-up questionnaire as a means of potentially increasing the response rate.
-
Amendment 4 (approved on 29 December 2017) – as part of the data-sharing agreement between the sponsor (ICNARC) and NHS Digital, the trial team sent a new issue of the patient newsletter to participants. The primary purpose of the newsletter was to inform participants of the data flow used by the trial team to obtain survival status information from NHS Digital, provide an update on trial progress and inform participants where they would be able to find the trial results.
Network support
To maintain the profile of the POPPI cluster RCT, regular updates on progress were provided at quarterly meetings of the NIHR CRN Critical Care Specialty Group and at local CRN meetings. In addition, updates were provided at national meetings, such as the CMP Annual Conference and the UK Critical Care Research Forum.
Chapter 5 Cluster randomised clinical trial results: sites and patients
Participants: sites
Site selection
Expressions of interest to participate in the POPPI cluster RCT were received from 118 NHS adult, general critical care units across England, Wales and Northern Ireland. Potential sites were asked to complete a site feasibility questionnaire, of which 86 completed questionnaires and were subsequently assessed for eligibility by the ICNARC CTU.
Relative to the target of 24 participating sites – of 86 potential sites completing a questionnaire – 32 sites did not meet inclusion criteria (e.g. had insufficient resources/capacity to deliver the trial or would see too few patients over the trial period). The final 24 sites were selected from the remaining 54 sites, with reasons for selection including good research track-record and display of a high level of enthusiasm. A small number of sites were already delivering early psychological support and so were not selected. In addition to the 24 sites, we selected eight reserve sites to be approached in the event that a selected site was no longer able to participate.
One site that was initially selected lost eligibility to take part as a result of inability to implement use of the CAM-ICU68 to assess delirium prior to the start of the trial (see Chapter 4 for the inclusion criteria for sites). A site from the reserve list was rapidly set up as a replacement.
Site set-up
A total of 24 sites (22 in England, one in Wales and one in Northern Ireland) obtained local NHS permissions/approvals and opened to recruitment in three groups of eight sites, at 2-month intervals. Sites allocated to the first group opened to recruitment on 1 September 2015. Groups two and three opened on 2 November 2015 and 4 January 2016, respectively. One site in group three opened 3 days late, owing to delays in obtaining local NHS permissions through the Christmas/New Year period. Site initiation visits were carried out at all participating sites prior to start of patient screening and recruitment. Sites were randomised in their second month of recruitment (during the baseline period). The flow of sites (clusters) through the cluster RCT is presented in Figure 12, according to the CONSORT extension for cluster trials. 80
FIGURE 12.
The CONSORT flow of sites (clusters) and patients. ICU, intensive care unit. Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
Overall, as planned, 24 sites participated in the POPPI cluster RCT. One site from the second group of sites closed in December 2016 prior to the end of the recruitment period because key staff were scheduled onto other projects when the recruitment period was extended. The remaining 23 sites remained open to screening and recruitment until the end of recruitment in January 2017.
Intervention site set-up
Following site randomisation, each intervention group site identified three self-selected POPPI nurses (plus one reserve), according to the person specification. Each POPPI nurse signed a form confirming their commitment to undertake the POPPI nurse role. Intervention site initiation visits were carried at all intervention group sites ahead of the 3-day training course and prior to commencement of the transition period.
In total, 38 POPPI nurses were trained across the series of three central 3-day training courses held in February, April and June 2016. For two sites, the reserve POPPI nurses was also trained – one because an original POPPI nurse went on a career break and the other to help provide increased cover for delivery of the stress support sessions at one of the larger participating intervention group sites.
Characteristics of participating sites
The characteristics of the 24 adult, general critical care units that participated in the POPPI cluster RCT compared with all other adult, general critical care units in the CMP (n = 191) are presented in Table 6. Overall, a representative mix of university and non-university/university-affiliated hospitals, geographically spread across England, Wales and Northern Ireland, took part in the cluster RCT. A higher proportion of larger units participated in the POPPI cluster RCT compared with all other units participating in the CMP. Use of diaries in the critical care units was reported at the start of the study at nine intervention and seven control sites.
| Critical care unit characteristic | Units in the POPPI cluster RCT (N = 24), n (%) | Units in the CMPa (N = 191), n (%) |
|---|---|---|
| Region | ||
| North | 8 (33.3) | 55 (28.8) |
| Midlands/East | 3 (12.5) | 35 (18.3) |
| London/South East | 2 (8.3) | 50 (26.2) |
| South West/South Central | 9 (37.5) | 29 (15.2) |
| Wales | 1 (4.2) | 14 (7.3) |
| Northern Ireland | 1 (4.2) | 8 (4.2) |
| Type of hospital | ||
| University | 8 (33.3) | 56 (29.3) |
| Non-university/university affiliated | 16 (66.7) | 135 (70.7) |
| Size of critical care unit (beds) | ||
| < 8 | 2 (8.3) | 33 (17.3) |
| 8–11 | 5 (20.8) | 68 (35.6) |
| 12–15 | 7 (29.2) | 36 (18.8) |
| ≥ 16 | 10 (41.7) | 54 (28.3) |
| Annual critical care unit admissions | ||
| < 500 | 2 (8.3) | 47 (24.6) |
| 500–749 | 5 (20.8) | 62 (32.5) |
| 750–999 | 8 (33.3) | 41 (21.5) |
| ≥ 1000 | 9 (37.5) | 41 (21.5) |
Participants: patients
Screening
Table 7 summarises data from screening and enrolment logs maintained by participating sites, stratified by treatment group (usual care or intervention) and time period (baseline, transition and intervention).
| Screening and recruitment | Baseline period | Transition period | Intervention period | |||
|---|---|---|---|---|---|---|
| Usual care | Intervention | Usual care | Intervention | Usual care | Intervention | |
| Not eligible, n/N (% of screened) | 4543/5051 (89.9) | 3724/4257 (87.5) | 830/927 (89.3) | 728/827 (88.0) | 7240/8106 (89.3) | 6432/7290 (88.2) |
| Did not meet stable criteria (n)a | 3644 | 3131 | 658 | 599 | 5828 | 5393 |
| < 48 hours in critical care unit, n (%) | 2129 (58.4) | 1898 (60.6) | 405 (61.6) | 391 (65.3) | 3351 (57.5) | 3461 (64.2) |
| No level 3 care in first 48 hours in unit, n (%) | 1407 (38.6) | 1089 (34.8) | 216 (32.8) | 176 (29.4) | 2253 (38.7) | 1652 (30.6) |
| Aged < 18 years, n (%) | 26 (0.7) | 19 (0.6) | 8 (1.2) | 0 (0) | 59 (1.0) | 30 (0.6) |
| Not English-speaking, n (%) | 45 (1.2) | 27 (0.9) | 6 (0.9) | 4 (0.7) | 80 (1.4) | 27 (0.5) |
| Previously recruited to the POPPI intervention, n (%) | 16 (0.4) | 16 (0.5) | 8 (1.2) | 2 (0.3) | 33 (0.6) | 24 (0.4) |
| Chronic cognitive impairment,b n (%) | 92 (2.5) | 49 (1.6) | 18 (2.7) | 14 (2.3) | 113 (1.9) | 97 (1.8) |
| Psychotic illness,b n (%) | 77 (2.1) | 76 (2.4) | 15 (2.3) | 15 (2.5) | 134 (2.3) | 134 (2.5) |
| Chronic PTSD,b n (%) | 9 (0.2) | 2 (0.1) | 1 (0.2) | 1 (0.2) | 23 (0.4) | 23 (0.4) |
| Did not meet transient criteria (n)a | 899 | 593 | 172 | 129 | 1412 | 1039 |
| Not able to communicate orally, n (%) | 384 (42.7) | 118 (19.9) | 54 (31.4) | 22 (17.1) | 426 (30.2) | 214 (20.6) |
| RASS score not between + 1 and –1, n (%) | 172 (19.1) | 76 (12.8) | 30 (17.4) | 11 (8.5) | 239 (16.9) | 158 (15.2) |
| GCS score of < 15, n (%) | 543 (60.4) | 304 (51.3) | 105 (61.0) | 59 (45.7) | 814 (57.6) | 563 (54.2) |
| Receiving end-of-life care, n (%) | 289 (32.1) | 248 (41.8) | 59 (34.3) | 48 (37.2) | 404 (28.6) | 394 (37.9) |
| Not able to consent, n (%) | 199 (22.1) | 81 (13.7) | 34 (19.8) | 21 (16.3) | 323 (22.9) | 191 (18.4) |
| Potentially eligible (n) | 508 | 533 | 97 | 99 | 866 | 858 |
| Missed (e.g. out-of-hours, no staff), n (%) | 96 (18.9) | 144 (27.0) | 17 (17.5) | 27 (27.3) | 194 (22.4) | 247 (28.8) |
| Other reasons not approached, n (%) | 32 (6.3) | 28 (5.3) | 4 (4.1) | 3 (3.0) | 46 (5.3) | 75 (8.7) |
| Approached for informed consent, n (%) | 380 (74.8) | 361 (67.7) | 76 (78.4) | 69 (69.7) | 626 (72.3) | 536 (62.5) |
| Eligible and approached for consent (n) | 380 | 361 | 76 | 69 | 626 | 536 |
| Declined consent, n (%) | 95 (25.0) | 76 (21.1) | 18 (23.7) | 27 (39.1) | 180 (28.8) | 194 (36.2) |
| Provided informed consent, n (%) | 285 (75.0) | 285 (78.9) | 58 (76.3) | 42 (60.8) | 446 (71.2) | 342 (63.8) |
| Withdrew consent, n (%) | 1 (0.4) | 2 (0.7) | 0 (0) | 0 (0) | 0 (0) | 2 (0.6) |
| Included in primary analysis (n) | 284 | 283 | 0 | 0 | 446 | 340 |
Of the patients who were deemed to be potentially eligible for the trial, the numbers of patients who were reported as being missed appeared to increase over time, in both usual care and intervention sites. The most commonly reported reason was that there was a relatively short time frame from a patient regaining capacity and being discharged from the critical care unit (the initial approach for consent was to be made in the unit). In addition, patients were often also missed because of a lack of resources to enable screening and recruitment in the evenings and at weekends. A similar trend was observed with regards to refusals of consent, with increasing refusals over time in both treatment groups, but higher rates of refusals being observed in patients at intervention group sites during the intervention period. The reported reasons for declining consent varied (but were consistent across treatment groups and time periods); often, patients felt too tired/fatigued to take part, others felt that they were not ‘up to’ completing questionnaires and that taking part in a research study would be burdensome to them. Some patients were already taking part in other studies and therefore did not want to consider a further study (even when co-enrolment was permitted between the studies).
Recruitment
A total of 1458 participants were recruited between 1 September 2015 and 3 February 2017. Figure 13 shows the cumulative patient recruitment against the anticipated pre-trial sample size and the revised minimum target. There was variation across the 24 participating sites in the rate of recruitment; the overall mean monthly recruitment rate was 4.1 (range 1.8–6.6) patients per site per month.
FIGURE 13.
Patient recruitment.
Patients were generally recruited into the POPPI cluster RCT during weekdays (Monday to Friday). Less than 10% of patients were recruited on either a Saturday or a Sunday (Figure 14). Most participating sites reported having insufficient resources to enable screening and recruitment outside usual office hours. Five participants (0.3%) withdrew consent, resulting in 1453 participants being included in the final sample. The 100 participants recruited during the transition (training) period were excluded from the primary analyses, as planned.
FIGURE 14.
Percentage of patients recruited by day of the week.
Patient characteristics
The groups were well matched across treatment groups and time periods (Tables 8–10). The mean age of patients was similar (baseline period: 57.2 years in the usual care group and 59.5 years in the intervention group; intervention period: 57.2 years in the usual care group and 60.4 years in the intervention group). The sample was predominantly white in ethnicity and ≈60% of recruited participants were male (this proportion was slightly lower at intervention group sites during the intervention period: 55% male).
| Characteristic | Baseline period | Intervention period | ||
|---|---|---|---|---|
| Usual care (N = 284) | Intervention (N = 283) | Usual care (N = 446) | Intervention (N = 340) | |
| Age (years) | ||||
| Mean (SD) | 57.2 (16.2) | 59.5 (16.0) | 57.2 (15.6) | 60.4 (15.0) |
| Median (IQR) | 60 (46–69) | 62 (48–72) | 58 (47–68) | 62 (51–70) |
| Sex, n (%) | ||||
| Female | 105 (37.0) | 115 (40.6) | 178 (39.9) | 153 (45.0) |
| Male | 179 (63.0) | 168 (59.4) | 268 (60.1) | 187 (55.0) |
| Ethnicity, n (%) | ||||
| White | 264 (93.0) | 254 (89.8) | 406 (91.0) | 320 (94.1) |
| Mixed | 1 (0.4) | 0 (0.0) | 2 (0.4) | 1 (0.3) |
| Asian | 3 (1.1) | 4 (1.4) | 6 (1.3) | 1 (0.3) |
| Black | 1 (0.4) | 7 (2.5) | 2 (0.4) | 3 (0.9) |
| Other | 0 (0.0) | 8 (2.8) | 4 (0.9) | 2 (0.6) |
| Not stated | 15 (5.3) | 10 (3.5) | 26 (5.8) | 13 (3.8) |
| Quintile of IMD 2015, n (%) | ||||
| 1 (least deprived) | 57 (20.1) | 41 (14.5) | 95/445 (21.3) | 57/338 (16.9) |
| 2 | 65 (22.9) | 46 (16.3) | 107/445 (24.0) | 74/338 (21.9) |
| 3 | 52 (18.3) | 56 (19.8) | 73/445 (16.4) | 76/338 (22.5) |
| 4 | 57 (20.1) | 71 (25.1) | 88/445 (19.8) | 73/338 (21.6) |
| 5 (most deprived) | 53 (18.7) | 69 (24.4) | 82/445 (18.4) | 58/338 (17.2) |
| Documented pre-existing anxiety/depression, n (%) | ||||
| Anxiety | 4 (1.4) | 3 (1.1) | 9 (2.0) | 12 (3.5) |
| Depression | 19 (6.7) | 19 (6.7) | 33 (7.4) | 32 (9.4) |
| Both | 8 (2.8) | 17 (6.0) | 13 (2.9) | 21 (6.2) |
| None | 253 (89.1) | 244 (86.2) | 391 (87.7) | 275 (80.9) |
| Characteristic | Baseline period | Intervention period | ||
|---|---|---|---|---|
| Usual care (N = 284) | Intervention (N = 283) | Usual care (N = 446) | Intervention (N = 340) | |
| Elective surgical admission, n (%) | ||||
| Yes | 24 (8.5) | 17 (6.0) | 37 (8.3) | 20 (5.9) |
| No | 260 (91.5) | 266 (94.0) | 409 (91.7) | 320 (94.1) |
| ICNARC Physiology Scorea | ||||
| Mean (SD) | 21.2 (7.1) | 21.1 (7.0) | 21.4 (7.2) | 21.0 (7.6) |
| Median (IQR) | 21 (16–25) | 21 (16–26) | 21 (17–26) | 20 (16–25) |
| APACHE II scorea | ||||
| Mean (SD) | 16.7 (5.8) | 16.9 (6.5) | 16.9 (6.2) | 17.7 (6.4) |
| Median (IQR) | 16 (13–20) | 16 (12–21) | 16 (13–21) | 17 (13–22) |
| Characteristic | Baseline period | Intervention period | ||
|---|---|---|---|---|
| Usual care (N = 284) | Intervention (N = 283) | Usual care (N = 446) | Intervention (N = 340) | |
| Duration of critical care unit stay prior to consent (days) | ||||
| Mean (SD) | 9.1 (7.4) | 9.8 (8.8) | 11.0 (11.6) | 12.1 (13.2) |
| Median (IQR) | 6 (4–11) | 7 (4–12) | 7 (4–13) | 7 (4–14) |
| CAM-ICU positive (delirium) days in critical care prior to consent | n = 113 | n = 162 | n = 180 | n = 147 |
| Mean (SD) | 2.3 (3.2) | 1.4 (2.2) | 2.8 (3.5) | 1.7 (3.0) |
| Median (IQR) | 1 (0–3) | 1 (0–2) | 2 (1–3) | 1 (0–2) |
| Days from critical care unit admission to consent | ||||
| Mean (SD) | 9.6 (7.5) | 10.5 (9.1) | 11.9 (11.7) | 13.2 (13.4) |
| Median (IQR) | 7 (4–12) | 7 (4–13) | 8 (5–14) | 9 (5–15) |
| Proportion of patients consented in critical care unit, n (%) | 236 (83.1) | 225 (79.5) | 337 (75.6) | 224 (65.9) |
| Last NEWS prior to consenta | ||||
| Mean (SD) | 3.1 (2.4) | 3.2 (2.2) | 2.8 (2.4) | 2.8 (2.1) |
| Median (IQR) | 3 (1–5) | 3 (2–5) | 2 (1–4) | 3 (1–4) |
| STAI-6 at time of consent | n = 284 | n = 282 | n = 445 | n = 340 |
| Mean (SD) | 43.6 (15.5) | 45.0 (16.0) | 42.1 (14.2) | 43.8 (17.1) |
| Median (IQR) | 43 (30–53) | 43 (33–57) | 43 (33–50) | 43 (30–55) |
| HRQoL (health thermometer score) at time of consent | n = 284 | n = 283 | n = 445 | n = 340 |
| Mean (SD) | 52.9 (23.3) | 52.4 (25.7) | 54.9 (23.3) | 51.0 (25.6) |
| Median (IQR) | 50 (40–70) | 50 (35–70) | 50 (40–70) | 50 (30–70) |
The majority of participants were medical admissions to the critical care unit, with a slightly higher proportion of planned surgical admissions at usual care group sites (than in intervention group sites). Patients’ severity of illness, assessed using the APACHE II scoring system and based on physiology variables during the first 24 hours of unit admission, was similar across treatment groups and time periods (median score of 16, and slightly higher at intervention group sites during the intervention period, with a median score of 17).
During the baseline period, patients were consented, on average, 9.6 (usual care group) and 10.5 (intervention group) days after admission to the critical care unit; compared with 11.9 and 13.2 days, respectively, during the intervention period. The proportion of patients consented in the unit, as opposed to on the wards after critical care discharge, was lower at intervention group sites during the intervention period than during all other trial time periods. At the time of consent, median STAI-6 and quality-of-life health thermometer scores were the same across treatment groups and time periods.
To assess case-mix representativeness, patients in the cluster RCT were compared with a best approximation of the POPPI eligibility criteria applied to the CMP database for both POPPI cluster RCT units and all other adult, general, critical care units participating in the CMP (non-POPPI units; Table 11). Patients who were recruited to the cluster RCT were representative of similar patients in the CMP, but were more likely to be white and not from the most deprived areas (according to quintile of IMD 2015). 76
| Characteristic | POPPI cluster RCT patients (N = 1453) | POPPI eligibility applied to the CMP dataa | |
|---|---|---|---|
| POPPI sites (N = 8189) | Non-POPPI sites (N = 50,208) | ||
| Age (years) | |||
| Mean (SD) | 58.0 (15.8) | 58.7 (17.1) | 59.3 (17.0) |
| Median (IQR) | 60 (48–70) | 61 (47–72) | 62 (48–73) |
| Sex (%) | |||
| Female | 41.2 | 39.6 | 41.0 |
| Male | 58.8 | 60.4 | 59.0 |
| Ethnicity (%) | |||
| White | 96.4 | 93.0 | 89.4 |
| Mixed | 0.3 | 0.3 | 0.7 |
| Asian | 1.2 | 3.0 | 4.8 |
| Black | 1.1 | 1.9 | 3.3 |
| Other | 1.0 | 1.8 | 1.9 |
| Quintile of IMD 2015 (%) | |||
| 1 (least deprived) | 18.0 | 16.2 | 14.6 |
| 2 | 21.0 | 17.8 | 17.4 |
| 3 | 19.2 | 19.9 | 19.4 |
| 4 | 21.8 | 21.5 | 22.6 |
| 5 (most deprived) | 20.0 | 24.6 | 26.0 |
| Elective surgical admission (%) | |||
| Yes | 7.1 | 7.8 | 6.7 |
| No | 92.9 | 92.2 | 93.3 |
| ICNARC Physiology Scoreb | |||
| Mean (SD) | 21.2 (7.3) | 20.8 (7.3) | 20.4 (7.2) |
| Median (IQR) | 21 (16–26) | 20 (16–25) | 20 (15–25) |
| APACHE II scoreb | |||
| Mean (SD) | 17.1 (6.3) | 17.4 (6.6) | 16.7 (6.4) |
| Median (IQR) | 17 (13–21) | 17 (13–22) | 16 (12–21) |
| Critical care unit length of stay (days) | |||
| Mean (SD) | 11.3 (13.5) | 9.3 (12.5) | 9.9 (12.0) |
| Median (IQR) | 6 (3–13) | 5 (3–10) | 5 (3–11) |
Patient follow-up
At 6 months, all participants were successfully followed up for mortality status. Of those surviving to 6 months, 79.3% completed the follow-up questionnaire (range 78.4–79.9% across treatment groups and time periods; Table 12). There were no differences in response rates between groups or time periods. The majority of questionnaires were completed on paper and returned via post.
| Follow-up | Baseline period | Intervention period | ||
|---|---|---|---|---|
| Usual care (N = 284) | Intervention (N = 283) | Usual care (N = 446) | Intervention (N = 340) | |
| Outcome at 6 months, n (%) | ||||
| Alive | 259 (91.2) | 245 (86.6) | 415 (93.0) | 314 (92.4) |
| Dead | 25 (8.8) | 38 (13.4) | 31 (7.0) | 26 (7.6) |
| Lost to follow-up | 0 (0.0) | 0 (0.0) | 0 (0.0) | 0 (0.0) |
| Returned questionnaire, n/N (% of those alive at 6 months) | ||||
| Completed | 203/259 (78.4) | 193/245 (78.8) | 331/415 (79.8) | 251/314 (79.9) |
| Refused | 22/259 (8.5) | 25/245 (10.2) | 31/415 (7.5) | 19/314 (6.1) |
| Lost to follow-up | 34/259 (13.1) | 27/245 (11.0) | 53/415 (12.8) | 44/314 (14.0) |
| Method of completion, n/N (% of completed questionnaires) | ||||
| Paper | 178/203 (87.7) | 173/193 (89.6) | 315/331 (95.2) | 240/251 (95.6) |
| Telephone | 25/203 (12.3) | 20/193 (10.4) | 16/331 (4.8) | 11/251 (4.4) |
| Method of refusal, n/N (% of those who refused) | ||||
| Paper | 13/22 (59.1) | 8/25 (32.0) | 15/31 (48.4) | 10/19 (52.6) |
| Telephone | 10/22 (45.5) | 17/25 (68.0) | 16/31 (51.6) | 9/19 (47.4) |
| Complete responses by instrument, n/N (% of completed questionnaires) | ||||
| PSS-SR | 201/203 (99.0) | 191/193 (99.0) | 330/331 (99.7) | 250/251 (99.6) |
| HADS anxiety | 196/203 (96.6) | 186/193 (96.4) | 327/331 (98.8) | 246/251 (98.0) |
| HADS depression | 196/203 (96.6) | 186/193 (96.4) | 327/331 (98.8) | 247/251 (98.4) |
| EQ-5D-5L | 197/203 (97.0) | 188/193 (97.4) | 320/331 (96.7) | 248/251 (98.8) |
| Health services | 135/203 (66.5) | 122/193 (63.2) | 226/331 (68.3) | 161/251 (64.1) |
Of returned completed questionnaires, data completeness for the primary outcome measure (PSS-SR score) was high (> 99%). Data completeness for the HADS67 and EQ-5D-5L56 were also high (> 96%), but less complete responses were received for the health services questionnaire (note that this questionnaire appeared last in the questionnaire booklet).
For participants alive at 6 months, response rates varied by patient characteristics (Table 13). Response rates were lowest in the youngest age group, and were generally lower for females than for males, for non-white ethnicities than for those of white ethnicity, and for patients with higher levels of deprivation. However, there appeared to be no clear relationship between response rate and either baseline acute severity of illness or anxiety and HRQoL at the time of consent. For patients receiving the intervention, the response rate increased with the number of stress support sessions received.
| Characteristic | Baseline period | Intervention period | ||
|---|---|---|---|---|
| Usual care (N = 259), n/N (%) | Intervention (N = 245), n/N (%) | Usual care (N = 415), n/N (%) | Intervention (N = 314), n/N (%) | |
| Age (years) | ||||
| 18–49 | 53/81 (65.4) | 50/68 (73.5) | 89/123 (72.4) | 47/68 (69.1) |
| 50–59 | 42/51 (82.4) | 41/50 (82.0) | 84/103 (81.6) | 61/73 (83.6) |
| 60–69 | 63/69 (91.3) | 57/67 (85.1) | 83/102 (81.4) | 77/91 (84.6) |
| ≥ 70 | 45/58 (77.6) | 45/60 (75.0) | 75/87 (86.2) | 66/82 (80.5) |
| Sex | ||||
| Female | 67/94 (71.3) | 77/104 (74.0) | 129/164 (78.7) | 113/145 (77.9) |
| Male | 136/165 (82.4) | 116/141 (82.3) | 202/251 (80.5) | 138/169 (81.7) |
| Ethnicity | ||||
| White | 190/240 (79.2) | 177/223 (79.4) | 298/377 (79.0) | 239/294 (81.3) |
| Non-white | 3/5 (60.0) | 9/14 (64.3) | 9/13 (69.2) | 2/7 (28.6) |
| Not stated | 10/14 (71.4) | 7/8 (87.5) | 24/25 (96.0) | 10/13 (76.9) |
| Quintile of IMD 2015 | ||||
| 1 (least deprived) | 50/54 (92.6) | 27/31 (87.1) | 79/89 (88.8) | 46/55 (83.6) |
| 2 | 45/55 (81.8) | 28/40 (70.0) | 84/97 (86.6) | 55/67 (82.1) |
| 3 | 32/48 (66.7) | 43/51 (84.3) | 52/70 (74.3) | 56/66 (84.8) |
| 4 | 40/51 (78.4) | 47/64 (73.4) | 63/83 (75.9) | 56/71 (78.9) |
| 5 (most deprived) | 36/51 (70.6) | 48/59 (81.4) | 52/75 (69.3) | 36/53 (67.9) |
| Pre-existing anxiety/depression | ||||
| Anxiety | 3/4 (75.0) | 0/1 (0.0) | 6/9 (66.7) | 9/11 (81.8) |
| Depression | 13/19 (68.4) | 13/18 (72.2) | 25/32 (78.1) | 24/31 (77.4) |
| Both | 4/8 (50.0) | 13/17 (76.5) | 9/11 (81.8) | 13/21 (61.9) |
| None | 183/228 (80.3) | 167/209 (79.9) | 291/363 (80.2) | 205/251 (81.7) |
| Elective surgical admission | ||||
| Yes | 16/19 (84.2) | 9/11 (81.8) | 29/36 (80.6) | 18/19 (94.7) |
| No | 187/240 (77.9) | 184/234 (78.6) | 302/379 (79.7) | 233/295 (79.0) |
| ICNARC Physiology Score | ||||
| < 17 | 59/73 (80.8) | 55/75 (73.3) | 83/105 (79.0) | 73/95 (76.8) |
| 17–21 | 56/75 (74.7) | 48/64 (75.0) | 89/118 (75.4) | 82/91 (90.1) |
| 22–26 | 44/55 (80.0) | 50/61 (82.0) | 82/103 (79.6) | 48/64 (75.0) |
| ≥ 27 | 44/56 (78.6) | 40/45 (88.9) | 77/89 (86.5) | 48/64 (75.0) |
| APACHE II score | ||||
| < 14 | 65/80 (81.3) | 58/79 (73.4) | 90/124 (72.6) | 65/89 (73.0) |
| 14–17 | 58/79 (73.4) | 58/71 (81.7) | 100/117 (85.5) | 72/89 (80.9) |
| 18–21 | 42/50 (84.0) | 37/45 (82.2) | 74/90 (82.2) | 52/61 (85.2) |
| ≥ 22 | 38/50 (76.0) | 40/50 (80.0) | 65/82 (79.3) | 62/75 (82.7) |
| CAM-ICU positive (delirious) in unit prior to consent (days) | ||||
| 0 | 19/33 (57.6) | 50/67 (74.6) | 20/29 (69.0) | 46/61 (75.4) |
| 1–2 | 33/41 (80.5) | 43/55 (78.2) | 65/84 (77.4) | 35/42 (83.3) |
| > 2 | 21/30 (70.0) | 17/22 (77.3) | 42/55 (76.4) | 30/35 (85.7) |
| Last NEWS prior to consent | ||||
| 0–1 | 56/77 (72.7) | 48/56 (85.7) | 124/157 (79.0) | 76/92 (82.6) |
| 2–3 | 67/81 (82.7) | 70/97 (72.2) | 109/130 (83.8) | 88/116 (75.9) |
| 4 | 26/35 (74.3) | 19/28 (67.9) | 25/37 (67.6) | 35/41 (85.4) |
| ≥ 5 | 54/66 (81.8) | 56/64 (87.5) | 73/91 (80.2) | 52/65 (80.0) |
| STAI-6 score at time of consent | ||||
| 20–30 | 54/71 (76.1) | 44/53 (83.0) | 81/103 (78.6) | 74/98 (75.5) |
| 31–43 | 58/70 (82.9) | 59/78 (75.6) | 111/145 (76.6) | 67/73 (91.8) |
| 44–53 | 44/58 (75.9) | 43/52 (82.7) | 78/96 (81.3) | 50/64 (78.1) |
| 54–80 | 47/60 (78.3) | 47/61 (77.0) | 60/70 (85.7) | 60/79 (75.9) |
| HRQoL health thermometer score at time of consent | ||||
| 0–38 | 49/62 (79.0) | 48/63 (76.2) | 71/86 (82.6) | 77/92 (83.7) |
| 39–50 | 66/82 (80.5) | 54/66 (81.8) | 97/125 (77.6) | 61/82 (74.4) |
| 51–70 | 41/53 (77.4) | 46/58 (79.3) | 80/102 (78.4) | 58/72 (80.6) |
| 71–100 | 47/62 (75.8) | 45/58 (77.6) | 83/101 (82.2) | 55/68 (80.9) |
| IPAT score of < 7 | – | – | – | 103/128 (80.5) |
| IPAT score of ≥ 7 by number of stress support sessions received | ||||
| None | – | – | – | 9/16 (56.3) |
| 1 | – | – | – | 13/19 (68.4) |
| 2 | – | – | – | 24/31 (77.4) |
| 3 | – | – | – | 102/120 (85.0) |
Multiple imputation
Table 14 reports all of the variables considered for multiple imputation, and for each variable, the number of missing values, and the imputation model chosen.
| Variable | Missing values, n (%) | Imputation model |
|---|---|---|
| Site-level covariates | ||
| Teaching status of hospital | 0 (0) | None required |
| Number of beds in the critical care unit | 0 (0) | None required |
| Number of critical care unit admissions receiving level 3 care staying ≥ 48 hours | 0 (0) | None required |
| Allocated treatment group | 0 (0) | None required |
| Patient-level covariates | ||
| Time period | 0 (0) | None required |
| Age | 0 (0) | None required |
| Sex | 0 (0) | None required |
| Ethnicity | 0 (0) | None required (‘not stated’ retained as separate category) |
| IMD 2015 quintile | 3 (< 0.1) | Singly imputed to category 3 (middle quintile) |
| Pre-existing anxiety/depression | 0 (0) | None required |
| Elective surgical admission | 0 (0) | None required |
| ICNARC physiology score | 0 (0) | None required |
| NEWS | 0 (0) | None required |
| HRQoL health thermometer score | 1 (< 0.1) | Singly imputed to mean |
| STAI-6 score | 2 (< 0.1) | Missing items singly imputed to mode |
| Duration of stay in the critical care unit (days) | 0 (0) | None required |
| Number of days of delirium | 0 (0) | None required (not assessed retained as separate category) |
| Number of days receiving sedatives/anxiolytics/anaesthetics | 0 (0) | None required |
| Number of days receiving sleep medications | 0 (0) | None required |
| Receipt of benzodiazepines | 0 (0) | None required |
| Number of days receiving antipsychotics | 0 (0) | None required |
| Number of days receiving vasoactive agents | 0 (0) | None required |
| Number of days receiving analgesics | 0 (0) | None required |
| Number of days receiving antidepressants | 0 (0) | None required |
| Number of days receiving mechanical ventilation | 0 (0) | None required |
| Duration of stay in hospital following discharge from the critical care unit (days) | 0 (0) | None required |
| Adherence to intervention | 0 (0) | None required |
| Length of stay in general medical wards (days) | 0 (0) | None required |
| Outcomes and resource use at 6 months | ||
| Costs of ICU stay (£) | 0 (0) | None required |
| Mortality | 0 (0) | None required |
| PSS-SR score | 283 (21.4) | Predictive mean matching |
| HADS anxiety score | 303 (22.9) | Predictive mean matching |
| HADS depression score | 302 (22.8) | Predictive mean matching |
| EQ-5D-5L health utility | 302 (22.8) | Predictive mean matching |
| Health services questionnaire costs (£) | 631 (47.7) | Predictive mean matching |
Chapter 6 Cluster randomised clinical trial process evaluation: methods and results
Introduction
The POPPI cluster RCT evaluated a preventive, complex (multicomponent) psychological intervention, delivered across multiple critical care units. Trials evaluating interventions that have several component parts can pose methodological challenges in understanding the effect of the intervention overall, as well as separating the effect of one component from another. In addition, when delivered across multiple units (such as in this case), they are susceptible to variation in how the intervention is implemented and delivered, potentially affecting patient outcomes and producing varying degrees of treatment effect between units. The complexity of the cluster RCT necessitated an evaluation of these processes. In the context of this process evaluation, we evaluated the three core elements (of which there are four components) of the POPPI intervention:
-
Element 1: creating a therapeutic environment in critical care.
-
POPPI online training.
-
Translating knowledge from the online training into practice.
-
-
Element 2: three stress support sessions for patients identified as acutely stressed.
-
Identifying acutely stressed patients with the IPAT, and delivering stress support sessions.
-
-
Element 3: relaxation and recovery programme for patients identified as acutely stressed.
-
Take-home booklet and DVD offered to all patients identified as acutely stressed.
-
The first element of the intervention, creating a therapeutic environment in critical care, facilitated primarily by an online training course, was a cluster (site)-level intervention for all critical care staff to participate in; thus, all patients would be exposed to its potential effects. The second and third elements (stress support sessions and relaxation and recovery programme) were patient-level interventions, delivered to patients identified as acutely stressed (indicated by a score of ≥ 7 points on the IPAT). The timeline for exposure to the POPPI intervention is outlined in Figure 15.
FIGURE 15.
Timeline for exposure to the POPPI intervention. R&R, relaxation and recovery programme; SSS, stress support session(s).
Aims
The aims of the process evaluation were to:
-
assess the uptake and level of adherence to the POPPI intervention
-
understand factors affecting the successful delivery of the POPPI intervention and cluster RCT
-
uncover the relationship between the variation in the POPPI intervention delivery and the POPPI cluster RCT primary outcome.
The process evaluation also provides increased understanding of the barriers to and facilitators of the wider cluster RCT delivery and augments understanding of any observed differences between units. This chapter outlines the methods of how this evaluation was undertaken and presents the results of aims one and two. The results of aim three are reported in Chapter 7.
Methods
Objectives
Guided by Steckler and Linnan’s86 recommendations for six key process evaluation components (fidelity, dose, reach, implementation, context and recruitment), the following objectives were developed to address the aims of the process evaluation.
Aim 1: to assess the uptake and level of adherence to the POPPI intervention
-
To what extent is the POPPI intervention delivered as intended (fidelity)?
-
How much of the intended POPPI intervention is delivered and received (dose)?
-
What proportion of intended recipients actually participated in the POPPI intervention (reach)?
Aim 2: to understand factors affecting the successful delivery of the POPPI intervention and cluster RCT
-
Which aspects of the unit culture influenced how the POPPI intervention and cluster RCT were implemented (context)?
-
What procedures were used to ensure that patients were recruited to the cluster RCT (recruitment)?
Aim 3: to uncover the relationship between the variation in POPPI intervention delivery and the cluster RCT primary outcome
-
Is there a difference in the primary outcome from lowest to highest overall intervention adherence (implementation: an overall weighted composite score summing fidelity, dose and reach)?
Logic model
The logic model for the POPPI intervention is a diagrammatical representation of the theory of how the intervention is intended to work (see Figure 1). In the context of the cluster RCT, the logic model illustrated the main elements of the POPPI intervention, how they were intended to interact to produce change, the expected outcomes, and the resources and mechanisms needed to ensure successful delivery, implementation and, therefore, outcomes. Constructing a logic model is useful in that it may uncover weaknesses in the assumptions, identify at which points clinicians may have conflicting understandings of the intended intervention, or help evaluators to think critically about potential unintended consequences. 87 A semistructured interview guide and structured observational data collection form were designed to gather information that could be used to test the assumptions of the logic model, alongside routinely collected cluster RCT data.
Data sources
The process evaluation adopted a mixed-methods approach drawing on data collected from a number of sources (Table 15).
| Qualitative | Quantitative |
|---|---|
| Observational data collected on site | Routinely collected cluster RCT data (see Chapter 4 for details) |
| On-site face-to-face interviews | |
| Telephone interviews |
The process evaluation involved field observation and discussion of the cluster RCT and the POPPI intervention with unit staff, so it was important that the researcher was sufficiently independent to minimise the introduction of bias into the evaluation, and for it to remain credible. 87 This independence ensured that the external researchers would neither view the intervention too positively, nor be unduly critical. The relationship between the process evaluation team and the wider trial management team was defined at the planning stage.
The evaluation was conducted in three phases, spanning the duration of the cluster RCT recruitment period from the beginning to the end. It was necessary to collect baseline data from the beginning of the trial (phase I) as contextual factors have the potential to influence how an intervention is implemented and may act as barriers to or facilitators of intervention implementation and delivery. 88 By understanding baseline contextual differences between units, it would be possible to elucidate how these may have influenced trial implementation and delivery. Collecting these data during the baseline period also ensured that it was not influenced by treatment allocation [as individual sites were randomised in their second month of recruitment (during the baseline period) – after the phase I site visits].
Collecting during-trial (phase II) data, after the intervention sites had transitioned to delivering the intervention, was key to understanding how the POPPI intervention and cluster RCT were being implemented and delivered within sites, and in identifying emerging themes regarding barriers and facilitators that affected this. Collecting during-trial data was also a fundamental step in the development of both the interview guides and the analytical framework to be used at the end of the cluster RCT.
It was essential that sites were then followed up at the end of the cluster RCT (phase III) to fully understand the uptake and the level of adherence to the POPPI intervention. This allowed for in-depth exploration with those working closely within the cluster RCT, to explore how and why the intervention was implemented, delivered and sustained over time (or not), and to uncover the relationship between the variation in intervention delivery and the cluster RCT primary outcome. Routinely collected cluster RCT data were collected throughout the entire trial trajectory and integrated with the qualitative data throughout.
Phase I
At the start of the patient recruitment period, during the baseline period, contextual data were gathered using individual interviews and field observation at all 24 sites. This information detailed the baseline psychological support for patients and environmental factors that may have affected a patient’s psychological well-being in the critical care unit.
Participants
Participants were the site PIs at all 24 sites. The PIs were chosen as they had primary responsibility for local management of the POPPI intervention and had in-depth knowledge of relevant local critical care unit practice and protocols. The PIs accompanied the researcher onto the unit to undertake a period of field observation.
Recruitment and consent
The PIs were informed of the process evaluation format by the trial management team at the site initiation visits (see Chapter 4 for details). PIs were told that the researcher would contact them directly to arrange the phase I visit. The researcher contacted all PIs, via e-mail, to invite them to participate in individual interviews at a time convenient to them and to request access to the unit to undertake the period of field observation. PIs were advised to check local NHS trust policies and inform the researcher of any pre-visit access checks/approvals that were mandated.
Individual interview data collection
The researcher used a semistructured interview guide to conduct individual face-to-face interviews with the PIs at each site (see Appendix 2 for the semistructured interview guide). Interviews lasted 45–60 minutes, during which all information was recorded in field notes. The data collected encompassed five categories:
-
Family care: what was usual practice in the unit regarding family/visitors? Were they included in patient care, and was there any information/support available to them?
-
Sleep: what steps were taken to manage patients’ sleep in the unit? Were there any non-pharmacological interventions or sleep protocols in use?
-
Pain, agitation, delirium, sedation (PADS): was there a PADS protocol in use? How was delirium managed, and what training did staff get?
-
Psychological support: were patients psychologically assessed on the unit, and was there any psychological support or follow-up available?
-
Management of serious incidents: how were patients protected from witnessing serious incidents on the unit, and were they debriefed afterwards?
Field observation data collection
A period of observation was conducted on each unit, which included observing part of a morning ward round. This lasted 45–60 minutes and observations were recorded using the data collection form, structured to capture relevant observable information (see Appendix 3 for a field observation data collection form). The observational data collected during Phase I encompassed five categories:
-
Patient orientation: what was being done on the unit to orientate patients to night and day?
-
Unit layout: were bed spaces cramped or cluttered, and was there anything of visual interest to patients?
-
Communication: how did staff address/include patients, and were patients encouraged to ask questions?
-
Unit noise: what was the noise level like on the unit (alarms, telephones, staff), and what efforts had been made to reduce noise?
-
Relaxation and distraction: was there any stimulation or distraction for patients (TV, radio, books, complementary therapies)?
Data analysis
Both the individual interview data and observational data collected from each unit were factual information collected in prespecified categories, so these were organised and tabulated for later cross-examination of baseline similarities/differences between sites. These data were used to understand how baseline differences between units affected the cluster RCT and the POPPI intervention implementation and delivery.
Phase II
To understand how the POPPI intervention and cluster RCT were being implemented in sites and to begin to identify barriers to and facilitators of successful delivery, during-trial exploratory telephone interviews were conducted for all intervention group sites. In addition, routinely collected cluster RCT data pertaining to patient recruitment and intervention delivery were summarised by the trial management team in the form of individual unit profiles and sent to the researcher to help understand how sites were engaging with the cluster RCT and the POPPI intervention. During-trial exploratory telephone interviews were also conducted for all usual care group sites, to explore barriers to and facilitators of cluster RCT implementation and delivery and to discover whether or not control sites remained engaged with the trial.
Telephone interviews
Participants
The trial management team identified a study driver at each site. The study drivers were PIs, POPPI nurses or research nurses, and had been involved in trial implementation, drove the project, and were involved in the POPPI intervention (at intervention group sites) and/or cluster RCT delivery. The study driver was chosen as the telephone interview participant, as they had a wealth of information about how the cluster RCT was accepted and worked in practice.
Recruitment
The researcher was provided with a list of study drivers for all units. At this stage of the trial, the researcher was familiar with staff across all units, both through meeting in person at the phase I visits and through observing monthly routine cluster RCT site teleconferences. The study drivers were contacted by the researcher via e-mail, inviting them to take part in a one-to-one telephone interview at a time convenient to them. They were provided with an overview of the topics that would be discussed in the interview, to enable them to reflect and make note of any relevant areas for discussion.
Data collection
Semistructured interview guides (see Appendices 4 and 5) were used to elicit information on emerging issues and themes, guided by the unit profiles and the cluster RCT logic model (see Figure 1). The interviews explored how the cluster RCT and the POPPI intervention had been received in the units, issues surrounding patient recruitment, how the cluster RCT and the POPPI intervention fitted into a given context (including fidelity) and the impact that it had on ‘routine’ clinical workload. Field notes were taken during the telephone interviews. The semistructured interview data provided qualitative information on the fidelity of intervention delivery across sites and explored the effect of contextual/external factors on implementation and identified barriers to and facilitators of adherence to the individual elements of the POPPI intervention.
During the telephone interviews, staff were encouraged to elaborate on both positive and negative aspects of the cluster RCT to date. This allowed key issues to emerge and enabled the preliminary identification of barriers to and facilitators of the intervention components. Cluster RCT implementation and delivery was explored during interviews with control site staff, including understanding any changes in engagement after sites were randomised.
Data analysis
A content analysis of the interview field notes was undertaken. Content analysis was chosen as it is a straightforward method of analysis, used to determine the presence of concepts within text by breaking that text down into manageable categories. In accordance with Krippendorf’s89 guidance, this began with familiarisation of the notes from the telephone interviews and making note of any relevant or interesting information in the margins. All information in the margins was then categorised, with descriptions given for each category. Categories were examined to see if there was any overlap, or if they could be merged into major and minor themes. The process was applied to each set of interview notes, after which the original notes were rechecked to ensure that all information had been categorised. The content analysis was double-checked by another independent member of the research team, after which the key themes were condensed and merged, resulting in a concise set of themes and subthemes. Once analysed, these data were used to inform the development of the final Phase III interview schedule, guiding appropriate areas for further exploration.
Routine cluster randomised clinical trial data
The trial management team provided the researcher with individual site profiles indicating performance and engagement within the cluster RCT prior to the conduct of telephone interviews. These included free-text comments detailing any emerging issues at the site, as well as considering team involvement and communication. For intervention group sites, the unit profiles also provided information to evaluate the dose and reach of the POPPI intervention. The unit profiles comprised the following:
-
screening and recruitment figures for the baseline period and transition and intervention periods (when relevant)
-
monthly patient recruitment figures
-
uptake of the POPPI intervention online training over time
-
number of patients assessed with the IPAT
-
number of stress support sessions received by patients identified as acutely stressed
-
number of patients receiving the booklet and/or DVD (as part of the relaxation and recovery programme).
Phase III
Towards the end of the cluster RCT patient recruitment, interviews were conducted at site visits with both PIs and POPPI nurses at all intervention group sites. Convenience samples of staff were also informally interviewed at all intervention sites. Routinely collected cluster RCT data pertaining to patient recruitment and intervention delivery were also used to explore indicators of unit engagement including uptake of the online training, and stress support sessions delivered. In addition, interviews were conducted at a purposively selected sample of usual care group sites, the purpose of which was to examine whether or not these units did anything different from baseline in terms of psychological support provided to patients.
Individual interviews
Participants
At intervention group sites, the participants were site PIs and POPPI nurses (which included all study drivers), chosen because of their close involvement in the cluster RCT and the POPPI intervention implementation and delivery. A convenience sample of critical care unit staff was also chosen on the day of the site visit, with whom the POPPI intervention online training was discussed. These staff must have had completed the POPPI intervention online training and be willing to discuss how it affected their practice. At the usual care group sites, interviews were conducted with PIs and research nurse(s) involved in cluster RCT screening and recruitment.
Recruitment
As the key unit contact for the cluster RCT, the study drivers were contacted by the researcher, via e-mail, and invited, along with the other PIs and POPPI nurses, to participate in a final comprehensive site visit by the researcher. The Phase III visit had been discussed during the Phase II telephone interviews, so the study driver was willing and had agreed to co-ordinate this visit. They liaised with critical care unit staff and agreed a date for the researcher to visit to conduct the final phase of interviews.
Data collection
A standardised interview schedule was devised using information collated from Phase I and Phase II data for uniform use across all intervention group sites (see Appendix 4). The interviews took place before the participants and the researchers knew the result of the cluster RCT, to avoid this knowledge influencing views about the cluster RCT or the POPPI intervention.
Any change in practice from baseline at usual care sites had the potential to close the difference between the effect of the intervention and usual care, and produce a null result;90 for this reason, a purposive sample of usual care group sites were visited in Phase III. The sampling of usual care sites was informed by the Phase II telephone interview data. Sites were selected on the basis of suggestions of a change in practice of psychological support from baseline. It was also anticipated that there may be a negative effect of usual care site allocation on enthusiasm and attitudes towards the study. This had the potential to affect the trial delivery at control sites, and clinician willingness to screen and recruit patients into the cluster RCT. A modified version of the standardised interview schedule was used in four purposively sampled usual care group sites (see Appendix 5). These interviews explored clinician experiences and understandings of the cluster RCT in the context of their critical care unit.
At all sites, one-to-one interviews were conducted with unit PIs, POPPI nurses (where relevant) and research nurses. All interviews were digitally recorded, audio files were stored on a password-protected drive, and audio files were erased after transcription. Audio files were transcribed in full, removing any identifiable information, and saved in Microsoft Word format. The transcripts were stored on a password-protected computer and each participant had a unique identification number.
Nine themes were explored in detail at the intervention group sites using the semistructured interview guide (see Appendix 6) aimed at understanding:
-
the fidelity, dose, and reach of the POPPI intervention, and using this to inform the overall implementation score
-
which aspects of the unit culture influenced how the POPPI intervention and cluster RCT were implemented
-
what procedures units used to ensure that patients were recruited into the cluster RCT.
Seven themes were explored in detail at the usual care group sites using the semistructured interview guide (see Appendix 7) aimed at understanding:
-
which aspects of unit culture influenced how the cluster RCT was implemented
-
what procedures units used to ensure that patients were recruited into the cluster RCT.
A convenience sample of unit staff (2 or 3 members of the multidisciplinary team) was selected on the day of the intervention unit visit at each unit, with whom the POPPI online training was discussed. These interviews were informal and field notes were taken, with questioning focused on:
-
identifying barriers to and facilitators of undertaking the POPPI online training
-
opinions of the POPPI online training content
-
whether or not the POPPI online training was something that staff felt that they had had the opportunity/was feasible to use in practice.
Data analysis
Interview data were thematically analysed91 using a seven-stage framework approach92 that allowed simultaneous analysis across themes and cases. The approach is a matrix-based method for analysing data that includes familiarisation with the data, indexing the data and creating summaries from the indexed data. An important feature of the approach is that it allows themes or concepts identified a priori to be specified as indexing categories from the outset (deductive analytical framework) and to be combined with other themes or concepts that emerge de novo by subjecting the data to inductive analysis (revising the data to see if anything has been missed).
A deductive approach was taken as existing ideas and themes were identified in the phase II interviews, so the content of the phase III interviews was broadly anticipated. The deductively developed data extraction framework (see Appendix 8) was applied to all intervention group site interview transcripts. A modified version was used for usual care group site interview transcripts (see Appendix 9). The data extraction frameworks were developed by the lead process evaluation researcher and reviewed by the wider research team. They were developed to systematically extract all data pertaining to the process evaluation aims.
To ensure confirmability and trustworthiness, the framework was piloted on a random sample of five interview transcripts. A second independent member of the research team extracted data from the same five transcripts. Once this stage was complete, the researchers convened to discuss the functionality of the framework in extracting all of the data. It was evident that some interview data remained unallocated to the framework, so the thematic framework was reviewed as it was applied to the data and revised a further two times until all data could be fitted in the thematic domains of the framework. Finally, the themes were mapped, and interpretation of those themes reviewed by the two researchers, to construct overall explanations of the data. This approach is less focused on producing new theory, but more about shaping and exploring existing ideas.
Routine cluster randomised clinical trial data
Routinely collected cluster RCT data were examined at the end of phase III, to evaluate the dose and reach of each intervention element. These data comprised:
-
the POPPI online training uptake
-
stress support session delivery
-
use of the relaxation and recovery programme
-
delivery of debriefing and support visit/calls with the POPPI nurses.
Implementation score
The inclusion of ‘implementation’ as a composite score allows for the grading of intervention group sites according to their degree of overall intervention delivery. This experimental approach was adopted to explore the nature of the relationship between the variation in the POPPI intervention delivery and the cluster RCT primary outcome. Implementation grading was developed using a combination of fidelity, dose and reach. Phase III qualitative interview data were distilled into scores, as were the phase III routine cluster RCT data, with the aim of grading sites into lowest, moderate or highest adherers.
Qualitative interview data
Interview data were considered for assessing the fidelity of intervention elements one (creating a therapeutic environment in critical care) and two (three stress support sessions for patients identified as acutely stressed). The fidelity to each of these elements was scored based on the unit fulfilling criteria to achieve one of the following scores:
-
3 = Full adherence: element delivered as intended; full effort made to follow recommendations/guidance.
-
2 = Mostly adhering: element predominantly delivered as intended; decent effort made to follow recommendations/guidance.
-
1 = Some adherence: element often not delivered as intended; mediocre effort made to follow recommendations/guidance.
-
0 = Low adherence: element rarely delivered as intended; poor/no effort made to follow recommendations/guidance.
To ensure inter-rater reliability, two researchers independently scored the same site, converting interview data into scores. They discussed any discordance between their scoring of the data and reached consensus on how to proceed with the scoring of a further two sites. Again, both researchers scored the interview data independently, and agreed on the allocated scores. Both researchers scored the remaining nine sites together. Sites that had the same score for any given element were cross-checked to ensure that the same standard had been applied to the scoring process throughout.
Routine cluster randomised clinical trial data
Routinely collected cluster RCT data were also examined at the end of phase III, to evaluate and score the dose and reach of the POPPI intervention, and comprised the following:
-
Percentage of clinical critical care unit staff completing the POPPI online training.
-
Time taken to reach 80% uptake (target) of the POPPI online training.
-
Percentage of patients receiving two or more stress support sessions.
-
Percentage of patients who scored ≥ 7 on the IPAT (i.e. were acutely stressed) who received no sessions.
-
Percentage of patients using the tablet computer as part of the stress support sessions.
-
Percentage of patients receiving the relaxation and recovery programme (DVD and/or booklet) to take home.
Integrating mixed-methods data
The interview data were combined with the routine cluster RCT data so that all three elements (four components) of the POPPI intervention had a score. The scores address fidelity, dose and reach of the POPPI intervention (Table 16). Component scores were combined giving units an overall composite score for implementation and ranking. Table 16 shows how the mixed-methods data were pooled to score (1) adherence to each intervention element and (2) overall intervention adherence. To ensure that each element of the intervention was weighted to account for its anticipated importance, the POPPI investigators were asked to independently weight the components using a total of 15 points each, representing probable overall importance.
| Intervention components | Component adherence scoringa | |||
|---|---|---|---|---|
| Dose | Reach | |||
| 1. Creation of a therapeutic environment: the POPPI online training | Time to achieving 80% uptake:
|
% Staff completing the POPPI online training:
|
||
| Fidelity | ||||
| 2. Creation of a therapeutic environment: translation of knowledge into practice | Qualitative interview:
|
|||
| Fidelity | Dose | Reach | Reach | |
| 3. IPAT assessments and stress support sessions | Qualitative interview:
|
Percentage of patients receiving two or more stress support sessions:
|
Percentage of patients with an IPAT score of ≥ 7 receiving 0 stress support sessions:
|
Percentage of patients who reported using the tablet computer at sessions:
|
| Reach | ||||
| 4. Relaxation and recovery programme | Percentage of patients receiving relaxation and recovery programme to take home (either DVD or booklet given):
|
|||
| Composite score | ||||
| Implementation (composite score) | Component 1 (0–6) | Component 2 (0–3) | Component 3 (0–12) | Component 4 (0–3) |
| Total score (0–24) | ||||
| Site ranking (1–12) | ||||
The results from aim three are reported in Chapter 7 as part of the subgroup analyses.
Results
Visits and interviews
Phase I visits were conducted within the first month of the recruitment period at all 24 participating sites. The visits were conducted during the baseline period, in which all sites were delivering usual care and had not yet been randomised. Interviews were conducted at each site with one or both site PIs. In the instance that both PIs were present, they were interviewed together. Interviews lasted 45–60 minutes, and a total of 18 PIs participated in the phase I interviews. A period of field observation was spent on all 24 units during the site visit, ranging from 45 to 60 minutes. All field observation was conducted in the presence of one site PI. On one unit, the ward round could not be observed, but this was discussed in detail with the medical PI. Phase I confirmed that all 24 intervention and control units had no routine psychological assessment or formal support for patients. The bedside nurses provided emotional support for patients on an ad hoc basis. Follow-up clinics were in place at 15 of the 24 sites, but these were used in inconsistent ways across the sites and targeted a minority of patients. Criteria for invitation to attend the follow-up clinics varied, and included a requirement to have been mechanically ventilated for anything from 24 to 96 hours; variable lengths of intensive care unit (ICU) stay regardless of mechanical ventilation duration; only patients receiving level 3 care; and one unit where there were no set criteria (so the outreach team chose who to invite).
Phase II telephone interviews were conducted in month 9 at all 24 sites, by which time intervention units had been delivering the intervention for 3 months post-transition period (usual care groups continued to deliver usual care). Telephone interviews were conducted with one study driver at each of the sites and ranged in duration from 20 to 30 minutes. In total, 24 study drivers participated in the phase II interviews. The themes and subthemes from the phase II content analysis (see Appendix 10) were used to inform the development of the final phase III interview schedules.
Phase III site visits were conducted in month 11 of the intervention period, at all 12 intervention sites and at a purposively selected sample of four control sites. Individual interviews were conducted at each site, comprising at least one PI and at least two POPPI nurses at intervention sites, and the research nurse(s) and at least one PI at control sites. In total, 44 staff participated in the phase III intervention group interviews. Convenience samples of unit staff were also informally interviewed during the site visit, with whom the online training was discussed. Two or three members of the multidisciplinary team participated in these interviews at each site and a total of 28 informal interviews were conducted for this purpose across the intervention group sites. The professional groups involved in these interviews incorporated physiotherapists, occupational therapists, doctors and nurses. At the four control sites, 11 staff participated in the phase III interviews. A total of 83 interviews were conducted during phase III.
Aims one and two: dose, reach and fidelity (implementation); context; and recruitment
Element 1: creating a therapeutic environment in critical care
Dose
To facilitate the creation of a therapeutic environment in critical care, the POPPI online training course was rolled out at intervention group sites at the start of the transition period. The POPPI online training was to be completed by all clinical critical care staff, against a target of 80% of staff completing the course.
Expressed as a percentage of staff completing the POPPI online training out of all staff working in the units in the transition month, the median uptake among critical care staff at intervention group sites (n = 12) at the end of the transition period was 58% (IQR 49–69%) and all sites achieved the 80% minimum target within 4 months (Table 17).
| Month | Sites achieving 80% uptake (N = 12), n (%) |
|---|---|
| Transition month | 1 (8.3) |
| Intervention month 1 | 3 (25.0) |
| Intervention month 2 | 5 (41.7) |
| Intervention month 3 | 3 (25.0) |
Participants at all intervention group sites were asked at interview what they considered to be the main facilitators of and barriers to meeting the 80% target for online training. There was consensus among all sites that relieving staff nurses from the bedside to complete the training was the main expediter of achieving the target. This was possible at 11 intervention group sites where at least one POPPI nurse had clinical critical care experience and could take over patient care to allow staff nurses time off the unit to complete the training:
. . . I had to be more hands-on and actually go out and say ‘I’ll look after your patient while you go and do the training’.
R-30
At the twelfth unit, this was not used as a strategy, as the POPPI nurses were constrained by their clinical roles. The POPPI nurses also discussed how the use of mobile or tablet computers enabled staff nurses to complete the training during quieter periods of their shift while remaining at the bedside; this was particularly convenient during nightshifts.
A conflicting/high clinical workload was reported as the main barrier to undertaking the POPPI online training. Participants felt that the existing burden of other mandatory training for clinical staff impeded the time available to complete the non-mandatory POPPI online training:
I think it’s just because it was an extra thing to do. You come into work, you’ve got a busy patient, people are nagging you to go spend half an hour on a computer to do an online training, and they’ve already got all the mandatory online training things.
R-12
Reach
All intervention group sites continued to enumerate staff members throughout the intervention period to ensure that new staff (e.g. rotations of junior doctors) received the POPPI online training and that leavers were no longer included in the monthly figures. Figure 16 shows the median uptake of the POPPI online training across intervention group sites from the transition period until the end of the intervention period. Figure 17 shows the monthly uptake for each of the intervention group sites (n = 12) across the same period.
FIGURE 16.
Median POPPI online training uptake across intervention group sites (n = 12) from the transition month until the end of intervention period.
FIGURE 17.
Monthly POPPI online training uptake at each intervention group site (n = 12) from the transition month until the end of intervention period. Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
During the interviews, the POPPI nurses discussed promoting awareness of the online training by displaying posters and bunting throughout the unit, and using incentives such as badges, pens and cakes to encourage participation. Several units incorporated the online training course into new staff induction programmes as an effective way to ensure high coverage despite staff turnover. The POPPI nurses reported that generic group e-mails were often unsuccessful in encouraging online training completion, but that taking an individualised approach and e-mailing staff (for whom the training was uncompleted) proved effective in improving the reach of this component.
A small number of participants reported poor information technology (IT) access while at work, meaning that they had to be willing to complete the training in their own time, usually at home. This also obstructed access to POPPI-related e-mails. However, the POPPI nurses stated that the majority of staff were amenable to this strategy and it did not appear to have any bearing on uptake rates. For those who did have IT access in work, computers occasionally lacked audio, preventing users from interacting with one element of the training (patient experience and communication videos); however, transcripts were available.
Fidelity
To assess the fidelity to the creation of a therapeutic environment it was important to explore in participant interviews whether or not those who completed the online training applied learning from it in their clinical practice, and what factors affected this.
Education by the POPPI nurses and/or local PIs emerged as key in translating the online training into practice. These individuals acted as study champions and used various mediums to reinforce the online training messages and encourage their application in clinical practice. There was evidence of study champions at half of the intervention sites. These study champions conducted informal bedside teaching with staff nurses, distributed newsletters and disseminated key messages and reminders throughout the unit, using posters and noticeboards. Participants in these units reported widespread multidisciplinary engagement and enthusiasm among the team. Suggestion boxes were used at one unit as a way of encouraging staff to contribute ideas for improving the psychological care of patients, enforcing the learning from the POPPI online training. To ensure that the online training was applied in practice, another unit initiated workstreams, developed to ensure implementation of strategies to address delirium, sedation, sleep optimisation and noise reduction:
. . . this sort of intervention is not something you can do as a one-off. It’s a sustained process, isn’t it, of trying to keep reminding people about, not just the trial, but the whole belief that’s behind keeping quiet at night, turning alarms off, let people sleep and so on and so forth.
R-07
The remaining half of intervention sites had no evidence of a study champion. In the absence of study champions, participants reported less engagement from a small number of staff nurses, manifested through resistance to the online training and an unwillingness to use the principles to improve practice. There appeared to be a passive approach to education at these units and educational efforts were limited to posters and signage in the units. Participants from these units reported a perception of the online training as a ‘tick-box’ exercise, which needed to be completed but was not particularly influential on their current practice. Some of the POPPI nurses stated that a small number of senior staff nurses felt insulted at being made to undertake training that taught them basic patient care, and this contributed towards an attitude of hostility. Despite this apparent lack of leadership and engagement at these units, the online training uptake target was reached on time:
It was ages ago that I did it, but it was fine and everyone’s done it . . . I mean, I don’t know how much impact it has but, yeah, we’ve all done it . . . You do it and . . . then you move on.
R-09
In all 12 intervention units, participants reported that the mechanistic elements were easier to implement than those elements that challenged long-standing unit culture. During the period of field observation, it was noted that units had readily instated clocks, sleep aids and soft-close bins (where they had been absent). Patient orientation, optimisation of sleep, and noise reduction were the main elements targeted for improvement among all units. However, all units also reported a very small number of staff who were resistant to any practice change in the unit, and this was not confined to the POPPI intervention. For example, large ward-rounds held at the end of the patient bed (conflicting one of the recommendations from the POPPI online training on communication) and regular multiple disturbances to the patient during the night (conflicting the training on sleep optimisation) were reported as practices that were most challenging to improve:
. . . I think you can’t underestimate how much nurses like routine and how difficult it is to get cultural change.
R-25
I think that’s an inbred thing here, always done for years, is ‘turn the lights down in the afternoon for people to have a little nap . . . The patients need rest, they need a nap.’ You try and explain the evidence, but no.’
R-28
. . . they still wander around in little groups and there are some consultants who will do it in the night-time as well, do their ward rounds at 3 o’clock in the morning. I don’t think it’s changed their practice at all.
R-41
Participants at two units reported difficulty translating the online training into practice because of the physical confines of the unit. In particular, they felt that the size or layout of the unit constrained their ability to implement the recommended environmental changes, especially in relation to noise reduction and finding suitable locations to place clocks and whiteboards within patients’ lines of sight.
Element 2: three stress support sessions for patients identified as acutely stressed
Dose and reach
A total of 340 patients were recruited at intervention group sites during the intervention period. Of these, 27 patients provided consent to receive only a follow-up questionnaire, resulting in 313 patients eligible for assessment with the IPAT (and subsequent receipt of the stress support sessions, if applicable).
All 313 (100%) patients consenting for an IPAT assessment were assessed. Of these, 199 patients (63.6%) were assessed as being acutely stressed and at a high risk of psychological morbidity (i.e. an IPAT score of ≥ 7 points) and, therefore, eligible to receive the three stress support sessions. The median IPAT score was 8 points (IQR 4–13 points).
Of the 199 patients assessed as acutely stressed, 127 patients (63.8%) received all three stress support sessions, 33 (16.6%) received two, 21 (10.6%) received one and 18 (9.0%) received none (Figure 18). During interview, both POPPI and research nurses reported that the IPAT screening tool was simple and user friendly. As evidence of this, 100% of potential patients were screened with the IPAT to determine eligibility for the intervention. With regards to duration of the session, the median time of each session was as follows: session 1, 30 minutes (IQR 30–35 minutes); session 2, 30 minutes (IQR 30–40 minutes); and session 3, 30 minutes (IQR 25–32.5 minutes).
FIGURE 18.
Number of stress support sessions received by patients (n = 199). Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
During the interviews, the POPPI nurses were asked to reflect on what strategies they had used to ensure that patients received all three sessions. All POPPI nurses reported that it was easiest to deliver the stress support sessions if they were able to adopt a flexible approach to their work. This included an ability to work outside paid hours and swap shifts to accommodate session delivery, which most were willing to do:
. . . I came in on my own time just to do them, just so that I knew that they were done and that I could definitely come at the time I was saying, so there’s not been an issue really. And I get my time back; work has been good and said that if you come in on your own time you can have it back. So it’s not been an issue really.
R-39
The POPPI nurses reported that sometimes they had minimal notice of a patient being discharged from hospital, which led to patients missing some of the sessions. When possible, the POPPI nurses combined the content of multiple sessions in an attempt to deliver the intervention as fully as possible. Conflicting clinical workload was also reported as a barrier to the delivery of the stress support sessions. It was not always feasible to have a POPPI nurse available consistently, and when they were absent or on annual leave, the burden of stress support delivery fell to the remaining POPPI nurses who struggled to meet the demands of intervention delivery. This meant that they needed to work outside their normal work hours to fulfil intervention delivery with enrolled patients. Despite the demands on the POPPI nurses to accommodate session delivery, there was only one unit at which they discussed an unwillingness to do this (because of personal commitments and a perception that the trial procedures had placed unrealistic expectations on them).
At 11 units, participants reported minimal or no obstruction to the release of the POPPI nurses from their clinical duties to deliver the stress support sessions. They stated a belief that the intervention was viewed as worthwhile within the units, and staff assigned importance to intervention delivery being fulfilled. At the twelfth unit, nurse managers obstructed the release of the POPPI nurses from their clinical duties to deliver the stress support sessions. These POPPI nurses explained that their ability to deliver the sessions was partly dependent on the priorities of the senior nurses in charge on any given shift, and that they would deliver sessions outside clinical hours to remedy this. It was ventured that a combination of unit busyness and a lack of belief in the intervention as worthwhile were contributing factors to this obstruction.
Despite some challenges faced by the POPPI nurses, there were only four occasions when sessions were missed specifically owing to unavailability of a POPPI nurse (Table 18). This is evidence of the level of commitment and flexibility that the POPPI nurses applied to ensure that the intervention was delivered as fully as possible.
| Reasons for not receiving stress support sessions | n (%) |
|---|---|
| Patients who received no stress support sessions (N = 18) | |
| Patient declined sessions | 6 (33.3) |
| Patient was discharged | 7 (38.9) |
| POPPI nurse was unavailable | 2 (11.1) |
| Other reasons | 3 (16.7) |
| Patients who received one stress support session only (N = 21) | |
| Patient declined further sessions | 5 (23.8) |
| Patient was discharged | 13 (61.9) |
| Patient died | 1 (4.8) |
| POPPI nurse was unavailable | 1 (4.8) |
| Other reasons | 1 (4.8) |
| Patients who received two stress support sessions (N = 33) | |
| Patient declined further session | 3 (9.1) |
| Patient was discharged | 29 (87.9) |
| POPPI nurse was unavailable | 1 (3.0) |
Unavoidably, there were several patient-centred barriers to this component of the intervention, at which patients declined some or all of the sessions. Participants were asked in interview to elaborate on reasons why patients declined to partake. The POPPI nurses reported that patients were often tired or unfit to participate in the sessions, and they wondered whether or not it was too early in a patient’s recovery trajectory to expect them to engage in such an intense intervention. If sessions were being delivered outside the critical care unit, the POPPI nurses sometimes found that the patient was unavailable at the arranged time. This was out of their control because patients may have had visitors or been receiving clinical interventions (e.g. scans, x-rays). Nonetheless, it was problematic to the POPPI nurses who were often working outside their normal hours to accommodate session delivery.
The personal action plan was developed for 121 out of 127 (95.3%) patients who received all three stress support sessions. These patients were re-assessed with the STAI-6 measure after the third stress support session (note that the STAI-6 was assessed in all patients, at both intervention group and control group sites, at the point of consent).
A total of 115 out of 127 (90.6%) patients who received all three stress support sessions were re-assessed with the STAI-6 prior to hospital discharge. Table 19 shows the comparison of the baseline with the post-stress support session 3 STAI-6 score for these 115 patients.
| Summary | Baseline | Post-stress support session 3 |
|---|---|---|
| Patients (n) | 115 | 115 |
| Mean (SD) | 49.3 (16.9) | 40.3 (13.5) |
| Median (IQR) | 47 (37–60) | 40 (30–50) |
In session one, patients were given a tablet computer containing a relaxation and recovery app. Of the 181 patients who received stress support session 1, 171 (94.8%) were given the tablet computer containing the relaxation and recovery app. One additional patient declined to receive the stress support sessions, but did accept the tablet computer to use the relaxation and recovery app. All POPPI nurses reported at interview that the main reasons why patients did not receive the tablet computer were an imminent discharge from hospital or they declined it.
Of patients receiving both stress support session 1 and the tablet computer, 131 (76.6%) reported use of the tablet computer to their POPPI nurse during the course of the sessions. The cluster RCT case report form captured reported reasons for why patients did not use the tablet computer; these were that patients preferred not to use it, were too ill, had a hearing impairment, lacked technical capability or found it difficult to use, were too tired, preferred TV or were discharged from hospital. During the interviews, the POPPI nurses were asked to expand on the main factors that they felt acted as barriers to and facilitators of the relaxation and recovery programme. They reported that sight problems, muscle weakness, poor concentration and problems with dexterity were the main barriers. They also described what appeared to be a generational barrier, with elderly patients being less willing to use the tablet (compared with relatively younger patients).
Family involvement was reported to be the main facilitator of the use of the relaxation and recovery programme. Families were able to encourage use of the tablet computer and the programme. In addition, they could assist in overcoming barriers associated with weakness, sight and dexterity:
. . . her whole family has got involved in it. They’re all listening to the DVDs and relaxation, and you know, she’s part of that. I heard her talking to her family and she was repeating what I had said to her. So it was good for the fact that I felt that she’d obviously taken it in.
R-36
Fidelity
To determine whether or not the stress support sessions were delivered as intended, it was planned that sessions would be audio-recorded and analysed by a member of the research team. However, the POPPI nurses were met with refusal from patients from the outset of the intervention period; subsequently, it appeared that they stopped approaching patients for consent to audio-record sessions. This meant that no sessions had their content analysed, and the fidelity of the stress support sessions is essentially an ‘unknown quantity’. What follows is a narrative based solely on what was discussed during interviews with the POPPI nurses at all 12 sites. There appeared to be an inclination to discuss the challenges, despite the researcher encouraging discussion of positive aspects of this component. Findings, therefore, cannot be extrapolated across the unit for the entire intervention period but reflect specific incidents and challenges that occurred.
Units were responsible for appointing their own POPPI nurses, and participants reported that an appropriate choice of POPPI nurse was key to successful intervention delivery. Their clinical roles varied, but there was consensus that senior/more experienced staff nurses were most suited to this role. Half of all POPPI nurses had senior clinical roles comprising nurse consultant, outreach practitioner and sister/deputy sister, with only one unit having three staff nurses in the POPPI nurse role. Some units selected non-clinical nurses for this role, namely research nurses, because it was felt that they could be more flexible in the time required to deliver the intervention. Research and outreach nurses in the POPPI nurse role were less acquainted with patients, having not been involved in their clinical care. At two units, research and outreach nurses reported finding it difficult to build rapport and trust with patients and ventured whether or not this would have been easier had the patient been known to them in a clinical nurse–patient capacity prior to intervention delivery.
All POPPI nurses reported pre-session preparation time as an important factor to facilitate smooth session delivery, including familiarisation with patient medical notes and the content of previous sessions. Most POPPI nurses discussed feeling that unrealistically short hours had been allocated by the trial for the delivery of this element when preparation time and post-session duties were factored in:
I take 10 minutes, sit with my file, go back over what needs to be achieved in session 1, session 2 or session 3 . . . So I have exactly what I need in here to deliver a session, and I’ll sit with what I need in front of me. So I’ve got it there. I can glance down and make sure that I’ve picked up on every point that I need to and know it’s always there from my previous session so that I can write up my notes with my next session in mind. I haven’t come away from a session and thought ‘I didn’t say this’, or, ‘oh no, I did say that’.
R-33
Each stress support session lasted for a median of 30 minutes (Figure 19). Some POPPI nurses reported difficulty in adhering to the stress support session guide with some patients difficult to keep on track, as patients often wanted to talk about their problems that were not related to the critical care unit. Some found it challenging to turn the conversation back to issues related to critical care, particularly when they felt that patients were opening up to them:
. . . you’ve got your, I think, very well-presented checklist of what you need to do in each session, [X] is very passionate that we tick all those boxes, and you’ve got this and this, and it’s really hard to try and make it mix . . . I can only describe it as sometimes, whilst you’re chatting, you’re almost trying to . . . fight with is the wrong wording, but almost fight with that patient to get them on track to the checklist . . .
R-08
FIGURE 19.
Duration of each stress support session. (a) Session 1; (b) session 2; (c) session 3; and (d) total.
The POPPI nurses stated that the personal action plan component of stress support session 3 was difficult to co-design with a certain subgroup of patients, namely those individuals who had particularly complex or life-changing injuries. They perceived this patient subgroup as difficult to engage with making long-term plans outside their hospital stay, and that these patients could, understandably, set only short-term goals while in the very early stage of their recovery trajectory.
For the 181 patients receiving stress support session 1, 72 (39.8%) received this session in the critical care unit. This proportion decreased with delivery of sessions 2 and 3, with the majority delivered in hospital wards, as patients came closer to hospital discharge (Table 20). The POPPI nurses at one-third of units also reported a lack of privacy in the unit as a barrier to patients discussing sensitive or emotive topics during the sessions.
| Delivery location | n (%) |
|---|---|
| Stress support session 1 (N = 181) | |
| Critical care | 72 (39.8) |
| Outside critical care | 109 (60.2) |
| Stress support session 2 (N = 160) | |
| Critical care | 27 (16.9) |
| Outside critical care | 133 (83.1) |
| Stress support session 3 (N = 127) | |
| Critical care | 14 (11.0) |
| Outside critical care | 113 (89.0) |
Debriefing and support for the POPPI nurses
Following the 3-day training course on delivering stress support sessions, nurses were to receive debriefing and support, consisting of one site visit by their allocated POPPI trainer, and three individual telephone calls with the same trainer. The visit was timed to be held after all POPPI nurses at a site had carried out sessions with at least one patient each. The individual calls were due to start after each nurse’s first patient, and at 2-month intervals thereafter.
Table 21 shows the delivery of debriefing and support visit and calls coded as green, blue and light green. Three sites were coded with three ‘greens’, while one site had three ‘light greens’. The other eight had varying combinations. Overall, of 48 cells in the table, most were ‘green’ (23 cells), followed by blue (15 cells) then light green (10 cells). Looking at the columns, visits were mostly coded ‘green’. Receipt of calls declined across the period, from call 1 to call 3. See Appendix 11 for an overview of the trainer’s perspective of the sessions.
| Site number | Call 1 | Visit | Call 2 | Call 3 |
|---|---|---|---|---|
| 01 | ||||
| 02 | ||||
| 03 | ||||
| 04 | ||||
| 05 | ||||
| 06 | ||||
| 07 | ||||
| 08 | ||||
| 09 | ||||
| 10 | ||||
| 11 | ||||
| 12 |
During the interviews, all but one of the POPPI nurses unfavourably reported the debriefing and support. Keeping patients on track with the session guide during the sessions sometimes posed a challenge, and the POPPI nurses raised this concern during the debriefing and support visits/calls. Despite seeking advice on how to deal with this (and to deliver sessions with as much fidelity as possible), the POPPI nurses recurrently reported being rigidly told to steer patients back towards the guide. Many did not feel that they had the skills to do this capably and would have benefited from more detailed support in this area:
. . . she’ll say ‘so you didn’t write down the action plan then?’ And I’ll go ‘no, I didn’t’, and I . . . I can sort of see, from her eyes, she’s thinking ‘well no, you should have done that and next time you should do that.’
R-08
In addition, the POPPI nurses reported feeling an element of ‘judgement’ from the debriefing and support team, in that they were being checked up on and disapproved of when they reported an inability to deliver sessions per protocol:
I said to [X] ‘I’ve had this patient who I felt like I was just patronising them and they didn’t really take it in’, and she said ‘that’s your stressful thought’, and I was like ‘oh my God, [X] is analysing me; she’s delivering me a stress support session and I’m talking to her about one I’ve done for somebody else!’
R-38
Overall, the POPPI nurses described feeling that if they had been able to discuss stress support session delivery less apprehensively with the debriefing and support team, they would have been more willing to disclose problems that arose in sessions, potentially addressing the barriers to successful delivery that presented in subsequent patients:
There was never any guidance anywhere, any shape or form if things didn’t go as per normal session.
R-18
. . . there were never any guidance of what we do if we get the session interrupted or the patient stops the session. How do you then move on? Do you stick a bit on or do you just say ‘right, we’ve missed that bit’, because it has a massive impact upon the outcomes of what we’re doing.
R-20
Element 3: relaxation and recovery programme for patients identified as acutely stressed
Reach
During stress support session two, patients were given a self-help booklet and a DVD for use after the sessions and then at home. In session 3, nurses were to co-design a ‘personal action plan’ with the patient and record this in the booklet.
For the 160 patients who received stress support session 2, most [n = 153 (95.6%)] received both the DVD and the booklet to use after the sessions and to take home with them. A small number [n = 6 (3.8%)] of patients did not receive the relaxation and recovery DVD but took the self-help booklet, and one patient received neither the DVD nor the booklet. A personal action plan was co-designed between the POPPI nurse and the patient for 121 (95.3%) of the 127 patients who received all three sessions.
Staff attitudes and unit culture
Study champions
Active study champions were key for positively affecting multidisciplinary team engagement and enthusiasm. At units with study champions, no division in levels of interest among the team and an overall willingness of staff to engage with the study were reported. As evidence of this, the POPPI nurses reported little or no resistance from their colleagues to completing the online training or to be released from clinical duties to deliver stress support sessions. Participants at these units viewed the intervention as worthwhile and of benefit to the psychological well-being of patients. On the contrary, in one unit, the nurse PI/POPPI nurse had a very poor attitude towards the study and disagreed with some of the core principles of the intervention. There was very poor engagement at this unit from both consultants and senior nurses/sisters, and it is possible that this negativity was contributed to by the attitude of the nurse PI. It may also have directly affected the delivery of the intervention, because the nurse PI reported omitting what they deemed the ‘airy-fairy’ elements of the stress support sessions.
Nursing staff
During the interviews, participants commonly reported viewing the POPPI intervention as a nurse-led intervention, and something that nurses felt that they could contribute to in a meaningful way. Nurses were, therefore, predominantly enthused across all intervention sites.
Medical staff
In units where the medical PIs were engaged and took a pro-active role in recruitment and promoting study awareness, this was cascaded down to junior doctors, who the POPPI nurses reported to be enthusiastic and receptive to POPPI-related training and teaching.
In units where the medical PI was disengaged or had minimal involvement with the study, the nurse PI carried the workload. During the interviews, the POPPI nurses at these units reported negative attitudes from both senior and more junior doctors, who were resistant to undertaking the online training and hostile towards implementing the POPPI training recommendations in practice. It may be postulated that this attitude from medical PIs in the units was directly conveyed to their consultant colleagues as well as more junior doctors. The POPPI nurses attributed the medical disengagement to the fact that nurses took ownership of the trial, and that medics viewed it as a nurse-led intervention.
It was reported at most units that, in general, younger, less experienced staff from all professional groups are more receptive to any type of practice change in the units, whereas more senior staff appear to be more resistant to training and education.
Control sites
Participants at all units reported that the trial had been well received and that, prior to control allocation, staff were enthusiastic and keen to be involved in the study with high interest in the role of the POPPI nurse. Across all four units, participants described a culture of heightened awareness of psychological support for patients, and of clinicians who had specific interests in psychological care and delirium. It was also unanimously reported that on being allocated to the control arm, there was widespread disappointment in units. During the interviews, the effect of this sense of disappointment was explored with participants. They all reported that, despite initial disappointment, there was no effect on enthusiasm in the units and the study ran smoothly, conducted by research staff ‘in the background’. Interestingly, during the interviews many participants discussed a sense of relief at being allocated to the control arm, as they had anticipated a high workload associated with delivering the intervention:
Disappointed and relieved at the same time . . . So I was relieved that we didn’t need to do that, but equally, at the same time, disappointed, because obviously you want to give a treatment or an intervention to patients too, because I can only imagine – I would hope that it would make peoples’ experience and lives better. So yeah, a bit of both really, but it’s what it is, so that’s fine. Somebody’s got to be the control group.
R-47
Research nurses reported the use of the STAI-6 (a brief questionnaire which assesses anxiety) with patients who were not receiving any subsequent psychological support as difficult at times:
. . . you bring up these feelings for them, which can be quite upsetting for them, because it’s probably the first thing they’ve been asked about, that, really. I mean, on the unit. It’s just hard to sort of walk away after you’ve brought out all these feelings for them. You give a bit of reassurance in terms of contacting their GP if, when they’re discharged, they still have a lot of these feelings, and just give them just general advice and reassurance. Tell them it’s a normal thing.
R-46
There did not appear to be anything that happened at the sample of four control sites that suggested any change in psychological support from baseline.
Recruitment factors
Intervention sites
Research nurses were primarily responsible for screening patients for recruitment into the cluster RCT, but most worked weekdays only and there was poor or no evening or weekend cover for screening. During the interviews, research nurses reported swapping shifts when possible to ensure that someone was on duty to avoid eligible patients being missed. Two units used WhatsApp Messenger (WhatsApp Inc., Mountain View CA, USA) groups to communicate among the team regarding potential eligible patients in an attempt to maximise recruitment. All research nurses reported that, by involving family members right from the beginning of the recruitment process, patients seemed more willing to be involved. Families acted as an additional channel of communication with the patient, and research nurses believed that families encouraged participation because they could see the effect that the critical care stay was having on the patient’s psychological well-being.
All research nurses were in agreement that giving patients sufficient time to read and digest the study information, and discuss it with their families, improved the recruitment process. They allowed patients to ask questions before enrolment and were willing to visit the patient multiple times until they were fully informed prior to consent. Research nurses discussed recognising potential patients early, who may become eligible in the near future, and making themselves familiar to the patient. Some POPPI nurses were directly involved in the patient’s clinical care, whereas others who were not would visit the patient for an informal chat to become acquainted with the patient. PIs would sometimes visit the patient to make themselves known to them, and informally discuss the study. If such approaches had been made, patients appeared to be more receptive when the time came to approach them for consent:
I’m this woman who has come to say ‘come tell me all your secrets, those dark nightmares that you’re having.’ . . . what I’ve found is that by recognising the patient early, before the 48 hours, and bringing yourself into the picture, makes you more acceptable . . . so by the time I actually go in and wanting to tell them more about POPPI, they already know me, and that seems to work well for me, very receptive after that.
R-27
At two units, participants reported recognising that some patients associated participation in the study with a stigma of being mentally ill, and this perception appeared to dissuade their participation:
That seemed to be the main thing, the fear of the stigma of being crazy in their records and their GP knowing about it.
R-25
One POPPI nurse described how this happened to her with a patient, but that recognising this meant that she adapted the way in which she approached subsequent patients, ensuring that this did not happen again:
I used the word counselling, which I shouldn’t have, because as soon as I said that she was like . . . and then I’ve never done that since . . . And she was like ‘no, I don’t want counselling. I don’t need it.’ I couldn’t backtrack then because I had said that word.
R-12
The use of lay language was also important in helping patients and families understand the study design. The POPPI nurses discussed avoiding the use of words such as ‘counselling’ or ‘mental health’ as they felt these carried negative connotations for patients and were more likely to increase refusal rates.
Despite measures put in place to promote recruitment, identifying patients in the critical care unit who had capacity to discuss the study before discharge was a significant challenge of the study. In the critical care units, patients were often quickly discharged to a ward as soon as they became medically fit. This short time frame to approach eligible patients frequently presented as a challenge to research nurses, as many reported during interview that patients were not capable of understanding and retaining sufficient information to give informed consent at this stage:
. . . they’re still quite muddled, and you can talk to them one day and say, ‘oh I’ll come back and speak to you later about it.’ Introduce the topic, come back later, ‘hello, do you remember we had a chat?’ And they go ‘no!’. . . . I think sometimes when they first come out of sedation or whatever, that there’s so much that they have to take on board and I’m not entirely sure that they retain the information we give them.
R-05
Control sites
Participants reported using the same strategies as intervention sites to promote recruitment. These consisted of ensuring good coverage of staff able to consent patients, promoting communication between the research team and involving families in the process. One unit took a further step in attempting to minimise missing eligible patients by training two staff nurses in Good Clinical Practice in addition to the research team, which ensured 24/7 coverage for recruitment. In keeping with the findings at intervention sites, research nurses also reported some difficulty in finding patients who had capacity to consent or digest study information while in the critical care unit:
. . . they’re still a bit too muddled to formally consent, and actually the window of opportunity to get these people, we’ve found a bit hard.
R-45
Aim three: relation between intervention delivery and primary outcome
Implementation grading for the subgroup analysis
Based on the data in the unit-level summaries, a detailed set of criteria was developed and applied to the interview data to allow the robust scoring of elements 1 (creating a therapeutic environment in critical care) and 2 (IPAT assessments and stress support sessions) from qualitative data to a quantitative score. Tables 22 and 23 show the criteria for scoring fidelity to elements 1 and 2.
| Element 1: creating a therapeutic environment in critical care | |
|---|---|
| Component scores | Scoring criteria |
| 3: Full adherence | Full effort made to implement training recommendations into practice; unit did all of the following:
|
| 2: Mostly adhering |
|
| 1: Some adherence | Mediocre effort made to implement training recommendations; unit did one or two of the above criteria |
| 0: Low adherence | Poor/no effort made to implement training recommendations; unit implemented only part of one of the above criteria |
| Element 2: IPAT assessments and stress support sessions | |
|---|---|
| Component scores | Scoring criteria |
| 3: Full adherence | Full effort made to deliver stress support sessions as intended. The POPPI nurses:
|
| 2: Mostly adhering | Decent effort made to deliver stress support sessions as intended. The POPPI nurses:
|
| 1: Some adherence | Mediocre effort made to deliver stress support sessions as intended. The POPPI nurses:
|
| 0: Low adherence | Poor/no effort made to deliver stress support sessions as intended. Unit could only score in this bracket if no effort was made to deliver any element of the stress support sessions |
Table 24 shows the POPPI intervention component scores for each of the intervention group sites (anonymised) along with the weighted composite scores. A detailed breakdown of these scores per intervention group site can be found in Report Supplementary Material 7. Reweighting did not change the ranking of intervention group sites but it did split those that had tied. At this stage, sites were split into tertiles, as specified in the a priori statistical analysis plan, according to overall intervention adherence:
-
3 = highest adherence
-
2 = moderate adherence
-
1 = lowest adherence.
| Unit ID | Component | Weighted composite score | Ranking | Category | |||
|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | ||||
| Maximum | 6 | 3 | 12 | 3 | 135 | 1–12 | 1–3 |
| 10 | 3 | 2 | 12 | 3 | 115.5 | 1 | 3 |
| 11 | 6 | 3 | 8 | 2 | 105.3 | 2 | 3 |
| 01 | 6 | 3 | 8 | 1 | 95.3 | 3 | 3 |
| 08 | 3 | 0 | 11 | 2 | 86.6 | 4 | 3 |
| 03 | 4 | 1 | 9 | 1 | 77.9 | 5 | 2 |
| 04 | 6 | 2 | 5 | 1 | 73.6 | 6 | 2 |
| 02 | 2 | 2 | 8 | 1 | 72.6 | 7 | 2 |
| 06 | 4 | 0 | 8 | 1 | 71.7 | 8 | 2 |
| 07 | 2 | 0 | 9 | 2 | 70.9 | 9 | 1 |
| 12 | 3 | 2 | 7 | 1 | 66.0 | 10 | 1 |
| 05 | 5 | 1 | 2 | 0 | 37.7 | 11 | 1 |
| 09 | 3 | 0 | 3 | 0 | 27.3 | 12 | 1 |
The results of the process evaluation subgroup analysis of implementation grade and cluster RCT primary outcome are reported in Chapter 7.
Summary of findings
High levels of uptake and adherence to each of the three elements of the POPPI intervention were seen during the intervention period at intervention sites. The units’ staff and cultural factors influenced their ability to fully implement the intervention, with variation between sites observed across each element.
By the end of the transition period, 971 out of 1669 critical care staff had completed the online training, equating to a median percentage of staff completing the POPPI online training of 58% (IQR 49–69%), with all ICUs achieving 80% (i.e. the prespecified minimum target) by intervention period month 3. Local initiatives to translate the online training into practice included optimisation of sleep (e.g. through sleep packs, night-time lighting and clustering of care), reduction of noise (e.g. through soft-close bins and minimisation of alarm and telephone noise), improved patient orientation (e.g. through clocks, staff–patient interaction and whiteboards) and increased family involvement. Some intervention sites found it challenging to change long-standing practices and/or were restricted by the physical environmental limitations of the unit.
During the intervention period at intervention group sites, 340 patients were recruited, of whom 313 consented to receive an IPAT assessment. All 313 were assessed; 199 of these patients were identified as acutely stressed and were eligible to receive stress support sessions. A total of 127 patients (63.8%) received all three, 33 (16.6%) received two, 21 (10.6%) received one and 18 (9.0%) received none. Of the patients who received session 1, 171 were given a tablet computer containing the relax and recover app to use between sessions and 131 (76.6%) reported use of it to their POPPI nurse. Most patients who received at least two sessions were given both the DVD and the booklet to take home. There was variation in delivery of stress support sessions across intervention sites: facilitators included the ability to work flexibly and pre-session preparation by the POPPI nurses, whereas barriers included unanticipated discharge of patients and conflicting clinical workload of the POPPI nurses.
Engagement of unit staff and a positive unit culture were key to ensuring effective implementation of both the online training and the stress support sessions. Positive study champions were key to this engagement of both nursing and medical staff. As a nurse-led intervention, nurses felt that they could contribute in a meaningful way and were predominantly enthused across all intervention sites. Engagement of medical staff was very dependent on the engagement of the medical PI. Younger, less experienced staff from all professional groups were more receptive to any type of practice change in the units, whereas more senior staff appeared to be more resistant to training and education.
With regards to control sites, there was initial disappointment at not being randomised to be intervention sites, but there did not appear to be any suggestion of any change in psychological support from baseline.
Recruitment processes were similar for intervention and control sites. Research nurses were primarily responsible for screening patients for recruitment into the cluster RCT, but most worked weekdays only and there was only some evening or weekend cover for screening. All research nurses reported that by involving family members right from the beginning of the recruitment process, patients seemed more willing to be involved. It was reported that, occasionally, participants recognised that some patients associated participation in the study with a stigma of being mentally ill, and this perception appeared to dissuade their participation (in the intervention sites during intervention period only). The use of lay language and avoidance of words such as ‘counselling’ or ‘mental health’ was also important. As patients were often quickly discharged to a ward as soon as they became medically fit, nurses reported that this short time frame to approach eligible patients presented a challenge to research nurses, as patients were often not capable of understanding and retaining sufficient information.
Considerations to maximise intervention implementation
As variation in implementation of the complex intervention was clearly observed, five factors that need to be considered when implementing this intervention in a unit have been identified. These are as follows.
Resources
To maximise the quality and fully deliver this intervention in an NHS setting, adequate and dedicated resources are required. With regards to the online training, to train all staff would require computer access in the unit, and the allocation of time in paid work hours for undertaking the training. This is likely to increase staff willingness to participate in such an initiative. In turn, mandating the online training for all staff would help to ensure compliance. For delivery of the stress support sessions, the POPPI nurses would also require protected time. Ideally, this would be outlined formally, as part of their clinical role, with dedicated time allocated. Owing to the flow of patients, this would need to be on a flexible basis to accommodate the patient-centred barriers that may be present and should not be subject to over-rule by nurse managers who may not prioritise psychological care in the unit. Although the POPPI nurses demonstrated a willingness to work outside their clinical hours and on days off to deliver the sessions for the research study, this is not sustainable in routine clinical practice and they need to be given protected time to fulfil the role.
Unit culture
It is a challenge to change long-standing practices in units. The POPPI nurses and clinicians discussed the resistance to change that is encountered from staff, and how all units have at least a small number of staff who are difficult to engage in any new practice. Measurable changes, such as noise levels and sleep optimisation, were more readily adopted in units, whereas other cultural practices (such as large ward rounds conducted at the end of patients’ beds) were viewed as the way things had always been done and were more difficult to change.
Environmental limitations of the unit may be very important in implementation. In very open-plan small units, staff found it difficult to even address the more simple elements such as reducing noise levels and orientating patients with whiteboards or clocks. Moving forward to future implementation in the NHS setting, it is clear that some units would be able to adapt to the intervention more fully than others, but this should not be equated with staff resistance to compliance with recommendations.
The selection of the POPPI nurses
The POPPI nurses who worked clinically in the units were in the unique position of being able to free ward staff from the unit who needed to undertake their training. The POPPI nurses were also often more familiar with recruited patients, so found it easier to build rapport with both patients and their families. However, these nurses struggled to balance their clinical duties with the demands of intervention delivery and frequently found themselves delivering stress support sessions in their own time. The POPPI nurses who worked in non-clinical roles (i.e. research nurses) were more flexible in their work commitments, and so were able to prioritise tasks to ensure that they could deliver stress support sessions at suitable times. However, they were not able to free staff from the bedside to undertake training because they did not work clinically in the unit and were not previously familiar with the patients with whom they delivered sessions to. The POPPI nurses who were research nurses also found it more difficult to connect with patients and families. Ideally, this role would be appointed to a dedicated core team of nurses in the unit who took sole responsibility for it, to overcome conflicting clinical workload barriers. Discussions with the POPPI nurses revealed that many agreed that this role would be best suited to nurses who had an existing interest in psychological care and were involved in the unit’s follow-up clinics.
Active education
A dedicated practice educator or a POPPI nurse should take responsibility for ongoing education in units, acting as intervention champions, which would help engage staff and maintain enthusiasm. This is particularly important when introducing a new intervention into routine clinical practice, and education could form part of induction programmes for new staff. As was found in units that integrated the online training into inductions, this is an effective way to ensure coverage of training programmes for units with high staff turnover. Champions are also key for engaging the multidisciplinary team.
POPPI nurse training and support
The POPPI nurses would benefit from further, more in-depth training to equip them to deal with the wide range of patient experiences that they are likely to face. They commonly discussed feeling out of their depth when patients were difficult to keep on track with the session guide and reported that there was a lack of guidance on dealing with these circumstances. More comprehensive training would equip them with the skill set to deal with patients with complex needs, and the confidence to deliver stress support sessions adapted to each individual patient’s needs. More comprehensive debriefing and support should be offered to the POPPI nurses to allow them to discuss their experiences without judgement, and to receive guidance on overcoming barriers with future patients. This may be better provided locally to allow for a more face-to-face approach.
Chapter 7 Cluster randomised clinical trial results: clinical effectiveness
Primary outcome: clinical effectiveness
At 6 months, the mean PSS-SR score for surviving patients in the intervention group had decreased from 11.8 (SD 11.2) points in the baseline period to 11.5 (SD 11.5) points in the intervention period. In the usual care group, the mean PSS-SR score had increased slightly from 10.1 (SD 10.6) points in the baseline period to 10.2 (SD 10.0) points in the intervention period. After adjustment, this corresponded to a primary treatment effect estimate (i.e. the interaction between treatment group and time period) of –0.03 (95% CI –2.58 to 2.52; p = 0.98) (Table 25). For full results of the primary outcome model, see Appendix 12.
| Baseline period | Intervention period | Treatment effect (95% CI) | p-value | ICC (95% CI) | |||
|---|---|---|---|---|---|---|---|
| Usual care (N = 284) | Intervention (N = 283) | Usual care (N = 446) | Intervention (N = 340) | ||||
| n | 259 | 245 | 415 | 314 | |||
| Mean (SD) | 10.1 (10.6) | 11.8 (11.2) | 10.2 (10.0) | 11.5 (11.5) | –0.03 (–2.58 to 2.52) | 0.98 | 0.007 (0.000 to 0.401) |
Secondary outcomes: clinical effectiveness
There were no significant differences between the groups in any of the secondary outcomes, including days alive and free from sedation (mean difference 0.47, 95% CI –1.03 to 1.96; p = 0.54), duration of critical care unit stay (mean difference –0.28, 95% CI –3.45 to 2.88; p = 0.86), PSS-SR score of > 18 points at 6 months (odds ratio 1.32, 95% CI 0.66 to 2.67; p = 0.43), HADS anxiety score at 6 months (mean difference –0.24, 95% CI –1.50 to 1.01; p = 0.70), HADS depression score at 6 months (mean difference –0.22, 95% CI –1.40 to 0.95; p = 0.71) and HRQoL at 6 months (mean difference 0.007, 95% CI –0.063 to 0.076; p = 0.85) (Table 26).
| Secondary outcome | Baseline period | Intervention period | Treatment effect (95% CI) | p-value | ICC (95% CI) | ||
|---|---|---|---|---|---|---|---|
| Usual care (N = 284) | Intervention (N = 283) | Usual care (N = 446) | Intervention (N = 340) | ||||
| Days alive and free from sedation to day 30 (n) | 284 | 283 | 446 | 340 | |||
| Mean (SD) | 24.3 (6.2) | 23.0 (7.8) | 24.0 (6.9) | 23.3 (7.7) | 0.47 (–1.03 to 1.96) | 0.54 | 0.001 (0.000 to 0.940) |
| Duration of critical care unit stay (days) (n) | 284 | 283 | 446 | 340 | |||
| Mean (SD) | 12.2 (13.6) | 14.0 (15.6) | 13.5 (14.4) | 14.6 (17.4) | –0.28 (–3.45 to 2.88) | 0.86 | 0.000 (0.000 to 0.000) |
| PSS-SR score of > 18 points at 6 monthsa | 259 | 245 | 415 | 314 | |||
| Percentage | 19.8 | 23.9 | 17.6 | 24.1 | OR 1.32 (0.66 to 2.67) | 0.43 | 0.001 (0.000 to 1.000) |
| HADS anxiety score at 6 monthsa | 259 | 245 | 415 | 314 | |||
| Mean (SD) | 5.9 (5.2) | 6.9 (5.3) | 5.7 (4.8) | 6.3 (5.3) | –0.24 (–1.50 to 1.01) | 0.70 | 0.007 (0.000 to 0.502) |
| HADS depression score at 6 monthsa | 259 | 245 | 415 | 314 | |||
| Mean (SD) | 5.3 (4.9) | 6.0 (5.1) | 5.3 (4.5) | 5.8 (5.0) | –0.22 (–1.40 to 0.95) | 0.71 | 0.000 (0.000 to 1.000) |
| HRQoL, as measured by the EQ-5D-5L, at 6 monthsa | 259 | 245 | 415 | 314 | |||
| Mean (SD) | 0.70 (0.27) | 0.66 (0.30) | 0.69 (0.28) | 0.67 (0.30) | 0.007 (–0.063 to 0.076) | 0.85 | 0.021 (0.007 to 0.066) |
Secondary analyses of the primary outcome
Of all patients surviving to 6 months, 21.2% were missing the primary outcome of PSS-SR score (21.2% in the usual care group and 21.1% in the intervention group). Sensitivity analyses, using complete-case analysis, which assumed that missing outcome data were MCAR had minimal effect on the primary outcome, reporting a treatment effect estimate of –0.02 (95% CI –2.52 to 2.47; p = 0.99).
Among patients with an IPAT score of ≥ 7 points who received at least two stress support sessions, the adherence-adjusted causal effect of the intervention on the mean PSS-SR score was –0.18 (95% CI –5.50 to 5.14; p = 0.95).
In addition, adjusting for site-level standardised mortality ratio in a post hoc analysis also had minimal effect on the primary outcome, reporting a treatment effect estimate of –0.14 (95% CI –2.66 to 2.38; p = 0.91).
Subgroup analyses of the primary outcome
There was no statistically significant interaction between the effect of treatment allocation and time period on PSS-SR scores at 6 months in any of the prespecified subgroups: age, sex, socioeconomic status – quintile of IMD (IMD 2015), duration of delirium, STAI-6 score, surgical status, overall intervention implementation score (for intervention group sites based on data from the process evaluation) or predicted PSS-SR score (heterogeneity of treatment effect). The p-values ranged from 0.08 to 0.93 (Figure 20).
FIGURE 20.
Subgroup analyses of the primary outcome. p-values are for tests of interaction. The x-axis is presented on a log scale. The solid line represents no difference between the groups. a, The IMD (2015) is reported by quintiles, with higher values indicating greater deprivation; b, duration of delirium is reported as the number of days on which patients were assessed as positive on the CAM-ICU for delirium; c, scores on the STAI-6 range from 20 to 80, with higher scores indicating greater anxiety; d, predicted PSS-SR score (heterogeneity of treatment effect) is reported by quintiles, from a prediction model for the primary outcome derived using data from patients receiving usual care and adjusted for a priori important covariates (i.e. age, sex, socioeconomic status, duration of delirium, STAI-6 score, surgical status);85 e, site implementation grade encompasses dose, fidelity and reach and is derived from data collected as part of the process evaluation (see Chapter 6 for more information). Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
Co-interventions received in the critical care unit
As the POPPI online training was delivered at the site level, it was possible that co-medical interventions received in the critical care unit by patients recruited at the intervention group sites during the intervention period could have been influenced by the online training. For all patients, receipt and duration of a number of critical care medical interventions (i.e. sedatives/anxiolytics/anaesthetics, sleep medications, benzodiazepines, antipsychotics, analgesics, antidepressants, vasoactive agents and mechanical ventilation) are summarised by treatment group and time period (Table 27).
| Co-intervention | Baseline period | Intervention period | ||
|---|---|---|---|---|
| Usual care (N = 284) | Intervention (N = 283) | Usual care (N = 446) | Intervention (N = 340) | |
| Sedatives/anxiolytics/anaestheticsa | ||||
| n (%) receiving intervention | 263 (92.6) | 264 (93.3) | 413 (92.6) | 311 (91.5) |
| Median (IQR) days among those receiving intervention | 3 (2–7) | 4 (2–8) | 4 (2–7) | 4 (2–9) |
| Mean (SD) days among all patients | 5.2 (5.2) | 5.5 (6.4) | 5.6 (7.0) | 6.2 (7.9) |
| Sleep medication | ||||
| n (%) receiving intervention | 78 (27.5) | 73 (25.8) | 143 (32.1) | 99 (29.1) |
| Median (IQR) days among those receiving intervention | 3 (2–6) | 3 (1– 7) | 3 (1–9) | 5 (2–9) |
| Mean (SD) days among all patients | 1.2 (3.1) | 1.6 (4.9) | 2.1 (5.4) | 2.7 (10.0) |
| Benzodiazepinesb | ||||
| n (%) receiving intervention | 75 (26.4) | 64 (22.6) | 129 (28.9) | 92 (27.1) |
| Median (IQR) days among those receiving intervention | 2 (1–5) | 2 (1–5) | 2 (1–5) | 2 (1–4) |
| Mean (SD) days among all patients | 1.0 (2.4) | 1.0 (4.1) | 1.3 (4.0) | 1.2 (4.5) |
| Antipsychotics | ||||
| n (%) receiving intervention | 68 (23.9) | 77 (27.2) | 123 (27.6) | 89 (26.2) |
| Median (IQR) days among those receiving intervention | 3 (1–7) | 3 (1–7) | 4 (2–11) | 2 (1–5) |
| Mean (SD) days among all patients | 1.2 (3.2) | 1.3 (3.5) | 2.0 (5.4) | 1.4 (4.8) |
| Analgesics | ||||
| n (%) receiving intervention | 280 (98.6) | 277 (97.9) | 443 (99.3) | 335 (98.5) |
| Median (IQR) days among those receiving intervention | 7 (4–11) | 7 (4–12) | 7 (4–13) | 7 (4–13) |
| Mean (SD) days among all patients | 8.7 (7.1) | 9.8 (9.9) | 10.7 (10.6) | 10.9 (13.3) |
| Antidepressants | ||||
| n (%) receiving intervention | 57 (20.1) | 63 (22.3) | 101 (22.6) | 66 (19.4) |
| Median (IQR) days among those receiving intervention | 5 (3–11) | 7 (3–11) | 7 (4–14) | 7 (3–17) |
| Mean (SD) days among all patients | 1.7 (5.3) | 2.3 (7.1) | 2.6 (8.1) | 2.6 (9.0) |
| Vasoactive agents | ||||
| n (%) receiving intervention | 237 (83.5) | 227 (80.2) | 383 (85.9) | 287 (84.4) |
| Median (IQR) days among those receiving intervention | 3 (2–5) | 3 (2–5) | 3 (2–5) | 3 (2–6) |
| Mean (SD) days among all patients | 3.3 (3.7) | 3.7 (4.2) | 4.1 (6.4) | 4.2 (6.3) |
| Mechanical ventilation | ||||
| n (%) receiving intervention | 269 (94.7) | 267 (94.3) | 415 (93.0) | 306 (90.0) |
| Median (IQR) days among those receiving intervention | 3 (2–7) | 4 (2–8) | 3 (2–8) | 3 (2–9) |
| Mean (SD) days among all patients | 5.9 (6.6) | 6.8 (9.5) | 6.8 (10.2) | 7.6 (11.8) |
Chapter 8 Economic evaluation: methods and results
Introduction
A CEA was embedded in the POPPI cluster RCT, aiming to evaluate the relative cost-effectiveness of the POPPI intervention versus usual care. The specific objectives were to:
-
compare incremental cost-effectiveness at 6 months of the POPPI intervention versus usual care (primary objective)
-
estimate the lifetime incremental cost-effectiveness between the treatment groups (secondary objective).
Methods
Overview
A full CEA was undertaken to assess the relative cost-effectiveness of the POPPI intervention versus usual care. The CEA was undertaken in two phases; first, resource use and outcome data that were collected as part of the cluster RCT database linked to the CMP and NHS Digital databases were used to report cost-effectiveness at 6 months. Second, the CEA used the 6-month data to project the lifetime cost-effectiveness of the POPPI intervention versus usual care. The cost analysis takes an NHS and Personal Social Services perspective. 93 The economic evaluation is reported in line with the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. 94
The CEA used information on HRQoL at 6 months, combined with information on survival status, to report QALYs. Each QALY was valued using the NICE-recommended threshold of willingness to pay for a QALY gain (£20,000), in conjunction with the costs of each strategy, to report the incremental net monetary benefits (INMBs) of the POPPI intervention versus usual care, overall and for the same prespecified subgroups, as for the evaluation of clinical effectiveness. The main assumptions of the CEA were subjected to extensive sensitivity analyses.
Resource use, unit costs, outcomes and cost-effectiveness up to 6 months
Resource use measurement
The resource use categories considered were chosen a priori; according to those for which the differences between the treatment groups were judged as being possible and likely to drive incremental costs, these were resource use associated with the POPPI intervention, hospital admissions (index admission and re-admissions), and visits to outpatients and community health-care services.
Resource use associated with the POPPI intervention
The costs of the POPPI intervention were estimated based on experience of delivering the intervention in a typical critical care unit and considering the rolling out of the POPPI intervention into routine NHS practice. Costing was based on the following guiding principle: key elements of the intervention that were deemed important to outcomes and would be provided in routine practice were considered, and elements that would incur costs in routine practice but were provided free of charge in the cluster RCT were costed and included. The cost items associated with the POPPI intervention were grouped into three elements (Table 28). The base-case intervention costs were estimated using resource use data recorded on the cluster RCT electronic case report form, and informed by expert clinical opinion and the process evaluation, reflecting the most plausible assumption for routine practice in the majority of critical care units (see Table 28), with alternative levels considered in the sensitivity analyses (see Table 30).
| Elements of the intervention | Content/delivery of element | Action | Level of resource use |
|---|---|---|---|
| Element one: creating a therapeutic environment in critical care | Delivery of online training | Take course (including test), co-ordinate delivery in unit | Absorbed in the NHS mandatory training cost |
| Creating a therapeutic environment | Seminars/teaching, core groups meetings/activities, individual activities/actions | ||
| Element two: three stress support sessions for patients identified as acutely stressed | IPAT assessment | Screening patient with questionnaire | 10 minutes of bedside nurse’s (band 5) time |
| Delivery of stress support sessions |
|
1.5 hours of the POPPI nurse’s (band 7) time | |
| Three-day training course |
|
Per-patient costs calculated from the actual training costs incurred | |
| Debriefing and support | Trainee/clinical supervisor’s meeting time | 0.5 hours per month of three POPPI nurses’ and a trainer’s (band 8) time, per site | |
| Element three: relaxation and recovery programme for patients identified as acutely stressed | Relaxation and recovery programme | Delivering the programme via tablet computer app, patient booklet and DVD | Absorbed in the NHS routine cost |
Element one (creating a therapeutic environment in critical care) required all clinical critical care unit staff to complete an online training course (the POPPI online training). The online training would be expected to be included in the NHS mandatory training, and completed by staff members on an annual basis, if the POPPI intervention were to be rolled out into routine NHS practice. Creating a therapeutic environment in critical care also required staff to deliver and attend seminars and other educational activities, and to discuss creation and promotion of the therapeutic environment at team meetings. It was considered that this element of the intervention would be built in to the regular training and educational activities of critical care units if the POPPI intervention were to be rolled out into routine NHS practice. Therefore, no additional staff time and costs were considered for this element.
Element two (three stress support sessions for patients identified as acutely stressed) included a staff member assessing a patient for acute stress (using the IPAT), delivery of stress support sessions for patients identified as acutely stressed (i.e. those with an IPAT score of ≥ 7 points), costs of the 3-day training course for the POPPI nurses, and continued debriefing and support for the POPPI nurses following the 3-day course.
All patients recruited from intervention group sites during the intervention period of the POPPI cluster RCT were assessed for acute stress using the IPAT. Informed by expert opinion and data from the process evaluation, base-case costing for this activity considered that an IPAT assessment session required, on average, 10 minutes of a bedside nurse’s (band 5) time and was costed accordingly. Delivery of stress support sessions to acutely stressed patients involved staff time for preparation (reviewing patient records/medical history and IPAT responses), carrying out of the three stress support sessions and the writing up of key actions/outcomes of the session(s). Expert opinion and data from the process evaluation suggested that the delivery of one stress support session required 1.5 hours of a POPPI nurse’s (band 7) time. Per-patient costs were calculated as per the data recorded on the POPPI cluster RCT electronic case report form on the frequency of stress support sessions (0–3 sessions) received by acutely stressed patients (i.e. those with an IPAT score of ≥ 7 points) in the cluster RCT. At the beginning of the transition period, all POPPI nurses at the intervention group sites attended the 3-day training course, organised centrally. From the actual costs incurred in the cluster RCT, costs per site were calculated, from which per-patient costs were calculated considering the mean annual number of critical care admissions who would probably meet the POPPI cluster RCT eligibility criteria (based on data from the CMP and the cluster RCT screening and enrolment logs). All of the POPPI nurses were allocated a clinical trainer from the POPPI training team to provide ongoing debriefing and support following the 3-day training course. Debriefing and support were focused on enhancing nurses’ skills and discussing individual patient cases. The level of debriefing and support considered per site was that all POPPI nurses (band 7) meet a trainer (health psychologist band 8) for 0.5 hours per month. The opportunity costs of debriefing and support time were calculated. Per-patient costs were calculated considering the mean annual number of critical care admissions who would probably meet the POPPI cluster RCT eligibility criteria (as above).
Element three (relaxation and recovery programme for patients identified as acutely stressed) required delivery of the programme via already-developed materials: a tablet computer app, a patient booklet and a DVD. These costs are negligible and are considered to be absorbed in routine NHS costs.
Hospital stay: index admission
The duration and location of the index hospital admission, that is, the hospital stay following recruitment, was recorded for each patient for up to 6 months post recruitment on the electronic case report form. The total duration of the index admission included the time spent in critical care and on general medical wards. The length of stay in critical care was calculated as the total duration in days (including fractions of days), from the date and time of admission for the stay in critical care during which the patient was recruited until the time of discharge from critical care or death. Within the index admission, the total duration of critical care stay included all of the time spent in critical care between admission to the critical care unit and discharge from acute hospital and included any transfers to critical care units in other hospitals, as well as those in the hospital where the patient was recruited. For each day in critical care, data on the number of organs supported was recorded in the CMP database. Each critical care episode was then assigned a Healthcare Resource Group (HRG) code, applying standard HRG grouper algorithm. 95
For the index admission, the total length of stay was calculated as the total duration in days from the date of recruitment to the date of ultimate hospital discharge, or death.
Re-admissions
A hospital re-admission was defined as a further hospital admission following ultimate hospital discharge from the index admission. The information on re-admissions was collected from two sources. First, data on re-admissions to critical care were accessed from the CMP database. 63 From the CMP database, information was accessed on the duration of stay in critical care and the total hospital stay, including subsequent transfer to other care areas (e.g. general medical wards) in the same hospital and to other hospitals. Second, information on re-admissions that did not include a further stay in critical care was collated from responses to the health services questionnaire administered to patients surviving to 6 months.
Hospital outpatient visits and community service use
The resource use items considered included the total number of hospital outpatient visits and community service use following discharge from the index admission, but before 6 months. The resource use items considered resources used both for reasons related to the admission to the critical care unit in which the patient was recruited and reasons unrelated to the initial critical care admission (i.e. other health reasons). The items of community service use included visits to the GP, nurses (i.e. from the GP clinic, hospital or a psychiatric nurse), health visitors, occupational therapists, speech and language therapists, counsellors, physiotherapists, psychiatrists, psychologists and critical care follow-up clinics. The levels of resource use were taken from responses to the health services questionnaire administered to patients surviving to 6 months.
Unit costs
The unit costs required for valuing the resource use data were taken from national unit cost databases,96,97 and are listed in Table 29. The unit costs associated with the additional staff time required to deliver the POPPI intervention were taken from national sources. 96 The costs per critical care bed-day by HRG and general medical bed-day were taken from the ‘Payment by Results’ database. 97 Unit costs for hospital outpatient visits and community service use were obtained from a recommended published source for Health and Social Care costs. 96 All unit costs were reported in 2015–16 GBP prices.
| Item | Unit cost (£) | Source |
|---|---|---|
| Staff time for delivering the POPPI intervention | ||
| Hospital nurse, band 5 (per hour) | 35 | Unit Costs of Health and Social Care 2016 96 |
| Hospital nurse (POPPI nurse), band 7 (per hour) | 53 | Unit Costs of Health and Social Care 2016 96 |
| Health psychologist, band 8 (per hour) | 60 | Unit Costs of Health and Social Care 2016 96 |
| Hospital costs (bed-day) | ||
| Critical care bed-day (number of organs supported) | ||
| 0 | 759 | NHS Reference Costs 2015–2016 97 |
| 1 | 1031 | NHS Reference Costs 2015–2016 97 |
| 2 | 1399 | NHS Reference Costs 2015–2016 97 |
| 3 | 1619 | NHS Reference Costs 2015–2016 97 |
| 4 | 1794 | NHS Reference Costs 2015–2016 97 |
| 5 | 1977 | NHS Reference Costs 2015–2016 97 |
| ≥ 6 | 2274 | NHS Reference Costs 2015–2016 97 |
| General Medical bed-day | 298 | NHS Reference Costs 2015–2016 97 |
| Outpatient and community health services | ||
| Hospital outpatient | 135 | Unit Costs of Health and Social Care 2016 96 |
| GP practice visit (per visit) | 36 | Unit Costs of Health and Social Care 2016 96 |
| GP home visit (per visit) | 118 | Unit Costs of Health and Social Care 2016 96 |
| GP nurse visita | 11 | Unit Costs of Health and Social Care 2016 96 |
| GP nurse home visita | 18 | Unit Costs of Health and Social Care 2016 96 |
| Hospital nursea | 9 | Unit Costs of Health and Social Care 2016 96 |
| Health visitora | 8 | Unit Costs of Health and Social Care 2016 96 |
| Health visitor home visita | 18 | Unit Costs of Health and Social Care 2016 96 |
| Occupational therapista | 8 | Unit Costs of Health and Social Care 2016 96 |
| Physiotherapista | 8 | Unit Costs of Health and Social Care 2016 96 |
| Psychiatrista | 16 | Unit Costs of Health and Social Care 2016 96 |
| Psychologista | 13 | Unit Costs of Health and Social Care 2016 96 |
| Counsellora | 8 | Unit Costs of Health and Social Care 2016 96 |
| Speech and language therapista | 8 | Unit Costs of Health and Social Care 2016 96 |
Mortality and health-related quality of life
Information on the date and time of deaths was used to calculate the survival time up to 6 months for each recruited patient. HRQoL at baseline (i.e. at the time of consent) was self-completed by patients using a visual analogue scale (a health thermometer with scores ranging from 0 to 100). HRQoL at 6 months was measured using the EuroQol-5 Dimensions (EQ-5D) (a generic measure), which requires patients to describe their health on five dimensions: mobility, self-care, usual activities, pain/discomfort, and anxiety/depression. The EQ-5D with five levels (EQ-5D-5L)56 was chosen, in which patients are required to state whether they have ‘no problems’, ‘slight problems’, ‘moderate problems’, ‘severe problems’ or ‘extreme problems’ for each dimension. Permission for reuse of the questionnaire was obtained from EuroQol [https://euroqol.org (accessed 1 July 2019)] and the EQ-5D-5L questionnaire was administered to survivors as part of the 6-month follow-up questionnaire (see Chapter 5). The responses to the EQ-5D-5L questionnaire were used to report each patient’s described health, which was then valued according to health state preferences from the general English population to calculate EQ-5D utility scores,98 anchored on a scale from 0 (death) to 1 (perfect health). 99 QALYs at 6 months were calculated by valuing each patient’s survival time by their HRQoL at baseline and at 6 months according to the ‘area under the curve’ approach. 100 For 6-month survivors, QALYs were calculated using the HRQoL at baseline and at 6 months, applying linear interpolation. For decedents between recruitment and 6 months, a linear interpolation was applied between the baseline HRQoL and the date of death, when a zero HRQoL was applied.
Cost-effectiveness
The CEA followed the ITT principle, and reported incremental costs, QALYs and cost-effectiveness up to 6 months, according to treatment group. Missing data for resource use and HRQoL were handled with multivariate imputation by chained equation,101 assuming that the data were MAR, conditional on baseline covariates, resource use and observed end points. 70 Patients who did not return or fully complete the EQ-5D-5L questionnaire administered at 6 months had their EQ-5D-5L scores imputed from those survivors who did fully complete the questionnaire. Similarly, for those eligible patients who did not return the health services questionnaire, information on the use of outpatient services up to 6 months was imputed from those patients who completed the health services questionnaire. The resultant estimates were combined with Rubin’s rules, which recognise uncertainty both within and between imputations. 102
Total costs up to 6 months were calculated by combining the resource use with unit costs. The CEA used a GLMM at the individual patient level (patients nested within sites and within treatment group/time period), with random effects at site level. The analysis adjusted for prespecified baseline covariates at both patient and site level (see Chapter 4 for further details), with the primary effect estimate being the interaction (difference in difference) between treatment groups and time period. 70 We reported adjusted mean differences between the treatment groups in 6-month HRQoL, QALYs and costs, together with 95% CIs. The differences in average costs and QALYs between the treatment groups were used to calculate the INMB of the POPPI intervention versus usual care. We valued the incremental QALY according to the NICE-recommended willingness-to-pay threshold for a QALY gain (£20,000) and subtracted from this the incremental cost. INMBs were reported overall, and for the same prespecified subgroups as the clinical evaluation (see subgroup analyses of the primary outcome for the clinical evaluation in Chapter 7).
The uncertainty around the differences in average costs and QALYs between the treatment groups was illustrated on the cost-effectiveness plane. 103 We estimated the incremental costs and QALYs with a multilevel regression model (described above). To express the uncertainty in the estimation of the incremental costs and QALYs, we used the estimates of the means, and variances from the regression model, to generate 500 estimates of incremental costs and QALYs from the joint distribution of these end points, assuming asymptotic normality. We then plotted these incremental costs and QALYs on the cost-effectiveness plane. We also reported cost-effectiveness acceptability curves, by calculating the probability that, compared with usual care, the POPPI intervention is cost-effective given the data, at alternative levels of willingness to pay for a QALY gain.
Base-case assumptions and subsequent sensitivity analyses
The main assumptions made in the base-case scenario and how each was relaxed in sensitivity analyses are detailed below and summarised in Table 30.
| Assumption | Base case | Sensitivity analysis |
|---|---|---|
| Nurse’s time for IPAT assessmenta | 10 minutes per patient | 5 minutes per patient |
| Nurse’s time for IPAT assessment | 10 minutes per patient | 20 minutes per patient |
| Nurse’s time for delivering stress support sessionb | Each stress support session requires 1.5 hours | Each stress support session requires 1 hour |
| Nurse’s time for delivering stress support session | Each stress support session requires 1.5 hours | Each stress support session requires 2 hours |
| Re-admissions from health services questionnairesc | Included in the analysis | Excluded from the analysis |
| HRQoL at time of consentd | HRQoL measured at time of consent was applied | Zero HRQoL at time of consent was applied |
| Distributional assumptionse | Costs and QALYs normally distributed | Costs and QALYs gamma distributed |
| Unit-level standardised mortality ratiosf | Not included in the analysis | Adjusted for in the analysis |
The results of the sensitivity analysis are reported as mean INMBs with corresponding 95% CIs.
Subgroup analyses
We undertook prespecified subgroup analyses (as per analysis of clinical effectiveness) to report INMBs according to age (quartiles), sex (males vs. females), socioeconomic status (quintile of the IMD76), duration of delirium (patients with no CAM-ICU defined delirium versus patients with CAM-ICU defined delirium less than the overall median duration of delirium versus patients with CAM-ICU defined delirium greater than or equal to the overall median delirium duration), STAI-6 score (quartiles), surgical status (emergency/urgent surgery vs. elective/scheduled surgery vs. non-surgical), overall intervention implementation score for intervention group sites (tertiles) and predicted PSS-SR score (quintiles). The base-case analysis was repeated, adjusting for adherence using a structural mean model105 with an instrumental variable of randomised allocated treatment to estimate the efficacy (adherence-adjusted causal effect, as per the analysis of the primary clinical outcome, see Chapter 4) of the stress support sessions among those patients consenting to the IPAT assessment and stress support sessions, assessed as being at a high risk of psychological morbidity (i.e. an IPAT score of ≥ 7) and receiving at least two stress support sessions.
Lifetime cost-effectiveness
Lifetime cost-effectiveness was projected by summarising the relative effects of alternative strategies on long-term survival and HRQoL as compared with that of an age- and sex-matched general population. 106,107 The survival of the patients who survived up to 6 months was extrapolated over the lifetime. This extrapolation compared the survival and HRQoL of POPPI cluster RCT patients with those of the age- and sex-matched general population. There is evidence to suggest that critical care survivors face a higher probability of death108 and lower quality of life109,110 after the critical care episode than the (age- and sex-matched) general population. However, there is no clear evidence as to the magnitude and duration of excess mortality in general critical care patients. In the POPPI cluster RCT, the survival probability of patients at 1 year was similar to those of the age- and sex-matched general population. There is evidence of a decrement in quality of life for up to 5 years following discharge from critical care. 109 In the POPPI cluster RCT, the HRQoL of POPPI survivors at 6 months was ≈87% of that of the age- and sex-matched general population. 111 We therefore assumed a HRQoL decrement in the first year, but with improvement over 5 years to match the quality of life of the age- and sex-matched general population. After 5 years, we applied HRQoL values for the age- and sex-matched general population. Lifetime QALYs were reported by combining life-years and HRQoL.
To project lifetime costs attributable to the initial episode of critical illness, we considered the re-admission (to critical care and general wards) costs recorded up to 1 year post recruitment in the POPPI cluster RCT. Mean annual re-admission costs in critical care and general wards were calculated for patients who survived for at least 6 months and were not censored between 6 and 12 months. For the outpatient and community service costs, we applied the 6-month costs to the subsequent 6 months for patients who survived for at least 6 months and were not censored between 6 and 12 months, and calculated mean annual outpatient and community costs. These mean costs were applied annually for up to 5 years, after which time the HRQoL per patients in one POPPI trial was assumed to be at the levels of the age- and sex-matched population. All future costs and life-years were discounted at the recommended rate of 3.5%. 93
Results
Cost-effectiveness up to 6 months
Resource use
For the index hospital episode, the mean length of stay in critical care was longer for patients recruited at intervention group sites than for those recruited at control group sites in the baseline period. In the intervention period, the mean length of stay in critical care from the index admission was lower for patients recruited at the intervention group sites than for those in the usual care group (Table 31). Length of stay in general medical wards from the index admission was higher for patients recruited from intervention group sites than for patients in the usual care group, in both baseline and intervention periods. The proportions of patients re-admitted were similar between treatment groups. The average length of stay in critical care from re-admission was lower in the intervention group than in the usual care group in both baseline and intervention periods. The mean length of stay in general medical wards from re-admissions was higher in the intervention group than in the usual care group in both time periods. The average total length of stay prior to 6 months post recruitment was 31.27 days in the intervention group and 27.52 days in the usual care group in the baseline period. In the intervention period, the average total length of stay prior to 6 months was 33.95 days in the POPPI intervention group and 31.16 days in the usual care group. Overall, the total length of stay was higher in the POPPI intervention group sites than in the usual care group sites, but the intervention reduces the differences in length of stay by ≈1 day in the intervention period compared with the baseline period.
| Admission | Time period | |||
|---|---|---|---|---|
| Baseline | Intervention | |||
| Intervention (n = 283) | Usual care (n = 284) | Intervention (n = 340) | Usual care (n = 446) | |
| Index admission, mean (SD) | ||||
| Days in critical care | 12.55 (12.13) | 11.46 (9.63) | 12.74 (10.86) | 13.05 (12.98) |
| General medical bed-days | 17.10 (24.95) | 13.92 (18.82) | 19.51 (28.42) | 16.96 (25.56) |
| Re-admissions | ||||
| Re-admissions, n (%) | 32 (11.31) | 36 (12.68) | 26 (7.65) | 32 (7.17) |
| Days in critical care, mean (SD) | 0.67 (2.75) | 1.21 (5.28) | 0.58 (3.80) | 0.67 (2.75) |
| General medical bed-days, mean (SD) | 0.94 (6.70) | 0.93 (4.70) | 1.13 (6.64) | 0.94 (6.70) |
| Total length of stay up to 6 months (days), mean (SD) | 31.27 (31.04) | 27.52 (25.06) | 33.95 (33.66) | 31.16 (31.96) |
Table 32 summarises the resource use reported from responses to the health services questionnaire for all patients recruited during the baseline and intervention periods. The results are presented for the samples with complete information. The number of complete responses/eligible patients at 6 months are as follows:
-
baseline period –
-
intervention group, 160 out of 245 (65%)
-
usual care group, 160 out of 259 (62%)
-
-
intervention period
-
intervention group, 187 out of 314 (60%)
-
usual care group, 257 out of 415 (62%).
-
| Resource use | Time period, mean (SD) | |||
|---|---|---|---|---|
| Baseline | Intervention | |||
| Intervention (n = 160) | Usual care (n = 160) | Intervention (n = 187) | Usual care (n = 257) | |
| Inpatient days (general medical) | 5.38 (14.35) | 4.60 (11.83) | 9.53 (28.39) | 6.82 (19.39) |
| Outpatient visits | 4.33 (5.40) | 5.49 (6.70) | 4.69 (6.18) | 5.09 (6.13) |
| GP contacts | 4.69 (5.52) | 4.08 (5.36) | 4.05 (4.77) | 3.59 (5.06) |
| Nurse contacts | 3.97 (5.85) | 3.75 (6.51) | 4.39 (7.83) | 3.31 (5.36) |
| Health visitor contacts | 1.41 (4.76) | 0.59 (1.76) | 1.49 (6.57) | 2.5 (16.64) |
| Occupational therapist contacts | 0.70 (1.54) | 1.37 (6.28) | 1.35 (6.08) | 1.49 (7.71) |
| Speech therapist contacts | 0.27 (1.13) | 0.08 (0.48) | 0.06 (0.31) | 0.19 (0.92) |
| Physiotherapist contacts | 1.30 (2.25) | 0.75 (1.89) | 1.22 (2.49) | 1.87 (6.63) |
| Psychiatrist contacts | 0.11 (0.43) | 0.07 (0.36) | 0.15 (0.72) | 0.22 (1.04) |
| Psychiatric nurse contacts | 0.19 (1.45) | 0.05 (0.31) | 0.17 (1.00) | 0.02 (0.19) |
| Psychologist contacts | 0.06 (0.34) | 0.10 (0.75) | 0.04 (0.28) | 0.21 (1.04) |
| Counsellor contacts | 0.36 (1.59) | 0.05 (0.31) | 0.22 (0.92) | 0.41 (1.51) |
The average number of inpatient days reported from admissions other than those to critical care were higher in the intervention group than in the usual care group in both the baseline and the intervention periods. Patients recruited at intervention group sites had higher average numbers of contact with GPs and nurses and lower outpatient visits than those in the usual care group. All other community care contacts up to 6 months were higher or similar in the intervention group than in the usual care group in the baseline period. In the intervention period, all other community care contacts were lower in the intervention group than in the usual care group.
Total costs
The cost of the POPPI intervention was £140 per patient, which represents 0.5% of the total cost of treatment for the intervention group in the intervention period. The intervention cost included the cost of the IPAT assessment (£6), delivery of stress support sessions (£109), training (£4) and debriefing (£21). The net result of combining the intervention, critical care, general medical ward, and outpatient and community care costs is that the intervention group had higher mean total costs per patient than the usual care group, in both baseline and intervention periods (Table 33). The mean total cost per patient in the baseline period was higher in the intervention group (£27,824) than in the usual care group (£26,176). In the intervention period, the mean total cost per patient was similar between the treatment groups (£30,100 for the POPPI intervention group and £29,618 for the usual care group).
| Costs | Time period, mean (SD) | |||
|---|---|---|---|---|
| Baseline | Intervention | |||
| Intervention (n = 283) | Usual care (n = 284) | Intervention (n = 340) | Usual care (n = 446) | |
| Intervention costs | 140 (128) | |||
| Hospital costs | ||||
| Index admission | ||||
| Critical care | 19,221 (19,183) | 17,424 (15,195) | 19,573 (18,083) | 20,495 (21,627) |
| General medical ward | 5095 (7435) | 4149 (5608) | 5814 (8471) | 5055 (7618) |
| Re-admissiona | ||||
| Critical care | 910 (3897) | 1679 (7486) | 873 (6877) | 782 (4378) |
| General medical | 279 (1995) | 277 (1401) | 337 (1979) | 170 (1364) |
| Outpatient and community costsa,b | 2319 (4356) | 2646 (4551) | 3363 (7026) | 3118 (5609) |
| Total costs up to 6 monthsa,b,c | 27,824 (23,793) | 26,176 (20,957) | 30,100 (25,403) | 29,618 (27,287) |
Health-related quality of life
The health status profiles reported from responses to the EQ-5D-5L questionnaires administered at 6 months are summarised by treatment group and time period in Table 34. The results are presented for the samples with complete information; the number of complete responses/eligible patients at six months are as follows: baseline period: intervention group 188/245 (77%); usual care group 197/259 (76%); intervention period: intervention group: 248/314 (79%); usual care group: 320/415 (77%). At six months, the proportion of patients who reported ‘no problems’ for each dimension of the EQ-5D in the POPPI intervention group was lower than for the usual care group, in both baseline and intervention periods.
| EQ-5D-5L component | Time period, n (%) | |||
|---|---|---|---|---|
| Baseline | Intervention | |||
| Intervention (n = 188) | Usual care (n = 197) | Intervention (n = 248) | Usual care (n = 320) | |
| Mobility | ||||
| No problems | 55 (29.26) | 66 (33.5) | 90 (36.29) | 123 (38.44) |
| Slight problems | 51 (27.13) | 55 (27.92) | 44 (17.74) | 67 (20.94) |
| Moderate problems | 43 (22.87) | 37 (18.78) | 57 (22.98) | 60 (18.75) |
| Severe problems | 31 (16.49) | 31 (15.74) | 42 (16.94) | 52 (16.25) |
| Extreme problems | 8 (4.26) | 8 (4.06) | 15 (6.05) | 18 (5.63) |
| Self-care | ||||
| No problems | 109 (57.98) | 137 (69.54) | 161 (64.92) | 207 (64.69) |
| Slight problems | 41 (21.81) | 29 (14.72) | 37 (14.92) | 53 (16.56) |
| Moderate problems | 24 (12.77) | 22 (0) | 28 (11.29) | 41 (12.81) |
| Severe problems | 12 (6.38) | 5 (2.54) | 12 (4.84) | 14 (4.38) |
| Extreme problems | 2 (1.06) | 4 (2.03) | 10 (4.03) | 5 (1.56) |
| Usual activities | ||||
| No problems | 47 (25.00) | 68 (34.52) | 68 (27.42) | 104 (32.50) |
| Slight problems | 59 (31.38) | 60 (30.46) | 68 (27.42) | 84 (26.25) |
| Moderate problems | 45 (23.94) | 38 (19.29) | 59 (23.79) | 68 (21.25) |
| Severe problems | 20 (10.64) | 22 (11.17) | 28 (11.29) | 31 (9.69) |
| Extreme problems | 17 (9.04) | 9 (4.57) | 25 (10.08) | 33 (10.31) |
| Pain/discomfort | ||||
| No problems | 45 (23.94) | 50 (25.38) | 73 (29.44) | 86 (26.88) |
| Slight problems | 61 (32.45) | 68 (34.52) | 69 (27.82) | 117 (36.56) |
| Moderate problems | 52 (27.66) | 50 (25.38) | 65 (26.21) | 75 (23.44) |
| Severe problems | 20 (10.64) | 19 (9.64) | 30 (12.10) | 25 (7.81) |
| Extreme problems | 10 (5.32) | 10 (5.08) | 11 (4.44) | 17 (5.31) |
| Anxiety/depression | ||||
| No problems | 79 (42.02) | 89 (45.18) | 112 (45.16) | 144 (45.00) |
| Slight problems | 51 (27.13) | 57 (28.93) | 53 (21.37) | 102 (31.87) |
| Moderate problems | 32 (17.02) | 31 (15.74) | 56 (22.58) | 46 (14.37) |
| Severe problems | 10 (5.32) | 10 (5.08) | 18 (7.26) | 21 (6.56) |
| Extreme problems | 16 (8.51) | 10 (5.08) | 9 (3.63) | 7 (2.19) |
The mean EQ-5D-5L utility scores were similar between the treatment groups. Patients recruited at intervention group sites had higher all-cause mortality in the baseline period than the usual care group, but in the intervention period, the all-cause mortality was similar between both groups. The resultant mean QALYs were also similar across the treatment groups (Table 35). The net effect is that the intervention has a very small effect on QALY in favour of intervention in the intervention period.
| EQ-5D-5L/mortality/QALYs | Time period | |||
|---|---|---|---|---|
| Baseline | Intervention | |||
| Intervention (n = 283) | Usual care (n = 284) | Intervention (n = 340) | Usual care (n = 446) | |
| EQ-5D-5L (survivors),a mean (SD) | 0.661 (0.303) | 0.698 (0.268) | 0.668 (0.302) | 0.690 (0.279) |
| All-cause mortality, n (%) | 38 (13.43) | 25 (8.80) | 26 (7.65) | 31 (6.95) |
| QALY,a mean (SD) | 0.263 (0.132) | 0.285 (0.115) | 0.274 (0.120) | 0.291 (0.112) |
Cost-effectiveness
The intervention reduced the mean differences in costs by ≈£755 in the intervention period compared with the baseline period, but the mean effect was surrounded by statistical uncertainty. The mean incremental QALY gain for the POPPI intervention versus usual care was small (0.004), and with a 95% CI that included zero (Table 36). The INMB for the POPPI intervention versus usual care was positive at £835, but with considerable statistical uncertainty (95% CI –£4322 to £5992).
| Cost effectiveness | Time period, mean (SD) | Incremental effect, mean (95% CI) | |||
|---|---|---|---|---|---|
| Baseline | Intervention | ||||
| Intervention (n = 283) | Usual care (n = 284) | Intervention (n = 340) | Usual care (n = 446) | ||
| Costs (£) | 27,824 (23,793) | 26,176 (20,957) | 30,100 (25,403) | 29,618 (27,287) | –755 (–5883 to 4374) |
| EQ-5D-5L (survivors) | 0.661 (0.303) | 0.698 (0.268) | 0.668 (0.302) | 0.690 (0.279) | 0.009 (–0.063 to 0.081) |
| QALYs | 0.263 (0.132) | 0.285 (0.115) | 0.274 (0.120) | 0.291 (0.112) | 0.004 (–0.023 to 0.031) |
| INMB (£)a | 835 (–4322 to 5992) | ||||
When the uncertainty in the incremental costs and QALYs is represented on the cost-effectiveness plane, the majority of the points are in those quadrants that show that the POPPI intervention, on average, has lower costs and gains QALYs, although the magnitude of these average QALY gains were small (Figure 21). The probability that the POPPI intervention is more cost-effective than usual care, given the data, is ≈60% when willingness to pay for a QALY gain is zero; this probability is never > 65%, irrespective of how much society is willing to pay for a QALY gain (Figure 22).
FIGURE 21.
Uncertainty in the mean costs (GBP) and QALY differences and their distribution for the POPPI intervention versus usual care. Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
FIGURE 22.
Cost-effectiveness acceptability curve, reporting the probability that the POPPI intervention is cost-effective (within 6 months) at alternative willingness-to-pay thresholds for a QALY gain. Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
The estimated INMBs were similar for the scenarios considered in the sensitivity analyses (Figure 23). For example, the INMB is robust and remains ≈£835 regardless of the alternative assumptions considered in the sensitivity analysis, that is whether additional staff time is required to deliver the IPAT assessment or the stress support sessions, an alternative assumption for baseline HRQoL or an alternative distributional assumptions. Similarly, excluding the health services questionnaire costs had only a small impact on the mean INMB compared with base case INMB (£1346 vs. £835).
FIGURE 23.
The sensitivity analysis that reports the mean (95% CI) incremental net benefit (at £20,000 per QALY) within 6 months according to alternative assumptions compared with the base case. Vertical line indicates no difference in net monetary benefits between comparator groups. SMR, standardised mortality ratio. HSQ, health service questionnaire. Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
The results of the subgroup analysis are presented in Figure 24, and show that the INMBs were similar across all subgroups. For the ‘duration of delirium’ subgroup, the mean INMB associated with the POPPI intervention, compared with usual care, increased when delirium was of longer duration. For all subgroups, as for the overall results, the CIs around the INMB included zero.
FIGURE 24.
The INMB (95% CI) (GBP) at 6 months, by subgroups, at the recommended willingness to pay threshold stipulated by NICE (£20,000 per QALY). Vertical line indicates no difference in net monetary benefits between comparator groups. Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
Lifetime cost-effectiveness
Long-term survival
The Kaplan–Meier survival curves show that when the time horizon was extended beyond 6 months, for those with survival data available, the probability of survival was similar across treatment groups in both the baseline and the intervention periods (Figure 25). In the intervention period, survival was similar between the treatment groups at 6 months and over a longer follow-up period. In the baseline period, survival in the intervention group was lower than the usual care group at 6 months, but the survival difference between the randomised groups was minimised over time, even though a large proportion of cases were censored beyond 1 year.
FIGURE 25.
Kaplan–Meier survival curves. Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
Long-term health-related quality of life
The long-term CEA required the HRQoL to be estimated over time. At 6 months, HRQoL was similar across randomised groups and periods. We used HRQoL from the general population at the age of 61 years (which was the median age of POPPI cluster RCT 6-month survivors) to predict the long-term HRQoL of patients recruited to the cluster RCT. HRQoL of POPPI survivors at 6 months was 13% lower than that of the age- and sex-matched general population. 112 We used linear interpolation to predict the mean HRQoL for each treatment group between 6 months and 5 years; therefore, after 5 years, the mean for each randomised group was similar to that for the age-matched general population.
Long-term costs
To project lifetime costs attributable to the initial critical care episode, we considered the mean re-admission costs (in critical care, general wards, and outpatient and community services) up to 1 year, estimated from the cluster RCT electronic case report form, the CMP database and the health services questionnaires. The mean costs for each time period were calculated for those patients who survived to at least 1 year. For each intervention period, these mean costs were similar (£7043 for baseline and £7969 for the intervention period). These mean costs were used to impute mean costs between years 1 and 5. After year 5, we assumed no additional morbidity costs.
Cost-effectiveness
Table 37 presents the resultant lifetime QALYs, lifetime costs and INMBs. Overall, at the NICE-stipulated threshold of £20,000 per QALY, the INMB was £4158, but with a wide 95% CI that included zero. The cost-effectiveness acceptability curves show that that the probability of the POPPI intervention being cost-effective is ≈70% at the NICE-stipulated threshold of £20,000 per QALY (Figure 26).
| Lifetime cost-effectiveness | Time period, mean (SD) | Incremental effect, mean (95% CI) | |||
|---|---|---|---|---|---|
| Baseline | Intervention | ||||
| Intervention (n = 283) | Usual care (n = 284) | Intervention (n = 340) | Usual care (n = 446) | ||
| Costs (£) | 56,319 (25,020) | 56,193 (22,115) | 64,406 (26,482) | 64,254 (28,823) | 362 (–5077 to 5801) |
| QALYs | 10.58 (6.01) | 11.52 (5.82) | 10.89 (5.29) | 11.81 (5.51) | 0.226 (–0.447 to 0.899) |
| INB (£)a | 4158 (–10,354 to 18,670) | ||||
FIGURE 26.
Cost-effectiveness acceptability curve, reporting the probability that the intervention is cost-effective (at lifetime) at alternative willingness-to-pay thresholds for a QALY gain. Reproduced with permission from JAMA 2019;321(7):665–75. 71 Copyright © 2019 American Medical Association. All rights reserved.
Chapter 9 Discussion and conclusions
Principal findings
To our knowledge, this was the first, large, randomised evaluation of a complex preventive psychological intervention conducted in the challenging setting of critical care units in the NHS. The POPPI trial, a parallel-group, cluster RCT conducted in 24 adult general critical care units, indicated that, among adults who stayed at least 48 hours in the critical care unit and received level 3 intensive care, the delivery of a complex, nurse-led preventive psychological intervention provided no significant difference in the primary clinical outcome (PTSD symptom severity at 6 months) when compared with usual care (treatment effect estimate –0.03 points on the PSS-SR; 95% CI –2.58 to 2.52; p = 0.98). There was considerable statistical uncertainty with regards to the cost-effectiveness results. On average, the POPPI intervention showed decreased costs and slightly improved QALYs, leading to a positive INMB at 6 months (£835, 95% CI –£4322 to £5992). The probability that the POPPI intervention is cost-effective (at a willingness-to-pay threshold of £20,000 per QALY) is ≈60%. When extrapolated to the lifetime, the INMB was larger, although with even greater uncertainty.
There was no significant interaction between the effect of treatment allocation and PTSD symptom severity at 6 months in any of the prespecified subgroups: age, sex, socioeconomic status, duration of delirium, anxiety (measure by the STAI-6), surgical status, how well the intervention sites adhered to the intervention (overall intervention implementation score) or heterogeneity of treatment effect (predicted PSS-SR score). There were no significant differences in any of the six secondary clinical outcomes, including anxiety and depression. At 6 months, just over 20% of responders scored > 18 points on the PSS-SR (i.e. the threshold warranting further investigation for probable PTSD), indicating substantial ongoing psychological morbidity in this patient group.
We suggest the following reasons why this preventive, complex psychological intervention may have not been effective in reducing longer-term psychological morbidity:
-
Contrary to evidence suggesting that focusing on acute stress and its causes early in the ICU may help,113 patients may have still been too ill to absorb and put into practice the therapeutic messages taught in the stress support sessions. This is shown by the high numbers of patients being discharged from critical care prior to regaining the necessary cognitive capacity to consent. The POPPI nurses frequently reported patient-centred barriers to intervention delivery, including patient tiredness, confusion and dexterity problems, with patients focusing on short-term goals, primarily leaving the critical care unit. If an intervention was targeted later in a patient’s recovery trajectory, it could include more challenging techniques (e.g. exposure to traumatic memories), which may be needed to reduce longer-term psychological morbidity in this population. 114
-
Patients did not receive the full dose of the three stress support sessions. Although protocol adherence was high, with > 80% [this ranged across sites from 50% (lowest adherence) to 94% (highest adherence)] of patients receiving at least two sessions, approximately one-third did not receive session 3. As shown in Chapter 6, there was a short-term reduction in anxiety, measured using the STAI-6, for patients who received all three sessions and had data. The main reason for not receiving subsequent sessions was hospital discharge; therefore, a future intervention that follows patients into the community may be necessary.
-
As described in Chapter 6, Introduction, some of the POPPI nurses did have difficulties in delivering the sessions as they were intended, such as struggling to keep patients on track with the session content. This may have been because critical care nurses are non-experts with regards to psychological support. Qualified psychologists may be required to deliver such interventions, or non-experts may require more in-depth training, guidance and support to help with more complex patients and situations.
-
The translation of knowledge from an online training course into practice was varied at sites. Uptake was high (> 80% of staff) with regards to completion of the online training, but this may not have translated as required into practice. The POPPI nurses reported the challenge of changing long-standing practices in units and how all units have staff who are difficult to engage in any new practice. Some units were restricted in the degree to which they could implement change by the environmental limitations of the unit. In very open-plan small units, staff even found it difficult to address the more simple elements such as reducing noise levels and orientating patients with whiteboards or clocks. Moving forward, if an environment intervention were to be implemented in the NHS critical care setting, it is clear that some units would be able to adapt to the intervention more fully than others, but this should not be equated with staff resistance to compliance with recommendations.
Summaries of key research recommendations
-
Conducting research in this area is challenging and future research needs to factor in, when relevant, several challenges identified during the conduct of this cluster RCT. These include the short time period during which patients have mental capacity prior to discharge from the critical care unit, which affects the ability to consent and deliver a critical care unit intervention; the potential stigma associated with psychological intervention(s), which can negatively affect the process of attempting to gain consent; and releasing bedside nurses from their busy work schedules in order to attend training, deliver psychological interventions and participate in debriefing and support.
-
Prior to development and evaluation of any subsequent psychological intervention in the critical care unit, there is much to learn from post hoc analyses of the rich quantitative and qualitative data from this cluster RCT:
-
Further exploration of why the nurse-led complex psychological intervention in the cluster RCT did not produce an effect. Potential areas identified above worth exploring include whether or not the intervention was the right intervention; whether or not the timing and duration of the intervention were correct; whether or not the right people to deliver the intervention (non-experts) were used; and whether or not the necessary amount and type of training and support were delivered.
-
Further exploration from the cluster RCT data of the patient factors, including risk factors, mediators and subgroups of patients at a greater risk of acute stress or longer-term psychological morbidity who may be more likely to benefit from a future psychological intervention.
-
Implications for health care/practice
The results of the cluster RCT do not support the adoption of this complex, nurse-led preventive psychological intervention into routine practice.
However, the cluster RCT results do indicate that high levels of both acute stress in the critical care unit (64%) and longer-term PTSD symptom severity (20%) exist, and that these levels are in line with previous studies. 16,115 This area, therefore, remains a priority area for critical care and those involved in the longer-term follow-up of patients.
The POPPI cluster RCT has unearthed a number of important health-care factors that are vital to consider when assessing how best to alleviate acute stress and prevent psychological morbidity among critically ill patients. There is huge room for improvement with the routine assessment of delirium. Less than 50% of patients who were recruited had evidence of a CAM-ICU assessment documented in their medical records, despite all 24 cluster RCT sites reporting, at the point of site identification and selection, that they routinely use this tool. The lack of routine screening with CAM-ICU in the POPPI cluster RCT may reflect a perceived unreliability of the tool among critical care staff, or again, may be a function of patients being discharged from the critical care unit very soon after regaining mental capacity. Another key consideration identified is that, if critical care staff were to provide psychological support to patients, they would need to have fully protected time away from other clinical duties, to allow for the additional workload that this would entail. This is difficult to achieve in the confines of a research study, but those organising and managing clinical practice would need to thoroughly consider this issue, to ensure that high-quality support is provided without causing additional stress to those delivering the support. With regards to the fact that it was observed to be difficult to change aspects of unit culture (shown by the huge variation in the success of translating the online training into clinical practice), this has a potential knock-on effect for how a large part of mandatory training is delivered in the NHS, to make sure that knowledge is actually being translated into practice. The final health-care factor identified was regarding the earliness of discharge from critical care. Interventions need to take into account the fact that patients are often discharged without regaining mental capacity.
Strengths and limitations
Strengths
This large, parallel-group, cluster RCT was rigorously conducted in a challenging setting. There were a number of potential strengths of the conduct of the cluster RCT:
-
The intervention was rigorously developed and refined, adapted from relevant theories and techniques to alleviate acute stress and prevent longer-term psychological morbidity for critical care patients, according to the MRC guidance,45 which was led by experts with vast psychology and adult critical care experience.
-
The RCT built on a detailed feasibility phase, which tested and refined delivery of the trial processes, the education package, the support tools and the complex psychological intervention.
-
Site set-up was completed efficiently and on time, according to the pre-planned schedule, with only one site opening shortly after its scheduled start.
-
The RCT was pragmatic, conducted in a representative sample of NHS adult general critical care units, with delivery of the intervention (including key staff selection) determined locally, as would be the case if the intervention were to be adopted into routine NHS care.
-
A high proportion of eligible patients were identified and approached for consent (69%), with a total of 1458 patients recruited across 24 critical care units over the 17-month recruitment period.
-
Loss to follow-up on the important patient-centred psychological outcome, in this complex patient group at 6 months, was low. There were no important differences in baseline characteristics between patients who did and those who did not respond.
-
We used the recommended approach of multiple imputation to address missing data, and imputed missing values, conditional on all the information observed, including the vast data available through the CMP.
-
All analyses were conducted according to a prespecified, published statistical analysis plan. 70
-
We included prospectively designed integrated economic and process evaluations; the process evaluation was essential given the complexity of the intervention. The economic evaluation ensured that resource use data were collected on both primary admissions and re-admissions for each patient recruited, and harnessed information from four sources: the electronic case report form, linked data from the CMP database and NHS Digital, and responses to the follow-up health services questionnaires at 6 months.
Limitations
There were a number of potential limitations of the conduct of the cluster RCT:
-
As for many cluster RCTs, there is a risk of selection bias, with potentially different cohorts of patients recruited in different groups. 116 In the cluster RCT, there was an indication that refusals of consent were higher in the intervention sites during the intervention period than during other periods. We attempted to mitigate this issue by providing the option for patients to consent to receive solely the follow-up questionnaire, but this was rarely taken up. Even though slightly differential recruitment rates were seen, mainly attributable to the increase in refusals of consent, no important differences in baseline characteristics between time periods were observed.
-
The nurse-led, preventive, complex psychological intervention could not be blinded to those caring for patients, but the risk of bias was minimised through central random allocation after sites had already started recruiting to the cluster RCT (during the baseline period). The use of a patient-reported primary outcome measure was not subject to observer bias.
-
Despite all sites that signed up indicating that they routinely used the CAM-ICU to monitor delirium, actual monitoring with the CAM-ICU was poor, with over half of the recruited patients not being assessed for delirium.
-
Although the stress support sessions were designed to be commenced in the critical care unit, the proportion of patients actually receiving their first session in the critical care unit was lower (40%) than expected.
-
We were unable to assess fidelity to the stress support session delivery by audio-recording sessions. Probably owing to the nature of the interaction, it was reported by the POPPI nurses that this aspect was felt to be intrusive and would make patients feel uncomfortable. Alternatively, more detailed ethnographic methods could have been used, but this may not have resolved patients feeling uncomfortable.
-
To better understand delivery of the intervention, additional patient feedback could have been beneficial. Although patient feedback was sought during the intervention feasibility study, and that feedback confirmed acceptability of the stress support sessions, only a voluntary mechanism to provide feedback was provided in the cluster RCT. Unfortunately, this opportunity was not taken up. Feedback from relatives may have also been of interest; however, this was not sought.
-
HRQoL data at baseline were self-reported by patients using a visual analogue scale, which was chosen because a visual analogue scale is a simple tool that is quick to administer and easy to complete for patients in critical care. There are concerns over whether or not critical care patients are able to provide valid ratings using simple visual analogue scales. On the other hand, there are concerns over whether or not valid HRQoL data for patients can be obtained from proxies (e.g. carers or health professionals). 117 Considering this, we performed a sensitivity analysis considering alternative values of HRQoL at baseline, and found that the cost-effectiveness results are not sensitive to alternative values of HRQoL at baseline.
-
As there is currently no NICE-recommended value set specific to the EQ-5D-5L, we decided to value the descriptive EQ-5D-5L patient data, collected as part of the cluster RCT, using a published EQ-5D-5L value set for England,98 which is one of the first value sets reported that is relevant to this population. An alternative approach would have been to use available crosswalk value sets (e.g. from the NICE-recommended EQ-5D-3L value set to the EQ-5D-5L) via a ‘mapping algorithm’;118,119 however, this approach was not adopted as further research is ongoing to establish an optimum mapping algorithm. The choice of value set (EQ-5D-5L value set versus EQ-5D-3L to EQ-5D-5L mapping) may have important implications for the cost-effectiveness results. 120
-
The estimation of lifetime cost-effectiveness inevitably required us to make assumptions, in particular about the costs in the time period beyond the observed data. However, the study made maximum use of the available trial data to inform these assumptions.
Acknowledgements
We wish to thank the NIHR HSDR programme for funding this trial.
We are very grateful to all of the participants who gave their time to participate in the two POPPI feasibility studies and the cluster RCT.
We would also like to thank Samsung Electronics (UK) Ltd for their kind donation of 38 ATIV Tab 3 tablet computers for use by patients in the cluster RCT as part of the relaxation and recovery programme.
In addition, we thank the following people for their contributions to the set-up and delivery of the study: Steven Saunders, Robert Darnell, Sian Martin, Abby Koelewyn, Bronagh Blackwood, Donatella D’Antoni and Alexina Mason.
Research and clinical staff at participating sites
We acknowledged that there have been many other individuals who made a contribution in the participating sites. It is impossible to thank everyone personally; however, we would like to thank the following research and clinical staff:
Bristol Royal Infirmary (Katie Sweet, Dr Sanjoy Shah, Lisa Grimmer, Kate Driver, Denise Webster, Chloe Searles), Countess of Chester Hospital (Maria Faulkner, Dr Mary Cardwell, Paula Povey), Darlington Memorial Hospital (Dianne Cruickshank, Dr James Limb, Amanda Cowton, Nicola Hewitson, Sally Horsley, Dawn Cameron, Jane Jenkinson), Freeman Hospital (Claire Randell, Dr Matthew Faulds, Verity Calder, Craig Samson), Hull Royal Infirmary (Neil Smith, Dr Andrew Gratix, Vicky Martinson, Caroline Abernethy, Louise Chadwick, Emma Limbert, Helen Russell), James Cook University Hospital (Lindsay Garcia, Dr Stephen Bonner, Dr Isabel Gonzales, Keith Hugill, Anna Wilson, Angela Richter, Jessica Reid), Medway Maritime Hospital (Catherine Plowright, Claire Pegg, James Cullinane), Musgrove Park Hospital (Patricia Doble, Dr Richard Innes, Moira Tait, Kirsty Adams, Claire Payton-Crisp), Peterborough City Hospital (Alan Pope, Dr Coralie Carle), Poole Hospital (Julie Camsooksai, Dr Henrik Reschreiter, Sarah Patch, Sarah Jenkins, Lee Tbaily, Helena Barcraft-Barnes), Queen Alexandra Hospital (Steve Rose, Dr David Pogson, Lindsey Cooke, Sandra Taylor, Jacklyn Newman), Queen Elizabeth Hospital (King’s Lynn) (Denise Reid, Dr Mark Blunt, Melissa Rosbergen, Ruth Hodgson, David Melhado, Christine Medley), Queen’s Medical Centre (Claudia Washbrook, Dr Daniel Harvey, Lucy Ryan, Jodie Bradder, Sonya Finucane), Royal Berkshire Hospital (Nicola Jacques, Dr Andrew Walden, Ria McMullen, Dariusz Pabiancyk, Sarah MacGill), Royal Cornwall Hospital (Karen Burt, Dr Jonathan Paddle, Sarah Bean, Lynne Donohue, Nicola Powell, Matt Jones), Royal Gwent Hospital (Una Gunter, Dr Nicholas Mason, Leanne Pearce, Gemma Williams), St George’s Hospital (Christine Ryan, Dr Mark Hamilton, Veronica Barnes, Helen Farrah, Rachael Shepherdson, Johannes Mellinghoff), St James’s University Hospital (Zoe Beardow, Professor Mark Bellamy, Elizabeth Wilby, Clare Howcroft, Dee Taylor, Cherry Salutan-Cuevas, Dawn Stevenson),The Ipswich Hospital (Stephanie Bell, Heather Blaylock, Dr Richard Howard-Griffin, Sue Brixey), Ulster Hospital (Samantha Hagan, Dr John Trinder), University Hospital Coventry (Caroline Hill, Dr Christopher Bassford, Geraldine Ward), University College Hospital, London (Dr David Brealey, Emma Davies, Emma Thompsett, Rosalind Edwards, Magda Rocha, Jung Ryu, Georgia Bercades), Warwick Hospital (Ian Purcell, Dr Ben Attwood, Sophie Mason, Alexandra Green, Lisa Thomas, Penny Parsons), Watford General Hospital (Xiao Bei Zhao, Jennie Haydock, Kalpana Giri Ghimire, Lillian Norris, James Cunningham, Sarah-Jane Turner, Tracey Temple), Whiston Hospital (Emma Whitby, Dr Julie Wood, Susan Dowling, Caron Jones) and York Hospital (Danielle Wilcock, Dr Rinus Pretorious, Kate Howard, Gill Valentine, Katie Chambers).
Investigators
Professor Kathryn M Rowan (Chief Investigator), Dr Dorothy Wade (Lead Clinical Investigator), Mr David Aaronovitch, Professor Chris R Brewin, Professor Richard Grieve, Professor David A Harrison, Dr Sheila Harvey, Professor Monty Mythen, Dr Zia Sadique, Ms Deborah Smyth, Professor John Weinman and Mr John Welch.
Trial Management Group
Mr Paul R Mouncey (Chairperson), Ms Helen Antwi, Professor Chris R Brewin, Professor Richard Grieve, Professor David A Harrison, Dr Sheila Harvey, Mr Nicholas Hudson, Professor Monty Mythen, Mr Alvin Richards-Belle, Professor Kathryn M Rowan, Ms Deborah Smyth, Dr Dorothy Wade, Professor John Weinman, Mr John Welch and Dr Jerome Wulff.
Trial Steering Committee
Professor Sallie Lamb (Chairperson), Mr David Aaronovitch, Mr Phil Crow, Professor Jill Francis, Professor Daniel Freeman, Dr Christina Jones, Dr Jacobus Preller, Dr Suman Prinjha, Professor Kathryn M Rowan, Dr Dorothy Wade and Ms Jill Winter.
Data Monitoring and Ethics Committee
Professor Marion Campbell (Chairperson), Professor Deborah Cook and Dr Valerie Page.
Expert Psychology Advisory Group
Professor Daniel Freeman (Chairperson), Ms Nicole Als, Dr Vaughan Bell, Professor Chris Brewin, Dr Dane Goodsman, Ms Mags Harvey, Dr Sheila Harvey, Dr Dorothy Wade, Professor John Weinman, Mr John Welch and Mr Chris Whitman.
Contributions of authors
Paul R Mouncey (Head of Research) led management of the study; contributed to the acquisition, analysis and interpretation of the data; and drafted and critically reviewed the manuscript.
Dorothy Wade (Health Psychologist) conceived the study; contributed to the design of the study; led the development and delivery of the intervention; contributed to the acquisition, analysis and interpretation of the data; and drafted and critically reviewed the manuscript.
Alvin Richards-Belle (Trial Manager) supported management of the study; contributed to the acquisition, analysis and interpretation of the data; and drafted and critically reviewed the manuscript.
Zia Sadique (Assistant Professor in Health Economics) conceived the study; contributed to the design of the study and the analysis and interpretation of the data; and drafted and critically reviewed the manuscript.
Jerome Wulff (Statistician) contributed to the analysis of the data and critically reviewed the manuscript.
Richard Grieve (Professor of Health Economics Methodology) conceived the study; contributed to the design of the study and the analysis and interpretation of the data; and drafted and critically reviewed the manuscript.
Lydia M Emerson (MRC–Hubs for Trials Methodology Research (HTMR) Research Fellow) carried out the process evaluation; contributed to the acquisition, analysis and interpretation of the data; and drafted and critically reviewed the manuscript.
Chris R Brewin (Professor of Clinical Psychology) conceived the study; contributed to the design of the study and the interpretation of the data; and critically reviewed the manuscript.
Sheila Harvey (Senior Research Fellow) conceived the study, contributed to the design of the study, contributed to the acquisition and interpretation of the data, and critically reviewed the manuscript.
David Howell (Divisional Clinical Director) conceived the study; contributed to the design of the study and the interpretation of the data; and critically reviewed the manuscript.
Nicholas Hudson (Data Manager) contributed to the analysis and interpretation of the data, and critically reviewed the manuscript.
Imran Khan (Study Manager) carried out the process evaluation, contributed to acquisition, analysis and interpretation of the data, and drafted and critically reviewed the manuscript.
Monty Mythen (Professor of Anaesthesia & Critical Care) conceived the study; contributed to the design of the study and the interpretation of the data; and critically reviewed the manuscript.
Deborah Smyth (Senior Nurse) conceived the study; contributed to the design of the study and the interpretation of the data; and critically reviewed the manuscript.
John Weinman (Professor of Psychology as Applied to Medicines) conceived the study; contributed to the design of the study and the interpretation of the data; and critically reviewed the manuscript.
John Welch (Consultant Nurse) conceived the study; contributed to the design of the study and the interpretation of the data; and critically reviewed the manuscript.
David A Harrison (Head Statistician and Honorary Professor in Medical Statistics) conceived the study; contributed to the design of the study and the analysis and interpretation of the data; and drafted and critically reviewed the manuscript.
Kathryn M Rowan (Director of Scientific & Strategic Development/CTU Director and Honorary Professor) conceived the study; led the grant application and design of the study; contributed to the acquisition, analysis and interpretation of the data; and drafted and critically revised the manuscript.
Publications
Richards-Belle A, Mouncey PR, Wade D, Brewin CR, Emerson LM, Grieve R, et al. Psychological Outcomes following a nurse-led Preventative Psychological Intervention for critically ill patients (POPPI): protocol for a cluster-randomised clinical trial of a complex intervention. BMJ Open 2018;8:e020908.
Wade D, Als N, Bell V, Brewin C, D’Antoni D, Harrison DA, et al. Providing psychological support to people in intensive care: development and feasibility study of a nurse-led intervention to prevent acute stress and long-term morbidity. BMJ Open 2018;8:e021083.
Wulff J, Sadique Z, Grieve R, Howell D, Mouncey P, Wade D, et al. Psychological outcomes following a nurse-led preventative psychological intervention for critically ill patients trial: statistical and health economic analysis plan. J Intensive Care Soc 2018;19:281–6.
Wade DM, Mouncey PR, Richards-Belle A, Wulff J, Harrison DA, Sadique MZ, et al. Effect of a nurse-led preventive psychological intervention on symptoms of posttraumatic stress disorder among critically ill patients: a randomized clinical trial. JAMA 2019;321:665–75.
Data-sharing statement
All data requests should be submitted to the corresponding author for consideration. Access to available anonymised data or trial materials may be granted following review.
Patient data
This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.
Disclaimers
This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care.
References
- Intensive Care National Audit & Research Centre . Key Statistics from the Case Mix Programme – Adult, General Critical Care Units 2017. www.icnarc.org/Our-Audit/Audits/Cmp/Reports/Summary-Statistics (accessed 25 July 2018).
- Parker AM, Sricharoenchai T, Raparla S, Schneck KW, Bienvenu OJ, Needham DM. Posttraumatic stress disorder in critical illness survivors: a metaanalysis. Crit Care Med 2015;43:1121-9. https://doi.org/10.1097/CCM.0000000000000882.
- Nikayin S, Rabiee A, Hashem MD, Huang M, Bienvenu OJ, Turnbull AE, et al. Anxiety symptoms in survivors of critical illness: a systematic review and meta-analysis. Gen Hosp Psychiatry 2016;43:23-9. https://doi.org/10.1016/j.genhosppsych.2016.08.005.
- Rabiee A, Nikayin S, Hashem MD, Huang M, Dinglas VD, Bienvenu OJ, et al. Depressive symptoms after critical illness: a systematic review and meta-analysis. Crit Care Med 2016;44:1744-53. https://doi.org/10.1097/CCM.0000000000001811.
- Ballenger JC, Davidson JR, Lecrubier Y, Nutt DJ, Foa EB, Kessler RC, et al. Consensus statement on posttraumatic stress disorder from the International Consensus Group on Depression and Anxiety. J Clin Psychiatry 2000;61:60-6.
- Boscarino JA. A prospective study of PTSD and early-age heart disease mortality among Vietnam veterans: implications for surveillance and prevention. Psychosom Med 2008;70:668-76. https://doi.org/10.1097/PSY.0b013e31817bccaf.
- Dedert EA, Calhoun PS, Watkins LL, Sherwood A, Beckham JC. Posttraumatic stress disorder, cardiovascular, and metabolic disease: a review of the evidence. Ann Behav Med 2010;39:61-78. https://doi.org/10.1007/s12160-010-9165-9.
- Jones C, Bäckman C, Capuzzo M, Flaatten H, Rylander C, Griffiths RD. Precipitants of post-traumatic stress disorder following intensive care: a hypothesis generating study of diversity in care. Intensive Care Med 2007;33:978-85. https://doi.org/10.1007/s00134-007-0600-8.
- Samuelson KA, Lundberg D, Fridlund B. Stressful memories and psychological distress in adult mechanically ventilated intensive care patients – a 2-month follow-up study. Acta Anaesthesiol Scand 2007;51:671-8. https://doi.org/10.1111/j.1399-6576.2007.01292.x.
- Granja C, Gomes E, Amaro A, Ribeiro O, Jones C, Carneiro A, et al. JMIP Study Group . Understanding posttraumatic stress disorder-related symptoms after critical care: the early illness amnesia hypothesis. Crit Care Med 2008;36:2801-9. https://doi.org/10.1097/CCM.0b013e318186a3e7.
- Myhren H, Tøien K, Ekeberg O, Karlsson S, Sandvik L, Stokland O. Patients’ memory and psychological distress after ICU stay compared with expectations of the relatives. Intensive Care Med 2009;35:2078-86. https://doi.org/10.1007/s00134-009-1614-1.
- Davydow DS, Zatzick D, Hough CL, Katon WJ. A longitudinal investigation of posttraumatic stress and depressive symptoms over the course of the year following medical-surgical intensive care unit admission. Gen Hosp Psychiatry 2013;35:226-32. https://doi.org/10.1016/j.genhosppsych.2012.12.005.
- Davydow DS, Zatzick D, Hough CL, Katon WJ. In-hospital acute stress symptoms are associated with impairment in cognition 1 year after intensive care unit admission. Ann Am Thorac Soc 2013;10:450-7. https://doi.org/10.1513/AnnalsATS.201303-060OC.
- Wade DM, Howell DC, Weinman JA, Hardy RJ, Mythen MG, Brewin CR, et al. Investigating risk factors for psychological morbidity three months after intensive care: a prospective cohort study. Crit Care 2012;16. https://doi.org/10.1186/cc11677.
- Ely EW, Siegel MD, Inouye SK. Delirium in the intensive care unit: an under-recognized syndrome of organ dysfunction. Semin Respir Crit Care Med 2001;22:115-26. https://doi.org/10.1055/s-2001-13826.
- Wade D, Hardy R, Howell D, Mythen M. Identifying clinical and acute psychological risk factors for PTSD after critical care: a systematic review. Minerva Anestesiol 2013;79:944-63.
- Novaes MA, Knobel E, Bork AM, Pavão OF, Nogueira-Martins LA, Ferraz MB. Stressors in ICU: perception of the patient, relatives and health care team. Intensive Care Med 1999;25:1421-6. https://doi.org/10.1007/s001340051091.
- Jones C, Griffiths RD, Humphris G, Skirrow PM. Memory, delusions, and the development of acute posttraumatic stress disorder-related symptoms after intensive care. Crit Care Med 2001;29:573-80. https://doi.org/10.1097/00003246-200103000-00019.
- Wade DM, Brewin CR, Howell DC, White E, Mythen MG, Weinman JA. Intrusive memories of hallucinations and delusions in traumatized intensive care patients: an interview study. Br J Health Psychol 2015;20:613-31. https://doi.org/10.1111/bjhp.12109.
- Department of Health . Comprehensive Critical Care: A Review of Adult Critical Care Services 2000.
- National Institute for Health and Care Excellence (NICE) . Rehabilitation After Critical Illness 2009.
- National Institute for Health and Care Excellence (NICE) . Post-Traumatic Stress Disorder (PTSD): Evidence Update December 2013 2013. https://arms.evidence.nhs.uk/resources/hub/1031525/attachment (accessed 23 May 2019).
- Strøm T, Martinussen T, Toft P. A protocol of no sedation for critically ill patients receiving mechanical ventilation: a randomised trial. Lancet 2010;375:475-80. https://doi.org/10.1016/S0140-6736(09)62072-9.
- Treggiari MM, Romand JA, Yanez ND, Deem SA, Goldberg J, Hudson L, et al. Randomized trial of light versus deep sedation on mental health after critical illness. Crit Care Med 2009;37:2527-34. https://doi.org/10.1097/CCM.0b013e3181a5689f.
- Hatch R, McKechnie S, Griffiths J. Psychological intervention to prevent ICU-related PTSD: who, when and for how long?. Crit Care 2011;15. https://doi.org/10.1186/cc10054.
- Jones C, Bäckman C, Capuzzo M, Egerod I, Flaatten H, Granja C, et al. Intensive care diaries reduce new onset post traumatic stress disorder following critical illness: a randomised, controlled trial. Crit Care 2010;14. https://doi.org/10.1186/cc9260.
- Peris A, Bonizzoli M, Iozzelli D, Migliaccio ML, Zagli G, Bacchereti A, et al. Early intra-intensive care unit psychological intervention promotes recovery from post traumatic stress disorders, anxiety and depression symptoms in critically ill patients. Crit Care 2011;15. https://doi.org/10.1186/cc10003.
- Beck AT, Dozois DJ. Cognitive therapy: current status and future directions. Annu Rev Med 2011;62:397-409. https://doi.org/10.1146/annurev-med-052209-100032.
- Bighelli I, Salanti G, Huhn M, Schneider-Thoma J, Krause M, Reitmeir C, et al. Psychological interventions to reduce positive symptoms in schizophrenia: systematic review and network meta-analysis. World Psychiatry 2018;17:316-29. https://doi.org/10.1002/wps.20577.
- Tyrer P, Cooper S, Salkovskis P, Tyrer H, Crawford M, Byford S, et al. Clinical and cost-effectiveness of cognitive behaviour therapy for health anxiety in medical patients: a multicentre randomised controlled trial. Lancet 2014;383:219-25. https://doi.org/10.1016/S0140-6736(13)61905-4.
- Turkington D, Kingdon D, Rathod S, Hammond K, Pelton J, Mehta R. Outcomes of an effectiveness trial of cognitive-behavioural intervention by mental health nurses in schizophrenia. Br J Psychiatry 2006;189:36-40. https://doi.org/10.1192/bjp.bp.105.010884.
- Gould RA, Mueser KT, Bolton E, Mays V, Goff D. Cognitive therapy for psychosis in schizophrenia: an effect size analysis. Schizophr Res 2001;48:335-42. https://doi.org/10.1016/S0920-9964(00)00145-6.
- Rector NA, Beck AT. Cognitive behavioral therapy for schizophrenia: an empirical review. J Nerv Ment Dis 2001;189:278-87. https://doi.org/10.1097/00005053-200105000-00002.
- Pfammatter M, Junghan UM, Brenner HD. Efficacy of psychological therapy in schizophrenia: conclusions from meta-analyses. Schizophr Bull 2006;32:64-80. https://doi.org/10.1093/schbul/sbl030.
- Wykes T, Steel C, Everitt B, Tarrier N. Cognitive behavior therapy for schizophrenia: effect sizes, clinical models, and methodological rigor. Schizophr Bull 2008;34:523-37. https://doi.org/10.1093/schbul/sbm114.
- Erickson DH. Cognitive-behaviour therapy for medication-resistant positive symptoms in early psychosis: a case series. Early Interv Psychiatry 2010;4:251-6. https://doi.org/10.1111/j.1751-7893.2010.00184.x.
- Bernard M, Jackson C, Jones C. Written emotional disclosure following first-episode psychosis: effects on symptoms of post-traumatic stress disorder. Br J Clin Psychol 2006;45:403-15. https://doi.org/10.1348/014466505X68933.
- Mueser KT, Rosenberg SD, Xie H, Jankowski MK, Bolton EE, Lu W, et al. A randomized controlled trial of cognitive-behavioral treatment for posttraumatic stress disorder in severe mental illness. J Consult Clin Psychol 2008;76:259-71. https://doi.org/10.1037/0022-006X.76.2.259.
- Jackson C, Trower P, Reid I, Smith J, Hall M, Townend M, et al. Improving psychological adjustment following a first episode of psychosis: a randomised controlled trial of cognitive therapy to reduce post psychotic trauma symptoms. Behav Res Ther 2009;47:454-62. https://doi.org/10.1016/j.brat.2009.02.009.
- Jones C, Skirrow P, Griffiths RD, Humphris GH, Ingleby S, Eddleston J, et al. Rehabilitation after critical illness: a randomized, controlled trial. Crit Care Med 2003;31:2456-61. https://doi.org/10.1097/01.CCM.0000089938.56725.33.
- Cuthbertson BH, Rattray J, Campbell MK, Gager M, Roughton S, Smith A, et al. The PRaCTICaL study of nurse led, intensive care follow-up programmes for improving long term outcomes from critical illness: a pragmatic randomised controlled trial. BMJ 2009;339. https://doi.org/10.1136/bmj.b3723.
- Turkington D, Kingdon D, Turner T. Insight into Schizophrenia Research Group . Effectiveness of a brief cognitive–behavioural therapy intervention in the treatment of schizophrenia. Br J Psychiatry 2002;180:523-7. https://doi.org/10.1192/bjp.180.6.523.
- Brabban A, Tai S, Turkington D. Predictors of outcome in brief cognitive behavior therapy for schizophrenia. Schizophr Bull 2009;35:859-64. https://doi.org/10.1093/schbul/sbp065.
- Peters E, Landau S, McCrone P, Cooke M, Fisher P, Steel C, et al. A randomised controlled trial of cognitive behaviour therapy for psychosis in a routine clinical service. Acta Psychiatr Scand 2010;122:302-18. https://doi.org/10.1111/j.1600-0447.2010.01572.x.
- Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Medical Research Council Guidance . Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008;337. https://doi.org/10.1136/bmj.a1655.
- Wade D, Moon Z, Windgassen S, Weinman J. Nonpharmacological interventions to reduce short-term or long-term psychological stress in ICU patients: a systematic review. Critical Care 2015;19. https://doi.org/10.1186/cc14636.
- Wade DM. Prevalence and Predictors of Psychological Morbidity and Quality of Life After Discharge from Intensive Care 2010.
- Fowler D, Garety P, Kuipers E. Cognitive Behaviour Therapy For Psychosis: Theory And Practice. Chichester: John Wiley and Sons Ltd; 1995.
- Brewin CR. Re-experiencing traumatic events in PTSD: new avenues in research on intrusive memories and flashbacks. Eur J Psychotraumatol 2015;6. https://doi.org/10.3402/ejpt.v6.27180.
- Lazarus RS, Folkman S. Stress, Appraisal, and Coping. New York, NY: Springer Publishing Company; 1984.
- Wade DF, Moon Z, Windgassen SS, Harrison AM, Morris L, Weinman JA. Non-pharmacological interventions to reduce ICU-related psychological distress: a systematic review. Minerva Anestesiol 2016;82:465-78.
- Wade DM, Hankins M, Smyth DA, Rhone EE, Mythen MG, Howell DC, et al. Detecting acute distress and risk of future psychological morbidity in critically ill patients: validation of the intensive care psychological assessment tool. Crit Care 2014;18. https://doi.org/10.1186/s13054-014-0519-8.
- Bullock I, Davis M, Lockey A, Mackway-Jones K. Pocket Guide to Teaching for Clinical Instructors. Chichester: John Wiley and Sons Ltd; 2016.
- Improving Access to Psychological Therapies (IAPT) . Good Practice Guidance on the Use of Self-Help Materials Within IAPT Services 2010. https://serene.me.uk/helpers/iapt-self-help-good-practice-guide.pdf (accessed 3 July 2018).
- Wade D, Als N, Bell V, Brewin C, D’Antoni D, Harrison DA, et al. Providing psychological support to people in intensive care: development and feasibility study of a nurse-led intervention to prevent acute stress and long-term morbidity. BMJ Open 2018;8. https://doi.org/10.1136/bmjopen-2017-021083.
- Herdman M, Gudex C, Lloyd A, Janssen M, Kind P, Parkin D, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011;20:1727-36. https://doi.org/10.1007/s11136-011-9903-x.
- Foa EB, Cashman L, Jaycox L. The validation of a self-report measure of posttraumatic stress disorder: the Posttraumatic Diagnostic Scale. Psychol Assess 1997;9:445-51. https://doi.org/10.1037/1040-3590.9.4.445.
- Irwin M, Artin KH, Oxman MN. Screening for depression in the older adult: criterion validity of the 10-item Center for Epidemiological Studies Depression Scale (CES-D). Arch Intern Med 1999;159:1701-4. https://doi.org/10.1001/archinte.159.15.1701.
- Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials 2007;28:182-91. https://doi.org/10.1016/j.cct.2006.05.007.
- Hayes RJ, Bennett S. Simple sample size calculation for cluster-randomized trials. Int J Epidemiol 1999;28:319-26. https://doi.org/10.1093/ije/28.2.319.
- Brewin CR, Fuchkan N, Huntley Z, Robertson M, Thompson M, Scragg P, et al. Outreach and screening following the 2005 London bombings: usage and outcomes. Psychol Med 2010;40:2049-57. https://doi.org/10.1017/S0033291710000206.
- Richards-Belle A, Mouncey PR, Wade D, Brewin CR, Emerson LM, Grieve R, et al. Psychological Outcomes following a nurse-led Preventative Psychological Intervention for critically ill patients (POPPI): protocol for a cluster-randomised clinical trial of a complex intervention. BMJ Open 2018;8. https://doi.org/10.1136/bmjopen-2017-020908.
- Harrison DA, Brady AR, Rowan K. Case mix, outcome and length of stay for admissions to adult, general critical care units in England, Wales and Northern Ireland: the Intensive Care National Audit . Research Centre Case Mix Programme Database. Crit Care 2004;8:R99-111. https://doi.org/10.1186/cc2834.
- Sessler CN, Gosnell MS, Grap MJ, Brophy GM, O’Neal PV, Keane KA, et al. The Richmond Agitation–Sedation Scale: validity and reliability in adult intensive care unit patients. Am J Respir Crit Care Med 2002;166:1338-44. https://doi.org/10.1164/rccm.2107138.
- Teasdale G, Jennett B. Assessment of coma and impaired consciousness. A practical scale. Lancet 1974;2:81-4. https://doi.org/10.1016/S0140-6736(74)91639-0.
- Ehring T, Kleim B, Clark DM, Foa EB, Ehlers A. Screening for posttraumatic stress disorder: what combination of symptoms predicts best?. J Nerv Ment Dis 2007;195:1004-12. https://doi.org/10.1097/NMD.0b013e31815c1999.
- Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983;67:361-70. https://doi.org/10.1111/j.1600-0447.1983.tb09716.x.
- Ely EW, Inouye SK, Bernard GR, Gordon S, Francis J, May L, et al. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). JAMA 2001;286:2703-10. https://doi.org/10.1001/jama.286.21.2703.
- Carter BR, Hood K. Balance algorithm for cluster randomized trials. BMC Med Res Methodol 2008;8. https://doi.org/10.1186/1471-2288-8-65.
- Wulff J, Sadique Z, Grieve R, Howell D, Mouncey P, Wade D, et al. Psychological outcomes following a nurse-led preventative psychological intervention for critically ill patients trial: statistical and health economic analysis plan. J Intensive Care Soc 2018;19:281-6. https://doi.org/10.1177/1751143718755016.
- Wade DM, Mouncey PR, Richards-Belle A, Wulff J, Harrison DA, Sadique MZ, et al. Effect of a nurse-led preventive psychological intervention on symptoms of posttraumatic stress disorder among critically ill patients: a randomized clinical trial. JAMA 2019;321:665-75. https://doi.org/10.1001/jama.2019.0073.
- Spielberger CD, Gorsuch RL, Lushene R, Vagg PR, Jacobs GA. Manual for the State–Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press; 1983.
- Marteau TM, Bekker H. The development of a six-item short-form of the state scale of the Spielberger State–Trait Anxiety Inventory (STAI). Br J Clin Psychol 1992;31:301-6. https://doi.org/10.1111/j.2044-8260.1992.tb00997.x.
- Carpenter JR, Kenward MG, Carpenter JR, Kenward MG. Multiple Imputation and its Application. Chichester: John Wiley and Sons Ltd; 2013.
- Quartagno M, Carpenter JR. Multiple imputation for IPD meta-analysis: allowing for heterogeneity and studies with missing covariates. Stat Med 2016;35:2938-54. https://doi.org/10.1002/sim.6837.
- Department for Communities and Local Government (DCLG) . English Indices of Deprivation 2015. www.gov.uk/government/statistics/english-indices-of-deprivation-2015 (accessed 25 July 2018).
- Harrison DA, Parry GJ, Carpenter JR, Short A, Rowan K. A new risk prediction model for critical care: the Intensive Care National Audit. Research Centre (ICNARC) model. Crit Care Med 2007;35:1091-8. https://doi.org/10.1097/01.CCM.0000259468.24532.44.
- Royal College of Physicians (RCP) . National Early Warning Score (NEWS): Standardising the Assessment of Acute Illness Severity in the NHS 2012. www.rcplondon.ac.uk/projects/outputs/national-early-warning-score-news (accessed 4 May 2017).
- Wang R, Lagakos SW, Ware JH, Hunter DJ, Drazen JM. Statistics in medicine – reporting of subgroup analyses in clinical trials. N Engl J Med 2007;357:2189-94. https://doi.org/10.1056/NEJMsr077003.
- Campbell MK, Piaggio G, Elbourne DR, Altman DG. CONSORT Group . Consort 2010 statement: extension to cluster randomised trials. BMJ 2012;345. https://doi.org/10.1136/bmj.e5661.
- Knaus WA, Draper EA, Wagner DP, Zimmerman JE. APACHE II: a severity of disease classification system. Crit Care Med 1985;13:818-29. https://doi.org/10.1097/00003246-198510000-00009.
- Draper D. Rank-based robust analysis of linear models. I. Exposition and review. Statist Sci 1988;3:239-57. https://doi.org/10.1214/ss/1177012915.
- Maracy M, Dunn G. Estimating dose-response effects in psychological treatment trials: the role of instrumental variables. Stat Methods Med Res 2011;20:191-215. https://doi.org/10.1177/0962280208097243.
- Ferrando-Vivas P, Jones A, Rowan KM, Harrison DA. Development and validation of the new ICNARC model for prediction of acute hospital mortality in adult critical care. J Crit Care 2017;38:335-9. https://doi.org/10.1016/j.jcrc.2016.11.031.
- Iwashyna TJ, Burke JF, Sussman JB, Prescott HC, Hayward RA, Angus DC. Implications of heterogeneity of treatment effect for reporting and analysis of randomized trials in critical care. Am J Respir Crit Care Med 2015;192:1045-51. https://doi.org/10.1164/rccm.201411-2125CP.
- Steckler A, Linnan L. Process Evaluation for Public Health Interventions and Research. San Francisco, CA: Jossey-Bass; 2002.
- Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015;350. https://doi.org/10.1136/bmj.h1258.
- Moore G, Audrey S, Barker M, Bond L, Bonell C, Cooper C, et al. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Community Health 2014;68:101-2. https://doi.org/10.1136/jech-2013-202869.
- Krippendorff K. Content Analysis: An Introduction to its Methodology. Thousand Oaks, CA: Sage Publications Inc.; 2014.
- Chen YF, Hemming K, Stevens AJ, Lilford RJ. Secular trends and evaluation of complex interventions: the rising tide phenomenon. BMJ Qual Saf 2016;25:303-10. https://doi.org/10.1136/bmjqs-2015-004372.
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77-101. https://doi.org/10.1191/1478088706qp063oa.
- Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol 2013;13. https://doi.org/10.1186/1471-2288-13-117.
- National Institute for Health and Care Excellence (NICE) . Guide to the Methods of Technology Appraisal 2013 2013.
- Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. BMJ 2013;346. https://doi.org/10.1136/bmj.f1049.
- NHS Information Standards Board . Critical Care Minimum Data Set 2012.
- Curtis L, Burns A. Unit Costs of Health and Social Care 2016. Canterbury: Personal Social Services Research Unit, University of Kent; 2016.
- Department of Health and Social Care (DHSC) . NHS Reference Costs 2015–2016 2016.
- Devlin NJ, Shah KK, Feng Y, Mulhern B, van Hout B. Valuing health-related quality of life: an EQ-5D-5L value set for England. Health Econ 2018;27:7-22. https://doi.org/10.1002/hec.3564.
- van Hout B, Devlin N, Shah K, Feng Y, Mulhern B. An EQ-5D-5L Value Set for England: Final Model Results. London: Office of Health Economics; 2014.
- Manca A, Hawkins N, Sculpher MJ. Estimating mean QALYs in trial-based cost-effectiveness analysis: the importance of controlling for baseline utility. Health Econ 2005;14:487-96. https://doi.org/10.1002/hec.944.
- White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med 2011;30:377-99. https://doi.org/10.1002/sim.4067.
- Rubin DB. Multiple Imputation for Nonresponse in Surveys. New York, NY: J Wiley and Sons Ltd; 1987.
- Claxton K. Exploring uncertainty in cost-effectiveness analysis. PharmacoEconomics 2008;26:781-98. https://doi.org/10.2165/00019053-200826090-00008.
- Ramsey SD, Willke RJ, Glick H, Reed SD, Augustovski F, Jonsson B, et al. Cost-effectiveness analysis alongside clinical trials II-An ISPOR Good Research Practices Task Force report. Value Health 2015;18:161-72. https://doi.org/10.1016/j.jval.2015.02.001.
- Fischer K, Goetghebeur E, Vrijens B, White IR. A structural mean model to allow for noncompliance in a randomized trial comparing 2 active treatments. Biostatistics 2011;12:247-57. https://doi.org/10.1093/biostatistics/kxq053.
- Dolan P, Gudex C, Kind P, Williams A. A Social Tariff for EuroQoL: Results From a UK General Population Survey. York: Centre for Health Economics, University of York; 1999.
- Office for National Statistics . Interim Life Tables, 2011–2013 2013.
- Wright JC, Plenderleith L, Ridley SA. Long-term survival following intensive care: subgroup analysis and comparison with the general population. Anaesthesia 2003;58:637-42. https://doi.org/10.1046/j.1365-2044.2003.03205.x.
- Cuthbertson BH, Roughton S, Jenkinson D, Maclennan G, Vale L. Quality of life in the five years after intensive care: a cohort study. Crit Care 2010;14. https://doi.org/10.1186/cc8848.
- Dowdy DW, Eid MP, Sedrakyan A, Mendez-Tellez PA, Pronovost PJ, Herridge MS, et al. Quality of life in adult survivors of critical illness: a systematic review of the literature. Intensive Care Med 2005;31:611-20. https://doi.org/10.1007/s00134-005-2592-6.
- Ara R, Brazier JE. Populating an economic model with health state utility values: moving toward better practice. Value Health 2010;13:509-18. https://doi.org/10.1111/j.1524-4733.2010.00700.x.
- Brazier JE, Roberts J. The estimation of a preference-based measure of health from the SF-12. Med Care 2004;42:851-9. https://doi.org/10.1097/01.mlr.0000135827.18610.0d.
- Roberts MB, Glaspey LJ, Mazzarelli A, Jones CW, Kilgannon HJ, Trzeciak S, et al. Early interventions for the prevention of posttraumatic stress symptoms in survivors of critical illness: a qualitative systematic review. Crit Care Med 2018;46:1328-33. https://doi.org/10.1097/CCM.0000000000003222.
- Birur B, Moore NC, Davis LL. An evidence-based review of early intervention and prevention of posttraumatic stress disorder. Community Ment Health J 2017;53:183-201. https://doi.org/10.1007/s10597-016-0047-x.
- Davydow DS, Gifford JM, Desai SV, Needham DM, Bienvenu OJ. Posttraumatic stress disorder in general intensive care unit survivors: a systematic review. Gen Hosp Psychiatry 2008;30:421-34. https://doi.org/10.1016/j.genhosppsych.2008.05.006.
- Bolzern J, Mnyama N, Bosanquet K, Torgerson DJ. A review of cluster randomised trials found statistical evidence of selection bias. J Clin Epidemiol 2018;99:106-12. https://doi.org/10.1016/j.jclinepi.2018.03.010.
- Hofhuis J, Hautvast JLA, Schrijvers AJP, Bakker J. Quality of life on admission to the intensive care: can we query the relatives?. Intensive Care Med 2003;29:974-9. https://doi.org/10.1007/s00134-003-1763-6.
- van Hout B, Janssen MF, Feng YS, Kohlmann T, Busschbach J, Golicki D, et al. Interim scoring for the EQ-5D-5L: mapping the EQ-5D-5L to EQ-5D-3L value sets. Value Health 2012;15:708-15. https://doi.org/10.1016/j.jval.2012.02.008.
- Hernandez Alava MWA, Pudney S. Methods for Mapping Between the EQ-5D-5L and the 3L. Sheffield: University of Sheffield; 2017.
- Wailoo A, Alava M, Grimm S, Pudney S, Gomes M, Sadique Z, et al. Comparing the EQ-5D-3L and 5L Versions. What Are the Implications for Cost Effectiveness Estimates?. Sheffield: University of Sheffield; 2017.
Appendix 1 Medical Research Council framework for developing and evaluating complex interventions
Reproduced from Wade et al. 55 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. Includes minor additions and formatting changes to the original.
Appendix 2 Semistructured interview guide
Version 1.0, 3 August 2015.
-
What is usual practice in the unit regarding family/visitors?
-
Open/timed.
-
Limits on bedside numbers/crowding.
-
Do family members participate in patient care? Is there consensus in the unit on how this is done?
-
Family information leaflets, psychological support services for relatives.
-
-
Are steps taken to manage patient sleep within the unit? If so, which of the following are practised?
-
Circadian rhythm promotion.
-
Are there sleep protocols?
-
Non-pharmacological interventions, for example bright light therapy, music therapy.
-
Ear plug/eye-mask provision.
-
Are alarms turned down at night for clinically stable patients?
-
Grouping patient care activities.
-
-
What is your policy regarding PADS?
-
Do you have a PADS protocol?
-
Combined/separate?
-
Do you have a non-pharmacological delirium prevention policy?
-
How do you treat delirium?
-
Do staff receive any training/education in how to respond to patients experiencing hallucinations and delusions?
-
-
Psychological:
-
Do you psychologically assess patients?
-
Do they have access to a psychologist on the ICU?
-
Any psychological support currently offered in the unit?
-
What happens if a patient seems very distressed?
-
Can family members of distressed/delirious patients stay the night?
-
Follow-up clinic? Is it funded or taken from ICU budget? Does it involve a psychologist? Which patients are invited?
-
-
Serious incidents:
-
For example, CPR (cardiopulmonary resuscitation) – how are other patients protected, visually and in terms of noise?
-
Do they debrief patients after such incidents?
-
Appendix 3 Field observation data collection form
Version 1.0, 3 August 2015.
-
Patient orientation:
-
Clocks, within patient view.
-
Windows/natural light.
-
Patient-controlled bed lights.
-
Written names of their nurse and consultant.
-
Mementoes from home/photographs in patient’s eye-line.
-
Charts/posters to involve patients in planning their care/daily routine.
-
-
Unit/layout:
-
Bed-spaces – cramped or spacious.
-
Presence of equipment/clutter.
-
Any visual interest, for example murals?
-
Do large groups of clinical staff crowd round the bedside at times?
-
Any prompts for staff regarding communication or psychosocial care (displayed at bedside?) for example, communication policy, protocols for non-pharmacological intervention for delirium.
-
-
Communication and ward rounds (observe part of one)
-
Which staff attend the ward round?
-
Are patients always addressed and included?
-
Do the clinicians crowd round the computer?
-
Are patients encouraged to seek information they require?
-
Are explanations given and given in patient-friendly language?
-
Between clinicians and nursing staff, for example on ward round, do nurses provide information or actively contribute?
-
Do staff members introduce themselves by name?
-
Do nurses/doctors take time to discuss patient concerns, or do they appear too computer/task-focused?
-
Do they patiently give the information patients want in patient-friendly language?
-
Do staff speak kindly or brusquely?
-
Are patients being enabled to communicate (that is communication charts/speaking valves)?
-
Body language – do staff lean attentively towards patients?
-
Do they speak slowly, gently and clearly to disorientated/drowsy patients? (Document when witnessed.)
-
How do they respond to patients who have had hallucinations/delusions? (When visually apparent or told.)
-
Do they show a positive and optimistic demeanour to the patients?
-
Do they talk to each other over patients?
-
Do they listen to patients and give them space to think and ask questions?
-
Written information for patients.
-
-
Noise levels
-
Telephones ringing.
-
Alarms sounding.
-
Bin lids banging.
-
Staff talking and laughing loudly together in clinical areas.
-
Presence of visitors.
-
-
Relaxation and entertainment
-
Distraction/stimulation provided to awake patients, for example radio (tuned to patient preference), television, DVD players, iPads (Apple Inc., Cupertino, CA, USA).
-
Access to newspapers, books, films.
-
Complementary therapies, relaxation DVDs, music therapy.
-
Hairstyling/manicure.
-
Appendix 4 Telephone interview guide: intervention group sites
Version 1.0, 4 April 2016.
-
How would you say the POPPI trial has been received in your ICU?
-
Attitudes positive/negative.
-
Viewed as worthwhile.
-
Enthusiasm.
-
Staff competency? Attitudes vary by experience.
-
Inter-professional collaboration: medical and nursing staff contribute equally?
-
-
How did you find things went regarding recruitment?
-
Research nurses, doctors, nurses: involvement.
-
Patients/families: involvement, received how?
-
Screening.
-
Reach/eligibility.
-
-
Was there anything your unit did to make the trial/intervention fit/work better?
-
Steps taken to fit trial into given context.
-
Fit with current practice.
-
-
Fidelity:
-
Adaptations to routine/environment.
-
Overcoming ‘teething’ problems.
-
Protocol deviations.
-
-
How did the trial fit alongside other ‘routine’ clinical care/workload?
-
Re-prioritising tasks.
-
Availability of staff.
-
Clinically acceptable to clinicians/patients?
-
Appendix 5 Telephone interview guide: usual care group sites
Version 1.0, 4 April 2016.
-
How would you say the POPPI trial has been received in your ICU?
-
Feelings about being randomised to control.
-
Effect on attitudes.
-
Enthusiasm: maintained or waned?
-
Any change in practice/behaviour on the unit.
-
-
How did you find things went regarding recruitment?
-
Research nurses, doctors, nurses: involvement.
-
Patients/families: involvement, received how?
-
Screening.
-
Reach/eligibility.
-
-
Was there anything your unit did to make the trial fit/work better?
-
Screening and recruitment: steps taken to fit trial into context.
-
Adaptations to routine/environment.
-
Overcoming ‘teething’ problems.
-
-
How did the trial fit alongside other ‘routine’ clinical care/workload?
-
Re-prioritising tasks.
-
Availability of staff.
-
Acceptability.
-
Appendix 6 Site visit interview guide: intervention group sites
Version 1.0, 6 February 2017.
-
How would you say the POPPI trial has been received in your ICU?
-
Attitudes positive/negative.
-
Viewed as worthwhile.
-
Enthusiasm maintained?
-
Variance by grade/profession.
-
Research in general received how – POPPI different from other studies?
-
-
How was the study implemented into the unit?
-
Awareness.
-
Cascaded down through team.
-
Training/education of staff.
-
-
Did the study have an impact on daily routine?
-
Workload.
-
Adaptations: anything changed to make it work better.
-
Re-prioritising tasks.
-
Availability of staff.
-
Clinical acceptability to staff/patients.
-
-
How did you find things went regarding recruitment?
-
Research nurses, doctors, nurses.
-
Patients/families.
-
Screening and IPAT.
-
Patient refusals – why? Strategies to overcome?
-
-
In your opinion, were all eligible patients entered into the study
-
Reach/eligibility.
-
-
Fidelity:
-
Were there times you could not deliver all 3 stress support sessions?
-
Ability to deliver all component parts within each stress support sessions?
-
Why?
-
Using the online training in practice.
-
Delivery of the relaxation and recovery programme.
-
-
How did you manage any problems?
-
In your view has practice on the unit changed from what it was before the study? How?
-
The purpose of the POPPI intervention is to find out if delivering this intervention improves patients’ well-being after a stay in the intensive care unit:
-
Do you think an intervention like this will have made a difference to patients’ psychological well-being?
-
When study ends, do you think it could continue when research team no longer running it?
-
Who would be responsible?
-
Appendix 7 Site visit interview guide: usual care group sites
Version 1.0, 6 February 2017.
-
How would you say the POPPI trial has been received in your ICU?
-
Attitudes positive/negative.
-
Viewed as worthwhile.
-
Impact of control status.
-
Enthusiasm maintained?
-
Variance by grade/profession.
-
Research in general received how – POPPI different from other studies?
-
-
How was the study implemented into the unit?
-
Awareness.
-
Cascaded down through team.
-
Training/got to grips how?
-
-
Did the study have an impact on daily routine?
-
Workload.
-
Adaptations: anything changed to make it work better.
-
Re-prioritising tasks.
-
Availability of staff.
-
Clinically acceptable to clinicians/patients?
-
-
How did you find things went regarding recruitment?
-
Research nurses, doctors, nurses.
-
Patients/families – receptive?
-
Screening and IPAT.
-
Acceptability of IPAT where no further support/intervention offered.
-
Is informal support provided ad hoc, after IPAT?
-
Patient refusals – why? Strategies to overcome?
-
-
In your opinion were all eligible patients entered into the study?
-
Reach/eligibility.
-
-
In your view has practice on the unit changed from what it was before the study? How?
-
How did you manage any problems?
Appendix 8 Data extraction framework: intervention group sites
Appendix 9 Data extraction framework: usual care group sites
Appendix 10 Phase II interview themes and subthemes
| Themes | Subthemes |
|---|---|
| Attitudes and perceptions towards the trial and intervention |
|
| Implementation of the study into the unit |
|
| Impact on daily routine |
|
| Recruitment |
|
| Patient eligibility |
|
| Fidelity |
|
| Managing problems |
|
| Impact on previous practice | Practice change from baseline |
| Perceptions of effectiveness and feasibility |
|
Appendix 11 Debriefing and support: trainer’s perspective
Group discussions
Points raised during group discussion at the site (during the site visit when all nurses from a site had delivered sessions to at least one patient) included the following.
Three stress support sessions
Training course
All participants found the 3-day training course interesting and stimulating. Many nurses felt confident after the training, whereas others found it overwhelming, intense or very tiring. Several said that the training should have lasted 4 days to reduce overload. Resources such as the training manual, session summaries, structured notes and glossaries of useful words and phrases were well received, and helped nurses to feel more confident when revising after the training course.
Delivering the sessions
Timing
Nurses reported that it was difficult to get time and cover by colleagues to deliver the sessions. Time needed for sessions included time to prepare and make notes before and after sessions. More time could be lost if relatives or staff were busy with patients, or patients were too tired at the appointed time. In response, the POPPI trainers tried to problem-solve with PIs how the POPPI nurses could be better supported to be able to go and deliver sessions.
Privacy
It was hard to find privacy to conduct sessions, particularly on wards outside critical care. The POPPI nurses were interrupted by ward doctors or visiting teams. It was decided that the POPPI nurses would ask bedside nurses to explain to visitors that this was protected psychological time, and that the patient should not be disturbed, as long as there was no emergency.
Emotions
The POPPI nurses sometimes felt guilty about going off shift and leaving the patient whom they were looking after clinically with another nurse. Some felt tearful after stress support sessions and felt that they needed time to recover before returning to their sick patient. In a very few cases, POPPI nurses reported to the trainers that other staff – ‘tough types, with big egos’ – were quite negative about the POPPI intervention, as recorded by one of the POPPI trainers during the debriefing and support calls with a POPPI nurse. In response, trainers reiterated that the POPPI nurses could contact them at any time for support. Nurses also said that they were using peer support to discuss cases and help each other emotionally.
Confidence
Most POPPI nurses felt confident in delivering the content of sessions, but there were some issues. Sessions were harder to deliver with real patients, rather than actors (as in the training course) or trainers (as in the skills development assessment). Many were worried about dealing with patients who had self-harmed or overdosed, or raised serious issues, such as abuse. Advice here was that the POPPI nurses could discuss this type of issue with their clinical team, the hospital mental health liaison team or with the POPPI trainers.
Relaxation and recovery programme
Nurses reported that patient recovery stories were particularly popular, but not all patients wanted to use the tablet computers.
Debriefing and support calls
First call
The first round of telephone calls with nurses focused mainly on confidence building. Nurses said that it felt strange talking to patients in a psychological role. Some had concerns about using unfamiliar words and phrases, ‘mumbo-jumbo’, and preferred to say things in their own words. Trainers told them that this was fine, as long as the meaning was not lost. Some nurses said that they were worried about not knowing what to say, or coming across as patronising. They were reassured that these thoughts were probably more in their own head and that patients were unlikely to perceive them in that way, and were given positive reinforcement about what they were doing right. New ways of talking to patients were modelled by trainers.
There was some uncertainty about specific therapeutic techniques such as identifying stressful thoughts and teaching patients how to ‘check out their fear’, taught in session 2. Trainers explained these techniques again, giving examples and pointing nurses to sections of the manual they could revise. A lot of encouragement was given to keep trying as it would get easier with practice.
Some nurses were also unsure about the best way to help patients create action plans in session 3. Trainers explained how to encourage patients to come up with ideas for goals that were achievable. Nurses were also reminded that action plans were about wider well-being, so goals should not be only physical. Nurses should help patients come up with goals if patients could not think of anything themselves.
Some nurses had patients who did not want to open up. Advice was given on gently coaching patients to say what was on their mind without pushing them. Nurses were reminded of the teaching video used on the training course, which modelled lots of good techniques. Attention was drawn to the glossary of the manual, which contained lots of useful phrases to use to encourage patients to be open. Trainers also reminded nurses that it is OK to have pauses and silences during sessions – and that this would give patients time to think about and express what was on their mind.
Nurses talked about getting muddled in their early sessions, and finding it hard to stick to the structure of the sessions. It was easy to drift into having a general chat and feeling like you were not really doing anything. They were advised to stick closely to the structure of the sessions, or sessions could be less helpful. Sometimes, patients mentioned so many problems that it was hard to know what to focus on. Trainers recommended techniques, such as stopping and summarising what the patient had told them, to regain control of the session, and also asking patients which were the most important problems to focus on.
At first, quite a few nurses said that patients did not want to or did not know how to use the tablet computers. Trainers encouraged nurses to ask bedside nurses and relatives to help with the computer, and to give patients better explanations of the rationale for using the computer, as a key part of the POPPI intervention. We asked nurses to spend more time familiarising themselves with the contents of the app and the section of the manual that explains which exercises might be useful for particular problems. They should be prescriptive about which practices, such as mindfulness and relaxation, they wanted patients to do between sessions.
Problems could occur if nurses did not connect with a patient personally, for example an alcoholic who reminded one nurse of someone similar in their family who had caused problems. Advice here was to try to depersonalise your responses to them, and to get support from a colleague or a trainer.
Finally, trainers encouraged nurses to join the monthly POPPI nurse teleconferences to get more peer support and useful tips.
Subsequent calls
During calls 2 and 3, many POPPI nurses, particularly those in sites where recruitment was high, had noticeably grown in confidence, and showed a good grasp of key concepts. These nurses found delivering the sessions more satisfying as time went on. Certain units reported that the intervention was making a change: ‘All the clocks are up, everyone is talking to patients in a better way, there has been a culture shift.’
Appendix 12 Full results of the primary clinical effectiveness outcome model
| Variables | Coefficient | 95% CI | p-value |
|---|---|---|---|
| Fixed effects at the site level | |||
| Teaching status of hospitala | 0.514 | ||
| Non-teaching | 0.00 | ||
| Teaching | –0.65 | –2.62 to 1.31 | |
| Number of beds in the critical care unit (per additional bed)a | –0.10 | –0.36 to 0.16 | 0.452 |
| Number of admissions receiving level 3 care staying at least 48 hours during the pre-trial period of 1 April 2014 to 31 March 2015 (per additional 100 admissions)a | 0.26 | –0.53 to 1.06 | 0.515 |
| Allocated treatment group | 0.263 | ||
| Usual care | 0.00 | ||
| Intervention | 1.22 | –0.92 to 3.36 | |
| Fixed effects at the patient level | |||
| Time period | 0.993 | ||
| Baseline period | 0.00 | ||
| Intervention period | –0.01 | –1.72 to 1.71 | |
| Interaction between time period and treatment group | –0.03 | –2.58 to 2.52 | 0.981 |
| Age in years (restricted cubic splines, 4 knotsb) | < 0.001 | ||
| Age spline 1 | 0.04 | –0.10 to 0.17 | |
| Age spline 2 | –0.39 | –0.67 to –0.11 | |
| Age spline 3 | 1.74 | 0.06 to 3.41 | |
| Sex | < 0.001 | ||
| Female | 2.35 | 1.06 to 3.65 | |
| Male | 0.00 | ||
| Ethnicity | 0.367 | ||
| White | 0.00 | ||
| Non-white | –2.25 | –6.20 to 1.69 | |
| Not stated | –1.38 | –4.25 to 1.49 | |
| Quintile of IMD 2015 | 0.006 | ||
| 1 (least deprived) | 0.00 | ||
| 2 | –0.61 | –2.48 to 1.26 | |
| 3 | 0.46 | –1.47 to 2.39 | |
| 4 | 1.62 | –0.35 to 3.59 | |
| 5 (most deprived) | 2.86 | 0.81 to 4.91 | |
| Pre-existing anxiety/depression | < 0.001 | ||
| Anxiety | –0.66 | –5.39 to 4.08 | |
| Depression | 5.70 | 3.41 to 7.98 | |
| Both | 8.13 | 4.86 to 11.41 | |
| None | 0.00 | ||
| Planned admission following elective/scheduled surgery | 0.788 | ||
| No | 0.00 | ||
| Yes | 0.33 | –2.07 to 2.73 | |
| ICNARC Physiology Score from the first 24 hours following admission to the critical care unit (restricted cubic splines, 4 knotsc) | 0.752 | ||
| ICNARC Physiology Score spline 1 | –0.08 | –0.48 to 0.31 | |
| ICNARC Physiology Score spline 2 | 0.43 | –0.86 to 1.71 | |
| ICNARC Physiology Score spline 3 | –1.46 | –5.96 to 3.04 | |
| Constant | 11.84 | 3.90 to 19.78 | 0.003 |
| Random effects parameters: | |||
| Cluster-level variance | 0.65 | 0.01 to 62.05 | |
| Individual-level variance | 93.02 | 84.89 to 101.92 | |
| ICC | 0.007 | 0.000 to 0.401 | |
List of abbreviations
- APACHE
- Acute Physiology and Chronic Health Evaluation
- CAM-ICU
- Confusion Assessment Method for the Intensive Care Unit
- CBT
- cognitive–behavioural therapy
- CEA
- cost-effectiveness analysis
- CES-D-10
- Center for Epidemiologic Studies Depression Scale – Short Form
- CI
- confidence interval
- CMP
- Case Mix Programme
- CONSORT
- Consolidated Standards of Reporting Trials
- CRN
- Clinical Research Network
- CTU
- Clinical Trials Unit
- DMEC
- Data Monitoring and Ethics Committee
- DVD
- digital versatile disc
- EPAG
- expert psychology advisory group
- EQ-5D-5L
- EuroQol-5 Dimensions, five-level version
- GLMM
- generalised linear mixed model
- GP
- general practitioner
- HADS
- Hospital Anxiety and Depression Scale
- HRG
- Healthcare Resource Group
- HRQoL
- health-related quality of life
- HSDR
- Health Services and Delivery Research
- ICC
- intracluster correlation coefficient
- ICNARC
- Intensive Care National Audit & Research Centre
- ICU
- intensive care unit
- IMD
- Index of Multiple Deprivation
- INMB
- incremental net monetary benefit
- IPAT
- Intensive care Psychological Assessment Tool
- IQR
- interquartile range
- ISRCTN
- International Standard Randomised Controlled Trial Number
- IT
- information technology
- ITT
- intention to treat
- MAR
- missing at random
- MCAR
- missing completely at random
- MRC
- Medical Research Council
- NEWS
- National Early Warning Score
- NICE
- National Institute for Health and Care Excellence
- NIHR
- National Institute for Health Research
- PADS
- pain, agitation, delirium, sedation
- PI
- principal investigator
- PIS
- patient information sheet
- POPPI
- Psychological Outcomes following a nurse-led Preventative Psychological Intervention for critically ill patients
- PSS-SR
- Post-traumatic Stress Disorder Symptom Scale – Self-Report
- PTSD
- post-traumatic stress disorder
- QALY
- quality-adjusted life-year
- RASS
- Richmond Agitation–Sedation Scale
- RCI
- Reliable Change Index
- RCT
- randomised clinical trial
- R&D
- research and development
- REC
- Research Ethics Committee
- SD
- standard deviation
- STAI-6
- State–Trait Anxiety Inventory – Short Form
- TMG
- Trial Management Group
- TSC
- Trial Steering Committee
- UCLH
- University College London Hospitals NHS Foundation Trust