A national quality improvement programme to improve survival after emergency abdominal surgery: the EPOCH stepped-wedge cluster RCT

Evidence-based care pathway (37 recommended processes of care)

The EPOCH trial care pathway was developed through an evidence-based Delphi consensus process to update existing guidelines published by the Royal College of Surgeons. 8 The purpose of the pathway was to define the gold standard of care for this patient group. Evidence for the component interventions was assessed using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) criteria. 28 The 37 component interventions are detailed in Box 1 and a graphical display, designed for the QI programme to show how the patient may move along the care pathway, is in Figure 1.

FIGURE 1.

The EPOCH trial care pathway. ABG, arterial blood gas; CCOT, critical care outreach team; CT, computerised tomography; EWS, early warning score; NMB, neuromuscular blockade; SIRS, systemic inflammatory response syndrome; VTE, venous thromboembolism; WHO, World Health Organization. Reproduced from Stephens et al. 22 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. The figure includes minor additions and formatting changes to the original figure.

Reproduced from Stephens et al. 22 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. The figure includes minor additions and formatting changes to the original figure.

The quality improvement intervention, comprising six core improvement strategies

Two clinicians with QI and training expertise (TS and CJP) developed the programme theory for the QI intervention, defining ‘the how’ and ‘the why’ of the QI intervention (Box 2 and Table 1). The EPOCH trial programme theory was based on current evidence and learning from a range of other QI programmes. 29–32 Six QI strategies were developed to facilitate the translation of the programme theory into practice by local QI leads at their hospitals. These were intended as a minimum set of activities for QI leads and colleagues to undertake. The strategies were:

1. QI leads hold a stakeholder meeting after activation.

2. Each hospital forms an interprofessional improvement team.

3. QI leads analyse their own data (NELA data ± case note reviews and local audit data) and feed back to colleagues regularly.

4. QI leads and team members use time series charts (‘run charts’) to inform progress.

5. QI leads and team members segment the patient pathway to assist implementation planning.

6. QI leads and team members use plan–do–study–act (PDSA) cycles to support process change.

Table 1 details the relationship between the EPOCH trial programme theory, the QI resources available and the QI strategies proposed.

Trial design and participants

The EPOCH trial was a multicentre, stepped-wedge cluster randomised trial of a QI intervention to promote the implementation of a perioperative care pathway for patients undergoing emergency abdominal surgery. The trial protocol was published prospectively by The Lancet (protocol 13PRT/7655) and on the trial website (URL: www.epochtrial.org/protocol; accessed 18 July 2019). The trial was prospectively registered at isrctn.com on 27 February 2014, but a registration number was not issued until 7 March 2014 (ISRCTN80682973).

NHS hospitals delivering an emergency general surgical service were eligible for inclusion, provided they undertook a significant volume of emergency abdominal surgery cases and contributed data to the NELA. Hospitals were required to nominate specialty leads from surgery, anaesthesia and critical care, and to secure support from their NHS trust board or equivalent. Hospitals that were already implementing a care pathway to improve treatment for this patient group were excluded. Patients were eligible for inclusion in the data analysis if they were aged ≥ 40 years and undergoing emergency open abdominal surgery in a participating hospital during the 85-week trial period (from 3 March 2014 to 19 October 2015). Patients were excluded from the analysis if they were undergoing a simple appendicectomy, surgery related to organ transplant, gynaecological surgery, laparotomy for traumatic injury, treatment of complications of recent elective surgery or if they had previously been included in the EPOCH trial.

Data collection

Trial data were collected through the NELA database (URL: www.nela.org.uk; accessed 6 September 2019) and then linked using unique patient identifiers to Hospital Episode Statistics (HES) and Office for National Statistics (ONS) in England and Wales, and the Information Services Division of NHS Scotland, to provide data describing mortality and hospital readmissions. The trial was approved by the East Midlands (Nottingham 1) Research Ethics Committee (reference number 13/EM/0415). Data were analysed without individual patient consent in accordance with section 251 of the National Health Services Act 2006. 45

Randomisation and masking

We planned to include 15 geographical clusters of five to seven hospitals. The QI intervention lasted 85 weeks, with one geographical cluster commencing the intervention each 5-week step from the 2nd to the 16th time period. Clusters were randomly assigned to 1 of 15 start dates for the QI intervention by an independent statistician, using a computer-generated random allocation sequence. As each geographical area started in the usual care group and ended in the QI group, there were 17 time periods in total. Local investigators in each geographical area were notified 12 weeks in advance of activation of the QI programme at their hospital. As they were engaged in delivery of the intervention, it was not possible to mask hospital staff. Patients were masked to trial group allocation. The organisation of hospitals into geographical clusters minimised any contamination between sites due to natural workforce movements between hospitals.

Trial intervention

The EPOCH trial care pathway and QI methodology are described in Chapter 2.

Outcome measures

The primary outcome measure was all-cause mortality within 90 days following surgery. Secondary outcomes were all-cause mortality within 180 days following surgery, duration of hospital stay after surgery and hospital readmission within 180 days of surgery. We selected 10 predefined process measures (key components of the care pathway) for inclusion in the main report: (1) consultant-led decision to operate; (2) consultant review of patient before surgery; (3) preoperative documentation of risk; (4) time from decision to operate to entry into operating theatre; (5) patient entered operating theatre within time frame specified by their urgency (< 2 hours, 2–6 hours, 6–18 hours, or > 18 hours); (6) consultant surgeon present in operating theatre; (7) consultant anaesthetist present in operating theatre; (8) cardiac output-guided fluid therapy used during surgery; (9) serum lactate measured at end of surgery; and (10) critical care admission immediately after surgery.

Statistical analysis

A stepped-wedge design was chosen to improve statistical power by facilitating within-cluster comparison. Sample size calculations were based on the Hussey and Hughes approach,46 for an analysis with fixed time effects and random cluster effects, modified to exclude data collected during the 5-week period in which the intervention commenced in individual clusters. Using HES data, we estimated that 27,540 eligible patients would be registered across 90 NHS hospitals over 85 weeks, with a 90-day mortality rate of 25% in the usual care group and a between-hospital coefficient of variation of 0.15. Assuming a constant case load (18 patients/5 weeks/hospital), independent hospital effects and a 5% significance level, the trial would have 92% power to detect a reduction in 90-day mortality from 25% to 22%. If the assumption of independent hospital effects was not met, and the 15 geographical clusters functioned effectively as 15 large hospitals, power would be reduced to 83%.

All analyses were conducted according to intention-to-treat principles. All eligible patients with available outcome data were included in the analysis and analysed according to the randomisation schedule. 47 Patients who presented during the 5-week time period immediately after QI activation were excluded from the analysis. Hospitals that initially agreed to participate but subsequently withdrew prior to the trial start date were excluded; however, hospitals that withdrew after the trial start date, or did not implement the intervention, were included in the analysis. Hospitals that merged with other hospitals during the trial period were included in the analysis up to the point of the merger.

We were unable to procure data describing survival status after hospital discharge for patients in Wales. We therefore changed our primary analysis from binary to a time-to-event approach allowing inclusion of mortality events censored at hospital discharge. This affected 909 patients in Wales, 179 (20%) of whom died in hospital and 730 (80%) of whom were censored at hospital discharge. All analyses included time period as a fixed effect using indicator variables, and were adjusted for age, sex and indication for surgery using fixed factors. 48 Age was included as a continuous covariate, assuming a linear association with outcome. 49 Missing baseline data for indication for surgery were handled using a missing indicator approach. 50 All-cause mortality within 90 days of surgery was analysed using a mixed-effects parametric survival model with a Weibull survival distribution. The model included random intercepts for geographical area, hospital and hospital period (i.e. the time period in hospital). This allowed additional correlation between patients in the same hospital and the same period, compared with patients in other periods, as is recommended. 51,52 All-cause mortality within 180 days was analysed using the same approach. Duration of hospital stay was analysed using competing risk time-to-event models, with mortality before the outcome event acting as the competing risk and robust standard errors to account for clustering by geographical area. The hazard ratio (HR) from this analysis measures the relative probability of hospital discharge between treatment arms, with a HR of < 1 indicating a lower probability of discharge in the QI group (and therefore longer hospital stay). Hospital readmission within 180 days was analysed using the same approach (with a HR of < 1 indicating a lower probability of readmission).

We performed two secondary analyses for the primary outcome. The first evaluated the effect of the intervention over time. This analysis included patients who presented to hospital during the 5-week period immediately after implementation of the intervention. We analysed patients according to the following four groups: (1) no QI implemented (usual care group); (2) QI implemented for < 5 weeks; (3) QI implemented for between 5 and 10 weeks; and (4) QI implemented for > 10 weeks. Our second analysis evaluated the intervention in other patient populations that may have been affected by the intervention. This included patients who either underwent laparoscopic surgery or were aged 18–40 years, and who met all other eligibility criteria. Owing to the small number of patients in this group, we summarised results descriptively rather than undertaking a formal statistical analysis.

Patient and public involvement

We actively involved patient and public representatives throughout this project. Our co-applicants on the original funding application included two patient representatives, one of whom has lived experience of emergency abdominal surgery. They have supported us in the oversight and conduct of the trial, advising on numerous issues from the preparation of patient-facing materials to advising on the patient perspective of registry data use without individual patient consent. Both are named authors of this report and have specifically helped in drafting the Plain English summary.

Process measures

Ninety-one out of 93 (98%) hospitals were represented at the initial QI meeting for the relevant geographical cluster and 53 out of 93 (57%) hospitals were represented at the follow-up QI meeting. This representation included a named hospital QI lead for 89 out of 93 (96%) hospitals at the first meeting and 47 out of 93 (51%) hospitals at the second meeting. Most meetings (13/15) occurred within 2 weeks of the activation date. Patient-level process measures are described in Table 4. In accordance with our analysis plan, we did not test these for statistical significance.

Clinical outcomes

Complete primary outcome data were available for > 99% patients (see Figure 2). The primary outcome of 90-day mortality occurred in 1393 usual care group patients (16%) compared with 1210 QI group patients (16%) [QI vs. usual care HR 1.11, 95% confidence interval (CI) 0.96 to 1.28] (Figure 3 and Table 5). Results were similar for 180-day mortality (HR 1.12, 95% CI 0.98 to 1.28) (Figure 4). Patients in the QI group had a lower probability of hospital discharge (HR for hospital discharge 0.90, 95% CI 0.83 to 0.97), leading to a marginally longer hospital stay {days in hospital: usual care 13 [interquartile range (IQR) 8–23] days vs. QI 13 (IQR 8–24) days}, although this difference was not clinically meaningful (Figure 5 and Table 6). There was no difference between groups in hospital readmission within 180 days [usual care, n = 1618 (20%) vs. QI, n = 1242 (18%), HR for readmission 0.87, 95% CI 0.73 to 1.04] (Figure 6 and Table 7). In a secondary analysis, we found no evidence that the QI strategy became more effective the longer it had been adopted (Table 8). To assess the impact of missing mortality data following hospital discharge from patients in Wales, we assessed the number of mortality events that occurred after hospital discharge but before 90 days in English and Scottish hospitals. Only 5% (631/13,034) of patients died between hospital discharge and 90 days, suggesting that few outcome events in Wales were missed. Analysis of the effect of the intervention over time is presented in Table 8 and of the inclusion of younger patients and those undergoing laparoscopic surgery in Table 9.

FIGURE 3.

Mortality within 90 days of emergency abdominal surgery.

FIGURE 4.

Mortality within 180 days.

FIGURE 5.

Duration of hospital stay after emergency abdominal surgery.

FIGURE 6.

Time to hospital readmission.

Data sources and data collection

Table 10 details the evaluation foci and the data sources used to investigate each. Following commencement of the trial, the variability of engagement with the QI programme prompted a wider-scale, post hoc process evaluation, with the aim of capturing data across the trial cohort. For the post hoc component of the process evaluation, we collected a range of QI programme activity data (see Table 10) and sent an exit questionnaire to all QI leads. The 37-item, online questionnaire, administered at the end of the trial, was designed to allow description of activities undertaken, as well as their overall experience of leading the improvement projects. The questionnaire comprised categorical, yes/no and free-text questions, with opportunities to elaborate on any answers as free text. The questionnaire was piloted multiple times in line with best practice. Only one response was required per hospital, but QI leads were asked to complete the questionnaire with colleagues. QI programme data were collected by the programme co-ordinator (TS) and included data on participation in programme activities, such as meetings and use of the trial VLE.

Data analysis

The programme activity and questionnaire data were analysed and reported using descriptive statistics [frequency (%) for categorical data or median (range) for continuous data]. Free-text data in the exit questionnaire were coded by two investigators, using both inductive and deductive content analysis techniques. Findings were discussed until themes were agreed and draft results were discussed with the ethnographic researchers to enhance external validity. 56

Recruitment of clusters

Hospitals were recruited following an open call and promotion through existing critical care and perioperative medicine research networks. All NHS hospitals in the UK (except Northern Ireland) were eligible to take part if emergency general surgery was performed on site and if there was no previous or ongoing improvement work focused on emergency laparotomy.

We have documented records of 14 hospitals expressing interest, but subsequently not participating due to existing improvement work in that site. For other hospitals that expressed an interest but subsequently did not join the trial, the most common reason given was that there was insufficient support from colleagues for the trial interventions.

Delivery of the intervention at the cluster level

A total of 15 face-to-face QI educational meetings, planned to coincide with cluster activation and 15 follow-up meetings (one for each geographical cluster) were held as part of the QI programme. Figure 7 summarises the EPOCH trial QI programme ‘as planned’ and ‘as delivered’; the major change to the plan was the addition of follow-up cluster meetings at 12–16 weeks post activation to the intervention. Aside from local QI leads (surgeons, anaesthetists and critical care physicians), research nurses, theatre nurses and trainees in surgery and anaesthesia were the most common groups to participate in the educational meetings. The number of participants from each hospital at the follow-up cluster meeting was substantially fewer than at the first meeting. Figure 8 displays the numbers of QI leads attending the meetings from each hospital. The median number of participants (both QI leads and other invited colleagues) at the educational meetings and follow-up meetings were three per hospital (range 0–19) and one per hospital (range 0–8), respectively. The web-based resources were housed in a VLE, which contained a total of 66 pages or resources, to be viewed online or downloaded, at the commencement of the programme, increasing to 84 pages or resources by the end of the trial. The site could be accessed only by registered EPOCH trial local QI team members. In total, 16,120 ‘hits’ (visits to the site, page view and resource views or downloads) were logged over the course of the trial period. The median number of VLE hits per hospital was 136 (minimum 11, maximum 519, interquartile range 70–194). The number of users per hospital ranged from one to seven, with a median of three users, but as some site teams were small, and as online resources could be downloaded and/or printed by one user for colleagues to view, we feel the total hits per hospital is a more useful metric of VLE usage. Given the number of pages/resources available (84 by the trial end), these data suggest probable appropriate usage by much of the cohort, but with some variability and a significant minority of low users.

FIGURE 7.

The QI programme as planned and as delivered. Reproduced from Stephens et al. 22 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. The figure includes minor additions and formatting changes to the original figure.

Reproduced from Stephens et al. 22 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. The figure includes minor additions and formatting changes to the original figure.

FIGURE 8.

Quality improvement lead attendance at QI meetings.

Response to the intervention at cluster level

Themes derived from responses to a free-text question in the exit questionnaire about the improvement programme are described in Table 11. Themes emerged pointing to the utility of the meetings, both for learning and for networking, the overall support offered and energy for change generated by the programme team, and the helpfulness of the run chart tool. Conversely, themes emerged from comments regarding the need for greater clarity about, and fewer components in, the intervention, and more meetings and input together with more time in the intervention period.

Quality improvement leads were also asked to rate the support they received from the QI programme team on a 5-point scale (very good to very poor). Of the 75 who responded to this question, 36 out of 75 (48%) QI leads rated support as very good, 30 out of 75 (40%) QI leads rated support as good and 9 out of 75 (12%) QI leads rated support as average.

Delivery to individuals at local site (quality improvement intervention fidelity)

The clinical intervention was a 37-component care pathway (see Box 1). Questionnaire data showed that, regarding the care pathway, only 11 care processes were the focus of improvement efforts in > 50% of responding hospitals; the remaining pathway components had more variable uptake (see Segmenting the pathway and Figure 9). The QI intervention comprised six strategies (see Figure 1 and Table 1). Questionnaire data showed that 10 out of 77 (13%) QI leads responding said that all six strategies had been used, 23 out of 77 (30%) QI leads indicated five had been used, 21 out of 77 (27%) QI leads indicated four had been used, 8 out of 77 (10%) QI leads had used three strategies, 10 out of 77 (13%) QI leads had used two and 5 out of 77 (6%) QI leads had used just one. No QI lead reported zero QI strategy usage. Table 12 shows the reported usage of each QI strategy and each is discussed briefly below.

Use of plan–do–study–act

At activation meetings, the use of PDSA cycles was presented to participating teams explicitly as a model for experimentation and the planning of change, with a clear set of instructions and supporting tools for putting it into practice. The data in Table 12 indicate that this approach was used, but perhaps not in the regular, methodical manner recommended.

Team approach

At the activation meetings, QI leads were strongly advised to recruit a formal team of ‘willing’ interprofessional colleagues to work with them on the local improvement activities. The data in Table 12 indicates that just under two-thirds of sites had a formal team to work on this major project.

Use of data feedback and run charts

At the activation meetings, using NELA data as a driver for engaging colleagues and monitoring improvement was promoted and tools provided to do so. The data in Table 12 shows that most, but not all, teams were analysing NELA data; far fewer were doing this on a regular (monthly/bi-monthly) basis. Many sites reported challenges in simply collecting the data, with only half of all questionnaire respondents indicating that systems had been set up to collect the audit data prospectively.

Engagement

At 5 weeks before activation to the intervention, sites were contacted and recommended to start planning a stakeholder meeting, to coincide with activation. Just over half of the respondents indicated that they had held such a meeting (see Table 12); when a meeting was held, most reported that it was successful (questionnaire data, not shown). The exit questionnaire also asked about senior support during the trial. Of the 71 who responded to this question, 15 (21%) described active executive board support for the QI work related to the EPOCH trial (e.g. funding staff time to support the project or making the project a board-level quality and safety priority).

Segmenting the pathway

At the activation meeting, QI leads were advised to consider segmenting the proposed pathway to make the workload of implementation more manageable. Advice was offered regarding selecting which elements of the pathway to work on first and how to plan a step-wise implementation of the pathway that would be workable in QI leads’ local context. The data in Table 12 suggests that the majority of sites appear to have followed this approach. However, the data in Figure 9 also suggests that most sites did not progress beyond this to attempt to introduce most or all of the pathway components.

FIGURE 9.

Clinical process change attempted during intervention period. CT, computerised tomograghy; VTE, venous thromboembolism; WHO, World Health Organization. Reproduced from Stephens et al. 22 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. The figure includes minor additions and formatting changes to the original figure.

Reproduced from Stephens et al. 22 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. The figure includes minor additions and formatting changes to the original figure.

Response of quality improvement leads: reflections on the change process

Quality improvement leads reflected on ‘what would you continue doing?’ and ‘what would you do differently if you were to do the EPOCH trial again?’. Ninety-six per cent (74/77) of respondents left a total of 299 comments. Eighteen themes were generated for each question (36 in total) and these were further grouped into nine high-level themes (Table 13). Two clear themes, from responses to both questions, were related to the importance of effective engagement and involvement of colleagues (themes 2 and 6), and to data collection and feedback systems (themes 1 and 7). Other reflections on what QI leads ‘would continue doing’, related to QI methodology (themes 3 and 5) and the utility of specific, recommended clinical interventions, particularly mortality risk estimate scoring (theme 4). When considering ‘what they would do differently’, QI leads also highlighted some of the ‘real-life’ challenges of delivering QI at the frontline, developing leadership and project management skills (theme 9) and the need to obtain strong senior support (theme 8).

The EPOCH trial approach as received and understood

Activation meetings were the starting point for sites’ improvement journeys in the trial. Activation was a central part of the EPOCH trial’s approach to seeking to improve care. Activation of sites in a regional cluster was akin to the initial administration of a treatment to perioperative care practice within a region, enabling the trial to distinguish between before- and after-treatment periods for measurement purposes. In other words, it marked the formal commencement of the intervention in each cluster. In terms of QI, the EPOCH trial approach was to use the activation meetings for multiple purposes. They were meant to generate energy and build momentum among site QI leads, and to provide them with tools and strategies to take back and use in implementation. Some components of the activation meetings had more than one role. Opening presentations by the EPOCH trial team, for example, used data to generate buy-in among the audience. They showed the magnitude of the problem and crafted a pretext for proposing the EPOCH trial as a legitimate and necessary solution. At the same time, the presentations showed participants how to use data themselves, and enabled local QI leads to replicate these methods later in their own sites.

The EPOCH trial approach provided participants with a range of tools and QI ‘knowledge bites’ to take away. Key messages revolved around introducing local leads to key concepts in QI. Among other things, leads were prompted to realise that the care provided by their institutions was not perfect and that doing the right thing was about working differently in the way front-line care was provided, rather than necessarily requiring more resources. Participants were also told that improving care involved combining evidence-based clinical practice with thinking about ‘softer skills’ invested into persuading colleagues, understanding variation and building up knowledge about how to do things differently. Data and measurement were said to occupy a central role in persuasion; and persuasion, engaging others and the so-called ‘soft periphery’ of activities around the clinical and process interventions proposed were to be placed at the heart of making change happen. Implementation framed as a social problem was at the heart of the theory of change. Therefore, the key message was that implementation was a social as much as a technical endeavour, and that social relationships and contexts were a precondition to successful implementation. Participants learned that generating a collective commitment among colleagues would enhance change, building commitment to the shared objectives was crucial in maintaining momentum and segmenting the care pathway into smaller, more achievable improvement projects delivered through rapid cycles of testing was likely to be more effective than lengthy negotiating of top-down implementation:

Just get on and do it, have these standing meetings, if people aren’t available don’t worry about them, if people are resisting forget about them, go with the grain and make it easy for yourselves. So I think that was the rationale underlying this whole thing, that basically OK yeah, we could spend a lot of time training them in the detail of QI methods, but that wouldn’t be time well spent, and ultimately this project doesn’t hinge on getting PDSA cycles and measurement just perfect, it hinges on enthusiasm and sustaining that enthusiasm.

Fieldwork debrief, cluster 2 activation meeting

The EPOCH trial approach was intended to be practical and relevant. To this end, the EPOCH trial team prompted local QI leads to start planning their first steps based on baseline overviews they had prepared before the meetings. To maintain the energy created during activation meetings, the EPOCH trial team asked participants what they were going to do ‘by next Tuesday’. Importantly, the EPOCH trial team also provided participants with a set of tools. Most notable among these, participants reflected, were run chart templates developed by the EPOCH trial team for participants to populate with a selection of their local NELA data. The run charts allowed sites to monitor nine of the EPOCH trial’s process measures and in-hospital mortality.

Participants perceived the EPOCH trial cluster activation meetings, as well as the 12-week follow-up meetings, positively. Unequivocally, they felt that the EPOCH trial team demonstrated the relevance of the project. They felt energised by them and expressed satisfaction about the useful tools and tips provided. They also reflected positively on the practical nature of the meetings and the opportunity to share ideas and learn from others:

It was quite useful actually because it was more sharing of ideas and lots of looking at each other’s data and sharing ideas basically. It seems to be the way it has worked so far. Someone comes up with a good idea and everyone else shares or steals it [laughs]. The things we have used here have been stolen from other people.

Anaesthetic senior house officer, site 5

The impact of activation meetings was demonstrated in a number of ways, evidenced by our data:

• Leads in all sites adopted the EPOCH trial narrative of the nature of the problem and the solution.

• Among the tools, participants reflected particularly positively on run charts provided by the EPOCH trial team to support data monitoring and visualisation in local engagement exercises.

• Leads professed a shared commitment to change and agreed to propagate this among their colleagues.

• Sites started to use tools provided to them by the EPOCH trial team. They mainly adopted the use of data as a tool for building commitment among colleagues and a way of bringing them on board of the improvement project.

• Leads started discussing and planning change that was within their direct reach and easier to control.

• Leads shared resources on Moodle [release 2.6, URL: https://moodle.org (accessed 17 July 2019], such as the checklist-based boarding card.

However, our observations and interviews showed that most of the local leads did not come to activation meetings as tabulae rasae, only to become enthused and educated. They were mostly senior clinicians who often had spent years as local champions, and sometimes lone enthusiasts, trying to improve perioperative care. Their buy-in was often already high and many had already achieved local improvements relevant to the EPOCH trial’s mission, long before the activation meetings. Nonetheless, even in those cases, the activation meeting was an important place for learning and sharing experiences. It was important for local enthusiasts to see that they ‘were not alone’ in struggling to improve perioperative care and learn how other sites managed to change aspects of care (e.g. to break into elective lists):

I’ve always felt that the emergency laparotomy patients have been treated as a bit of an underclass, you know, we could be doing things a bit better for them, there’s never been the interest or the resources thrown at it, so when NELA turned up I thought that’s a fantastic idea and was keen to get involved. [ . . . ] And then EPOCH [ . . . ], it was just really looking at the NELA website and the EPOCH study, and [colleague] said ‘oh yes, we enrolled onto this study as well’.

Anaesthetic lead, site 2

There was quite an interest from some of the other participants, in how is it that they achieved this change, how, how did they reorganise it. And then he went on explaining how they simply shifted the . . . well of course the mind, but also the structure, the organisational structure of operating, using theatres, and they are willing to send elective cases home, I think, that was the bottom line of it.

Fieldwork debrief, cluster 4 activation meeting

The EPOCH trial approach as enacted

On the face of it, the six sites followed a similar path in how they went about putting the EPOCH trial into practice. Site leads undertook education of their peers, focusing on showing colleagues how well or poorly they performed in terms of outcomes, record-keeping, compliance with evidence-based processes, introducing what they wanted to do [typically using the Portsmouth Physiological and Operative Severity Score for the enumeration of Mortality and morbidity (P-POSSUM) mortality risk assessment tool] and making the tools available to fellow clinicians. They tweaked existing systems (e.g. intervening to make the P-POSSUM part of existing theatre booking systems) and designed and introduced new tools (e.g. boarding cards that had to be completed prior to surgery) to support, monitor and enforce the pathway elements. They communicated within teams, monitored their plans, looked at NELA-based run charts and tried to devise alternatives when their plans were not materialising as planned. All of these activities had a strong basis in the activation meetings, in which options were discussed and interventions that had been successful elsewhere were advocated. For example, the ‘boarding card’ approach was usually presented at the activation meetings, and both this intervention and interventions to enforce P-POSSUM completion as part of theatre booking processes were highlighted as good practice in the first report of NELA. 2

In terms of improvement planning, all sites used a combination of NELA data, five case reviews and older local audit exercises. Invariably across sites, they found that they were ‘doing things right’ in some aspects of the pathway and were not so strong in others. Indeed, this variation in clinical practice was exactly what was targeted by the EPOCH trial:

Some parts are quite good. There are a lot of parts that are very inconsistent and unfortunately a lot of those are very basic though in theory should be easy to change.

Consultant anaesthetist, site 6

Use of plan–do–study–act

As noted above, at activation meetings, the use of PDSA cycles was presented to participating teams explicitly as a model for experimentation and the planning of change. In addition to practical activities designed to demonstrate the usefulness of the approach – and its preferability to developing a blueprint for change without testing it in practice – the activation meetings provided references and reading materials that would help each team to follow the PDSA approach in practice. In this sense, PDSA was presented to QI leads in a ‘textbook’ format, with a clear set of instructions and supporting tools for putting it into practice.

No site, however, applied the formal PDSA methodology by the book. Instead, sites adopted a less formal planning approach, which included the general tenets trying out, reviewing and improving small changes, but which was otherwise much looser (e.g. typically excluding the setting of numerical goals against which to measure progress). This less formal planning was seen by site leads as more practical and intuitive. It followed a basic planning cycle of assessing the situation, planning what to change, preparing the approach to change, doing an intervention and evaluating their actions.

This more informal, less tight approach appeared, in part, to be a function of the fact that many parameters were predetermined by the EPOCH trial format: data collection methodology, process measures, as well as an idea about the kinds of changes needed. Site leads knew what shifts in indicators meant improvement and what improvements they wanted to achieve (higher compliance with process measures). This meant that less deliberation was needed about what kinds of changes to make; the focus was to a large extent determined by the trial. The leads did, in general, continue to use the language of ‘cycles’ of improvement, but these cycles did not follow the (quite specific and exacting) PDSA blueprint. Rather, typically, the starting point was a hunch or intuition around what the initial focus should be, informed by discussions in the activation meetings, rather than something systematically thought through and planned; the fact that they did not adhere fully to the formal PDSA cycle should not be taken to mean that sites failed to engage in creative experimentation. Local deliberation focused on which elements of the pathway to choose to implement, which tools to use to support implementation, how to adapt them or devise new ones and whether or not to change strategy when the tools, even after adaptation, were found not to be working. When boarding passes and other interventions were used as tools of improvement, they were thus iteratively modified, albeit not in the tight, systematic way prescribed by the PDSA methodology:

When we first started, we thought right, I know [person 7] and [person 5] are really keen than rather than try to introduce all the interventions together that we would do much smaller cycles which is why we started with [the boarding pass]. And we felt that the compliance was so poor because there were almost too many bits of paper, they were picking up a NELA document then they had to remember to get the end-of-care sticker and then they had to remember to get the boarding pass, and we just were going, ‘It is too many things’, and people were getting confused what they were doing. So we said well let’s get rid of those two and incorporate it into one, stick it on the front and then they just pick up that bundle of paperwork and it is all there for them.

Consultant anaesthetist, site 2

This more intuitive approach to the cycle of improvement continued through the process.

Team approach

Developing a team approach was suggested by the national EPOCH trial team and all sites committed to it by formally nominating representatives of the three specialisms recognised as key in the EPOCH trial: (1) surgery, (2) anaesthesia and (3) critical care. However, the degree of continued involvement from the three specialisms varied significantly and some sites reported difficulties in securing ongoing engagement from surgery in particular. This typically meant that, in practice, project development was carried out by anaesthetists and/or critical care specialists.

When active surgical involvement in the core EPOCH trial team was lacking, participants saw this as a disadvantage for engaging surgical colleagues in improving the pathway (see Engagement). This appeared to be a particularly important barrier in site 3.

In contrast, three of the sites (sites 2, 5 and 6) secured ongoing, regular engagement from surgeons as part of the core EPOCH trial teams and they reported at least some success in interesting practitioners in the surgical department.

The experiences of site 1 are perhaps particularly instructive in demonstrating the value of multidisciplinary involvement in the core local team and, particularly, the importance of strategically positioned actors. Here, activity relating to the EPOCH trial really took off only after the appointment of a clinical surgical lead in emergency surgery who took the view that NELA and the EPOCH trial were integral parts of improving the field. That single, but interested and strategically placed, actor dramatically improved access to others, which had not been achieved before by a handful of less strategically positioned EPOCH trial team members.

Use of data

All six sites tried hard to collect and use data in their improvement efforts. All used run charts based on NELA data to monitor care process and to attempt to mobilise support from colleagues. However, sites often struggled with using sampling batches of five sequential case notes suggested to monitor changes in compliance with key processes through time. Most used this technique, which had been recommended by the EPOCH trial team, to establish their baseline prior to the activation meetings. However, no site maintained this method of monitoring 6 months after activation.

Some sites (sites 4 and 6) were able to build on existing audit work that predated NELA and the EPOCH trial. The experience that these sites had in collecting and making use of data for improvement seemed advantageous, in that leads were aware of the power of such data and how to present them, and their colleagues were more immediately accepting of the value of data for identifying areas of weakness and targeting improvement efforts. However, this more mature ‘data culture’ also meant that areas for improvement apparent in the data had already been identified and sometimes addressed, and so the data were not so helpful in identifying ‘low-hanging fruit’ and swift potential wins. In site 6, for example, a series of local audits that had commenced several years before the EPOCH trial had already led to concerted efforts to improve care, and these had continued with active efforts to make use of learning from NELA, in collaboration with local commissioners. This meant that several key changes [to preoperative risk stratification, theatre capacity and intensive care unit (ICU) admission] had already been made, potentially reducing ‘head room’ for further improvements through involvement in the EPOCH trial:

We had looked at our data, on the basis of NELA we had looked at our data and it continued to be poor. [ . . . ] So a commissioner-led, unilateral decision was made to admit all patients with emergency laparotomies into ITU [intensive therapy unit]. So we admit about 92% of our patients now who go directly to ITU. The default is ITU and if there is a problem or someone is deemed to not be suitable for ITU that is the only reason they don’t go there.

Engagement

Some sites had better experience of effectively engaging with colleagues and relevant departments than other sites. Notable in particular was the range of stakeholders engaged across the six sites, going far beyond doctors and nurses in the core three specialties of surgery, anaesthesia and critical care. Local teams drew on the local connections they enjoyed to their advantage, pulling in contacts in management, other disciplines such as radiology, and clinicians and administrators with particular responsibilities relevant to the pathway, for example sepsis identification and treatment:

What professional groups are involved, so the three main medical specialisms?

Yeah so in terms of, well obviously anaesthetics, critical care and surgery. Medicine are a group that need to be involved but up until now they haven’t really had much involvement and when I e-mailed [colleague] her reply was ‘I thought this was more anaesthetic, critical care, surgery type of thing’. So when I then wrote back and said about the care of the elderly role I think she probably thought ‘oh OK that’s quite a major thing’. So at the stakeholder meeting tonight I’m hoping that [colleagues] have had a talk about it already. Other professional groups, radiology in terms of the CT [computerised tomography] requests. We already have good back-up already in terms of 24-hour access to microbiology and CT reporting. And then we’ve got the allied care, so we’ve got the physio[therapist]s, the dietitians. Dietitian is a bit, we do have, there’s not enough dietitians and they have got business cases already to try to get more dietitians, not on the back of this but on the request that we’ve got for critical care,’ cause at the minute they come in maybe there times a week and we’re trying to get them on the ward rounds every day. So they’ve got a business case there.

Anyone from the board?

In terms of the board we’ve got yeah, we’ve got the business unit manager. I’ve tried to get our safety lead but he’s away today, he can’t come. We’ve got one of the execs [executives] and then we’ve got – who else is going to come? Head of theatres. Yeah so obviously there’s the whole theatre team as well, so there’s a lot of teams. Yeah.

The ability to engage colleagues successfully and encourage active involvement in improvement efforts, depended to a large extent on existing relationships and, more broadly, on the local organisational culture and its receptiveness to challenge and change. Similar efforts in sites to use, for example, existing data on process compliance and patient outcomes to convince colleagues of the need to change could result in very different degrees of engagement, with scepticism and hostility resulting in some sites, whereas in others, they brought new recruits to the cause. A strong network of existing relationships among supportive colleagues seemed crucial in making the difference between efforts that floundered and efforts that resulted in tangible change. The nature of engagement that was viable also depended on the attitudes towards local performance and improvement efforts that prevailed prior to the activation of the EPOCH trial: there were notable differences in the degree to which local stakeholders had examined their own performance and intervened to improve in the years prior to the EPOCH trial, which translated into differential enthusiasm for the EPOCH team itself, when it was introduced.

Site 1

Site 1 was a large urban hospital. At activation, the local team were rather pessimistic about the prospects for instigating change in response to the EPOCH trial, mainly due to ongoing organisational changes in the trust as a whole and in surgery in particular. Through time, they fostered closer links with other key stakeholders, notably the trust’s research and development (R&D) team, which provided assistance with data monitoring. Towards the end of the trial, a recently appointed QI lead was hoping to achieve some momentum.

In terms of the EPOCH team pathway itself, site 1 focused largely on postoperative care. The intention was that emergency laparotomy patients be admitted to critical care by default. When critical care lacked the capacity to take the patients and they needed to be cared for on a ward instead, an ICU outreach team would provide their support. This was, indeed, mandated in the first months. They also planned to focus on improving recognition of emergency laparotomy patients, particularly those who risked ‘disappearing’ in the course of prolonged stays on medical wards, to ‘make this group more overt to the whole hospital and make the whole hospital aware of what happens to these patients if a patient does badly’ (consultant anaesthetist).

Participants in site 1 saw the size of their organisation as a key challenge, making it particularly complicated to achieve high consultant presence, speedy access to radiology and getting the patient into theatre within the target of 6 hours. Nevertheless, by the end of the trial, the local team felt that there had been focus on all three of these fronts. However, they played down the role of their own efforts in this, seeing them instead as byproducts of the wider reorganisation, along with a local, longer-established focus on improving care for trauma patients:

Getting to theatre is much better than it used to be because of the peer review stipulations for trauma and the need to simply crash into theatre. So we have Code Red cases quite often, and as a consequence when emergency laparotomies come through, we treat them in the same way. [ . . . ] I think they’re seen as having the same degree of urgency as someone would have coming through with major trauma.

Consultant surgeon

We are doing all the work on that at the moment. I think it probably won’t be fully operational by the time EPOCH finishes. It will be there but it won’t have had any chance to impact.

Intensive care consultant

Site 2

In site 2, the team began the trial with a plan to implement a selection of pathway items using the boarding card and the end-of-surgery sticker as implementation tools:

We’re not going to try to do all 37. [ . . . ] At the meeting we thought that some of the end-of-surgery bundle we could implement fairly easily and [name 1] felt that we could do the P-POSSUM score very easily at the start [ . . . ] and then I get them to do cardiac output monitoring and the admission to critical care. Yeah they’re probably the first ones we’ll start with. They seem the easier ones to do, I feel like we’ve got more control over them compared to trying to get a computed tomography [CT] within 2 hours and care of the elderly review there. They seem like a bigger challenge at the minute.

Consultant anaesthetist

Soon, however, they extended their scope. Along with the bundle logic of improvement, site 2 decided to implement the ‘whole bloomin’ lot’, with the exception of the postoperative checklist:

[The] end point is reduced mortality and reduced morbidity for emergency laparotomy patients. My view would be, look, we really don’t know, just do the whole bloomin’ lot and then see what happens, very much in the way that the models were done for reducing central line sepsis in the States [where they] drastically reduced the amount of central line sepsis by just putting a whole sheet of measures in and say sod it, we don’t matter which one does it, let’s just do the whole lot.

Consultant surgeon

Another reason for changing the course was an attempt to integrate multiple tools into one:

. . . we kind of felt that the compliance was so poor because there were almost too many bits of paper, so they were picking up a NELA document then they had to remember to get the end-of-care sticker and then they had to remember to get the boarding pass, and we just were going it is too many things and people were getting confused what they were doing so we said well let’s get rid of those two and incorporate it into one, stick it on the front and then they just pick up that bundle of paperwork and it is all there for them.

Consultant anaesthetist

Their main implementation tool was, thus, a checklist that brought all the EPOCH trial bundles together. But, by the end of the trial, they were still discussing the need to ‘implement the checklist’ as progress had not been as rapid as they had hoped. Nevertheless, as with other sites, there was partial success in increasing compliance with the process, evidenced through the NELA-based run charts produced by the team.

Site 3

The team in site 3 was characterised from the start by a degree of pessimism about the prospect of much change in response to the EPOCH trial. As in site 1, the team was acutely conscious of ongoing, large-scale changes at the trust level, including financial challenges and efforts to rationalise theatre space:

Taking it from the top, in terms of speaking to senior management, this hospital has been discussing for a year how to rebuild and where to rebuild and made a decision last week, so that’s a year’s worth of people’s time taken up with that. Four months ago they decided to completely rejig the theatre schedules, and that has caused huge amounts of ructions. They want to change everyone’s job plans and contracts for Saturday working, [ . . . ] half of the department doesn’t know, half of us will not have too many changes, but we will have a new, they’re extending the working day, so that affects everybody who works in the afternoon, and then they’ve got to decide if you’re going to work extra lengths on some days what are you going to give up, and it has a knock-on effect on our on-call rotas and so many things. [ . . . ] Now, since everyone seems to think different things, worry about different, actually we don’t know exactly how our job planning is going to be for the next, starting [in winter], so somehow, at least for me, there’s no space, no room for new things.

Intensive care consultant

At the initial activation meeting, one team member even went as far as to suggest that site 3 might represent a ‘control site’, against which other sites, able to undertake more active work, might be compared. Nevertheless, some progress was evident in this site through time.

Site 4

Site 4 was a large teaching hospital, in which efforts to improve the emergency surgery pathway were already well under way. The site had developed a local version of the emergency laparotomy pathway that might be seen as a ‘cousin’ to the EPOCH trial pathway, rather than directly deriving from it. The pathway was developed at the same time as the EPOCH trial and both took the 2011 guidance as their reference point. Three years in design, site 4’s pathway was intended to be easy ‘for the end user to buy into’ (intensive care consultant). It was more concise than the full EPOCH trial pathway and the focus on a smaller number of key components was seen as advantageous. The pathway had been fully designed by the time of activation, together with a dummy version ready to be integrated into a new electronic patient record system that was shortly to be launched.

Site 5

As with other sites, site 5 had some experience of improving emergency laparotomy before the EPOCH trial. Local champions among surgeons and critical care specialists carried out an audit of care in 2011 and initiated changes to care consistent with the pathway recommendations, namely consultant presence, access to radiology and access to intensive care. The mortality figure at activation was similar to the national average and the team saw joining the EPOCH trial as an opportunity to further local improvement. Formal risk scoring and other pathway elements clustered in the Surgical Survival Six were explored as a good starting point, with the view of implementing the whole pathway. The team also appropriated tools suggested by the national team, such as the ‘boarding card’ and NELA-based run charts. They translated the end-of-surgery bundle into a formal ‘landing card’:

The ideal that we are aiming for would be to have all of the 37 points done consistently for everybody and, although the way that I think we have approached it is to cater for the ones that are perhaps easier to understand and implement, I think in my mind I was hoping that what we do is start with the easy ones, that are more obvious to people who haven’t looked into it and easier to implement and then, on the back of those, introduce the rest of them.

Anaesthetic senior house officer

According to the local lead, in this site over the course of the trial, the team achieved most of what it had set out to do. The list of achievements referred to areas that had been targeted before the trial, as well as well as through it. In the post-EPOCH trial period, the ongoing plan was to focus on implementing the postoperative bundle, which had not been achieved during the trial period:

Our major achievements were HDU [high-dependency unit] beds; we’re getting most of our patients to HDU. Consultant review, CT [computerised tomography] scanning, [ . . . ] an improvement in risk assessment. Our challenges were, the major challenges were cardiac output monitoring, whether or not we believe that is something that we need to work on more is still out for debate, although we probably feel we should be better. [ . . . ] The other challenge we had was the postoperative review by care of the elderly physician [ . . . ] well vast majority of trusts aren’t doing it. I think in 80, 90 per cent of places, most hospitals, over 70s aren’t being seen by care of the elderly physicians. So that was the other major challenge, and [person 1] is working on that.

Intensive care consultant

Site 6

Efforts to improve emergency laparotomy care in site 6 had started several years before the EPOCH trial. In 2011 and 2012 a group from the hospital assessed their service against the then-published standards and initiated changes after reporting to the hospital management:

. . . at least from 2011 to now, there have been several clinicians who are at the front end of delivering this service, highlighting the issues and the concerns about whether or not we hit the targets set by the college and that have been accepted by the Department of Health so they are essentially national guidelines.

Consultant surgeon

The main achievement stemming from these early initiatives was an arrangement to admit all emergency laparotomy patients to an intensive therapy unit. At activation, 9 in 10 emergency laparotomy patients were admitted to ICU, compared with 5 in 10 in 2012. Staff in site 6 also had worked on improving access to screening and access to theatres. This led to halving mortality in emergency laparotomies. In contrast to most other sites, where staff responded to an invitation from the EPOCH trial to participate, the team in site 6 proactively contacted the national EPOCH trial team to request to be included:

And then EPOCH was happening so I . . . we hadn’t actually heard about it and I am not quite sure how. I mean, we nearly missed the boat so I ended up contacting [national EPOCH lead] directly and just saying ‘can we join in?’.

Intensive care consultant

When comparing itself to the national standards at activation, the site was not surprised to be doing well in ICU admissions, where much of its prior efforts had focused. Participants expressed more surprise at their levels of goal-directed fluid therapy, which also stood at 90% compliance. Overall, though, process compliance at activation in the EPOCH trial was similar to the other sites: in some areas their compliance was consistently high, in other areas they were ‘not doing so good’. The local team in site 6 planned to approach the issue of high-risk patients differently to the EPOCH trial. Rather than relying on P-POSSUM scoring immediately before and after surgery, they planned to introduce a risk assessment earlier in the patient journey, which they hoped would speed up the process of patient identification, risk stratification and time to theatre.

Overview

The analysis was undertaken from a UK health service perspective and costs are expressed in Great British pounds at 2016/17 prices. Health outcomes were estimated in terms of quality-adjusted life-years (QALYs). As the intervention was expected to have long-term effects on quality and duration of life, its cost-effectiveness was assessed both within the trial period and over the lifetime of patients.

Data sources

To assess cost-effectiveness within the trial period, survival outcome, health-care resource use and HRQoL were estimated using individual patient data on a subsample of patients from the trial. In eligible patients in 8 of 93 participating hospitals, data on hospital readmissions after surgery, outpatient appointments and primary care health-care resource use were collected using a questionnaire at the end of the trial period (i.e. 180 days after surgery). HRQoL data were collected using the EuroQol-5 Dimensions, three-level version (EQ-5D-3L) before surgery (baseline) and at 90 days and 180 days after surgery. Assumptions were employed to extrapolate costs and health outcomes beyond the 180-day period of the trial. Patient-level data were collected and collated by the NELA in all participant hospitals from the start of the trial. These data were then linked to the ONS, HES and the Information Services Division of NHS Scotland databases, using patient identifiers to allow collation of outcome data, including mortality and hospital readmissions.

Resource use and costs

Resource use associated with inpatient stay after surgery, during the trial period, were extracted from the NELA database and then combined with NHS reference unit costs to calculate the costs of inpatient stay after surgery. 77 Costs of readmissions and costs of outpatient and primary care were calculated using resource use reported in the questionnaire and the NHS Reference Costs 2015 to 201677 and Personal Social Services Research Unit Unit Costs of Health and Social Care 2016. 78 Unit costs are presented in Table 16. Costs of the QI intervention were estimated by the clinical trial team, by considering the resource use associated with the QI intervention, and is assumed to be the same across all hospitals.

Health-related quality of life

The EQ-5D-3L responses were transformed into HRQoL weights using the UK tariff,79 on a scale in which 0 represents death and 1 represents full health. The HRQoL weights between measurement points were multiplied by the time between those points, along with mortality data from the ONS, to estimate total QALYs gained over the trial period, using the area under the curve method and linear interpolation between time points,80 and then adjusted for baseline EQ-5D-3L, to calculate the incremental QALYs associated with the QI intervention.

Missing data

Multiple imputation was performed to replace each missing observation with a set of plausible imputed values, including EQ-5D-3L scores and three cost variables, following the method recommended by Faria et al. 81 The imputation was implemented separately by randomised treatment allocation. Prediction mean matching was used to ensure imputed values were in the appropriate range (e.g. no negative costs or EQ-5D-3L scores of > 1). We used multiple imputation by chained equations in Stata^® (StataCorp LP, College Station, TX, USA), using the command ‘mi impute chained’, and Rubin’s rules were implemented using the command ‘mi estimate’, also in Stata. The multiply imputed data sets were then used in cost-effectiveness analysis.

Analysis

For the within-trial analysis, costs and QALYs were calculated per patient and then analysed with the seemingly unrelated regression model to estimate the incremental mean costs and QALYs. 82 If the QI intervention was associated with QALY gain and increased costs, the incremental cost-effectiveness ratio (ICER) was generated and then compared against the threshold values of the National Institute for Health and Care Excellence, which is ranged from £20,000 to £30,000 per QALY gained. 83 The threshold of £13,000 per QALY, representing the marginal productivity of the NHS estimated by Claxton et al. ,84 was also used here to consider whether or not the QI intervention represents a cost-effective use of resources. The probability of the QI intervention being cost-effective at different thresholds of willingness to pay was calculated85,86 and represented visually on a cost-effectiveness acceptability curve. 87,88

Cost-effectiveness over the lifetime horizon was estimated by incorporating assumptions of HRQoL after the trial period, general population survival data from the ONS and health-care costs from the literature. HRQoL of patients alive at the end of the trial period were assumed to return from their EQ-5D-3L at day 180 to the population average for someone of their age and sex over the next 3 years, and subsequently have the population age- and sex-adjusted EQ-5D-3L scores until death. 89 QALYs over the 3 years after the trial were estimated based on linear interpolation, which is standard practice. 90 Patients whose life expectancy was < 3 years were assumed to have the HRQoL at day 180 until death. Life expectancy was estimated to match the population average based on their age and sex, reported in the ONS National Life Tables, United Kingdom: 2012–14. 91 We assumed that patients alive were attributed average health-care costs by age and sex over the rest of their modelled lives. 92 The expected lifetime costs and QALYs were discounted using 3.5% per annum in line with UK guidelines. 83 Cost-effectiveness acceptability curves were also presented to reflect decision uncertainty.

Scenario analyses were conducted to explore the cost-effectiveness of the intervention in the whole trial population. As data on inpatient stay after surgery were available for the EPOCH trial population, costs of inpatient stay were calculated using the data directly from the trial, and costs of readmissions and costs of outpatient and primary care were estimated using generalised linear model (GLM) regressions based on the subsample. Baseline HRQoL and HRQoL change from randomisation were estimated using regression models developed based on the subsample and then combined with the mortality data for the EPOCH trial population, to calculate QALYs over the trial period using area under curve and linear interpolation method. A GLM with gamma distribution and log-link was employed to estimate the HRQoL decrement due to the emergency laparotomy before surgery, accounting for relevant covariates and the distribution of the data. 93 A panel data approach was then employed to estimate changes in HRQoL after randomisation, differentiating between intervention and usual care. 94 The assumptions of long-term effects used in primary analysis were also applied to estimate costs and QALYs over the lifetime horizon.

Sensitivity analyses were conducted to assess the impact of different assumptions of HRQoL beyond trial period on the trial results:

• Patients alive at the end of trial followed the group mean EQ-5D-3L scores until death.

• Patients alive at the end of trial had their individual HRQoL at day 180 until death.

Complete-case analyses were also conducted as part of sensitivity analyses, using data on patients with complete data on all variables at all follow-up points.

Subgroup analyses were performed to establish whether or not cost-effectiveness varies between high- and low-risk patients, defined by P-POSSUM score (risk for morbidity and mortality). All statistical analyses were performed using Stata 14.

Patient characteristics

The characteristics of patients in the subsample and the EPOCH trial population are summarised in Table 17. A total of 680 eligible patients were included in the subsample, with 265 patients in the QI intervention group and 415 patients in the usual care group. The EPOCH trial population included 15,856 eligible patients (8482 in the usual care group and 7374 in the QI group). Overall, there were no marked differences in age, sex, indication for surgery and P-POSSUM scores between the intervention and usual care groups in both the subsample and the EPOCH trial sample, but there were more older patients, females and patients at high risk of morbidity and mortality defined by P-POSSUM score in the full EPOCH trial population than in the subsample. In the subsample, patients in the QI group had higher baseline EQ-5D-3L scores than the usual care group (0.262 vs. 0.168).

Missing data

The missingness of data on health-care resource use and EQ-5D-3L collected in the subsample is summarised in Table 18. Missing data on inpatient care ranged from < 1% to 4.15% and there were more missing values in outpatient and primary care (23.4% in the usual care group and 26.4% in the intervention group). For EQ-5D-3L scores, there were more missing values at follow-ups than baseline (ranging from 19.0% to 30.6%).

Health-care resource use

The health-care resources used by this subsample over the trial period are summarised in Table 19.

Costs

Table 20 reports the mean costs of health-care resource use, costs of intervention and total costs per patient in the QI intervention and the usual care groups in the subsample. For the usual care group, the mean costs of inpatient stay after surgery, of readmissions and of outpatient and primary care were £6486, £934 and £797, respectively, and the corresponding costs for the QI group were £7001, £881 and £760. The cost of the QI intervention was estimated to be £32 per patient. The total cost for a patient in the usual care group was £8216, whereas the cost for a patient in QI group was higher, at £8675. During the trial period, the QI intervention was associated with £458 higher costs (95% CI –£812 to £1728). The cost of inpatient stay after surgery is the principal costs driver, accounting for around 80% of the mean total costs.

Health-related quality of life

The EQ-5D-3L scores and QALYs are summarised in Table 21. At baseline, the QI patients had higher EQ-5D-3L scores than those in the usual care group, but the follow-up EQ-5D-3L scores were similar between the two groups. The mean QALYs over the 180 days after surgery were 0.274 (95% CI 0.260 to 0.288) per patient in the usual care group and 0.287 (95% CI 0.269 to 0.306) per patient in the QI intervention group. After adjusting for baseline EQ-5D-3L scores, the difference in QALYs between the two groups was minimal, with fewer QALYs in the QI intervention group (mean –0.002, 95% CI –0.021 to 0.017).

Base-case analysis

The cost-effectiveness results within the trial period and over the lifetime horizon in the subsample are presented in Table 22. Within-trial analyses showed that the QI intervention was associated with higher total costs, but the result was not significant (mean difference £458, 95% CI –£812 to £1728). After adjusting for baseline EQ-5D-3L scores, the QI and usual care groups showed similar QALYs gained over the trial period (mean difference –0.002, 95% CI –0.021 to 0.017), although QALYs were fewer in the QI group. The probability of being cost-effective at £13,000, £20,000 and £30,000 per QALY threshold was 24.0%, 24.3% and 24.9%, respectively. The cost-effectiveness acceptability curve is shown in Figure 10. Therefore, the QI intervention was dominated by the current practice and it does not appear cost-effective in the subsample over the 180 days of the trial period.

FIGURE 10.

Cost-effectiveness acceptability curve for the QI intervention over (a) trial period; and (b) lifetime horizon.

In the lifetime perspective, the QI intervention was associated with higher costs (mean difference £1508, 95% CI –£1108 to £4124) and more QALYs (mean difference 0.131, 95% CI –0.640 to 0.903), which led to an ICER of £11,511 per QALY, lower than the three cost-effectiveness threshold values considered. The probabilities of being cost-effective at £13,000, £20,000 and £30,000 per QALY threshold were 51.8%, 56.3% and 58.7%, respectively (see Figure 10). Therefore, the QI intervention may be cost-effective for patients in the subsample over the longer term, but this is subject to substantial uncertainty.

Regression

Scenario analysis

Using the regression models, the estimated costs, EQ-5D-3L scores and QALYs for the EPOCH trial population are presented in Table 26. Within the trial period, the QI intervention was associated with higher costs, at £296, but fewer QALYs, at –0.013. Over the lifetime horizon, the QI intervention was also associated with higher costs, at £601 and more QALYs, at 0.019, resulting in an ICER of £31,632 per QALY, which is higher than the highest cost-effectiveness threshold value of £30,000 per QALY. Therefore, the intervention was probably not cost-effective for the EPOCH trial population.

Sensitivity analysis

Subgroup analysis

High-risk patients

According to P-POSSUM score, patients were categorised into high- and low-risk groups, using the median value. For the high-risk patients, the QI intervention was associated with increased costs and more QALYs within the trial period and the ICER was £119,400 per QALY, which is much higher than the highest cost-effectiveness threshold value of £30,000 per QALY, so it was unlikely to be cost-effective over the trial period. However, over the lifetime horizon, the QI intervention was associated with more QALYs at an increased cost, generating an ICER of £3839 per QALY. The probabilities of being cost-effective at the threshold of £13,000, £20,000 and £30,000 per QALY were around 99%, showing it as very likely to be cost-effective in the high-risk patients. The cost-effectiveness acceptability curves are shown in Figure 11. Sensitivity analyses were also conducted for this subgroup and results were similar, but when assumption 2 was used, there was substantial uncertainty around the decision. Therefore, the QI intervention is likely to be cost-effective for patients at high risk of morbidity and mortality before surgery (Table 30).

FIGURE 11.

Cost-effectiveness acceptability curve for the QI intervention in the high-risk group (a) within the trial period; and (b) over the lifetime horizon.

TABLE 30 - Cost-effectiveness analysis in the subgroups defined by P-POSSUM score: high-risk group

High-risk group	Usual care (N = 198)	QI (N = 140)
180 days’ costs (£), mean (95% CI)	9322 (8156 to 10,487)	10,515 (9076 to 11,954)
Incremental costs (£), mean (95% CI)		1194 (–627 to 3014)
180 days’ QALYs, mean (95% CI)	0.246 (0.224 to 0.267)	0.273 (0.246 to 0.300)
Adjusted incremental QALYs, mean (95% CI)		0.010 (–0.018 to 0.039)
ICER (£/QALY)		119,400
Probability (%) of cost-effectiveness at
£13,000/QALY		14.3
£20,000/QALY		17.1
£30,000/QALY		21.5
Lifetime costs (£), mean (95% CI)	44,458 (41,609 to 47,307)	49,487 (46,413 to 52,561)
Incremental costs (£), mean (95% CI)		5029 (811 to 9248)
Lifetime QALYs, mean (95% CI)	7.153 (6.509 to 7.796)	8.444 (7.676 to 9.211)
Adjusted incremental QALYs, mean (95% CI)		1.310 (0.317 to 2.303)
ICER (£/QALY)		3839
Probability (%) of cost-effectiveness at
£13,000/QALY		98.9
£20,000/QALY		99.3
£30,000/QALY		99.4
Assumption 1
Adjusted incremental QALYs, mean (95% CI)		0.916 (0.038 to 1.793)
ICER		5490
Probability (%) of cost-effectiveness at
£13,000/QALY		94.1
£20,000/QALY		96.3
£30,000/QALY		97.1
Assumption 2
Adjusted incremental QALYs, mean (95% CI)		0.396 (–0.919 to 1.712)
ICER (£/QALY)		12,699
Probability (%) of cost-effectiveness at
£13,000/QALY		50.6
£20,000/QALY		59.1
£30,000/QALY		63.9

East Anglia

East Midlands

Kent, Surrey and Sussex

Manchester, Merseyside and Yorkshire

North East London

North-east of England

North Lancashire and Cumbria

North West London

Scotland

South London

Wales

Thames Valley

Wessex

West Country

West Midlands