Care bundles to reduce re-admissions for patients with chronic obstructive pulmonary disease: a mixed-methods study

Epidemiology of chronic obstructive pulmonary disease

Chronic obstructive pulmonary disease is one of the most common respiratory diseases in the UK and it is estimated that the number of people with a diagnosis is 1.2 million, although around 2 million more may have undiagnosed COPD. Along with lung cancer and pneumonia, COPD is one of the three leading contributors to respiratory mortality in the UK; there are 30,000 deaths from the disease each year. 1

The majority of people with COPD also have other medical problems, most commonly ischaemic heart disease (which occurs in some 25% of patients). 2 Many people discharged from hospital after an acute exacerbation of chronic obstructive pulmonary disease (AECOPD) also report feelings of depression (64%) and anxiety (40%), with > 80% having at least one other condition, such as coronary heart disease. 2 This multimorbidity means that managing the health-care needs of people with COPD is challenging for patients, carers and health-care professionals. 3–5

Chronic obstruction pulmonary disease and emergency hospital admissions

Chronic obstructive pulmonary disease accounts for 10% of emergency hospital medical admissions, which total > 90,000 annually in the UK. 2 Nearly one-third of these patients are re-admitted to hospital within 28 days of discharge,6 and this proportion is steadily rising, with a 2% increase in re-admission rates between 2003 and 2008. 2 During the same time period, in-hospital mortality rates fell slowly – estimated at 7.5% in 2003, 7.7% in 2008 and 4.3% in 2015. 2,6 As well as being an important cause of emergency admissions, COPD is the second most common cause of emergency admission to hospital1 and the fifth largest cause of re-admission,1 costing the NHS an estimated £491M per year. Overall, the number of admissions has increased by 50% in the last decade and COPD now accounts for one million bed-days per annum. These figures suggest that acute, urgent and emergency COPD health care will continue to challenge the NHS for the foreseeable future and create considerable pressure on managers and clinicians to work to resolve the issue.

Evidence-based chronic obstructive pulmonary disease care

Emergency admissions to hospital for long-term conditions, including COPD, form part of the NHS Outcomes Framework. 7 A Royal College of Physicians Audit6 found that, on average, patients spend 8.7 days in hospital during an admission for COPD but also highlighted wide variation in terms of both treatment provision and outcomes among hospitals. This disparity was particularly marked in relation to mortality. It also showed that a significant proportion of the observed variability could be explained by availability and access to expert care and evidence-based interventions. This presents a potential opportunity to improve outcomes for patients with COPD by ensuring that their care is consistently provided to a high standard.

Chronic obstructive pulmonary disease admission care bundle

The COPD admission care bundle is designed to facilitate co-ordinated and timely care for patients admitted to hospital with an acute exacerbation of COPD. 9 The first bundle element aims to ensure that a correct diagnosis of AECOPD has been established. The diagnostic process begins with a history and physical examination and should be supported by early availability of an electrocardiogram (ECG) and chest X-ray. These two diagnostic tests are, therefore, key to supporting successful completion of admission bundle item 1. The aim of this bundle element is to allow alternative diagnoses to an acute exacerbation of COPD to be excluded (e.g. pneumonia, heart failure, cardiac ischaemia). Spirometry is excluded, as this measurement is considered unreliable in the context of an acute admission.

Early recognition and response to hypoxia is critical; however, patients with severe COPD may have a reduced hypoxic respiratory drive. An oxygen assessment should be undertaken and the correct target range prescribed within 30 minutes. For patients with COPD, a target saturation range of 88–92% is suggested pending the availability of blood gas results. 10

Staff should recognise and respond to respiratory acidosis within 1 hour of admission. Patients with the highest mortality from COPD following hospital admission are those who are admitted in respiratory failure; thus, early recognition and an appropriate response to respiratory acidosis are key to improving early mortality. 10 This requires an arterial blood gas for all patients admitted to hospital with oxygen saturations of ≤ 94% (on air or controlled oxygen). Following interpretation of the results of this investigation, early assessment for suitability for non-invasive ventilation (NIV) is required. Current guidelines suggest that patients should be placed on optimum medical therapy (controlled oxygen and nebulised therapy) for 1 hour, following which the need for NIV should be assessed.

Correct prescription of medications (including nebulisers, steroids and antibiotics) is also necessary. Medication (steroids and nebulisers) should be administered to patients within 4 hours of admission. This is important, as the mean mortality rate among patients admitted to hospital with an infective exacerbation of COPD is 7.7%. 6 Their treatment and assessment should be timely, as for any other seriously ill patient. Correct prescription of medications (including nebulisers, steroids and antibiotics) within 4 hours is appropriate given the severity of some COPD patients’ condition.

Finally, as results of the 2003 national COPD audit suggest that review by a respiratory specialist reduces in-hospital mortality,6 and given that the majority of deaths occur within 72 hours of admission, all patients admitted with an acute exacerbation of COPD should be seen by a member of the respiratory team within 24 hours of admission. This could be a specialist nurse or physiotherapist, specialist registrar or consultant. In the BTS pilot, provision of a prescription for oxygen therapy had the greatest impact on mortality. Patients seen by respiratory specialists had markers of COPD exacerbation severity suggesting a higher level of acuity but had a lower mortality than those seen by non-specialists. 11 The components of the admission bundle are reflected in the acronym DARTS (Box 1):

In full, the components of the admission bundle are:

• Statement 1 – a correct diagnosis of AECOPD should be confirmed.

• Statement 2 – an oxygen assessment should be undertaken and the correct target range prescribed within 30 minutes.

• Statement 3 – recognise and respond to respiratory acidosis within 1 hour of admission.

• Statement 4 – medication (steroids and nebulisers) to be administered within 4 hours of admission.

• Statement 5 – review by respiratory team to take place within 24 hours of admission.

Chronic obstructive pulmonary disease discharge care bundle

At 25–30%, the 90-day re-admission rate for patients discharged following an admission with COPD is high but, as yet, there is little evidence for individual interventions that consistently reduce this figure. 6 Structured discharge planning is one intervention that has been shown to reduce further hospital admissions. 12 A consensus was reached on the key elements of a COPD discharge bundle. It was agreed that the elements should be aimed at ensuring that patients have been assessed appropriately prior to discharge and are confident in the use of their medications. It was also felt to be important that patients have ready access to advice and assistance should they deteriorate following discharge from hospital.

The discharge bundle incorporates five elements. The first bundle element states that all patients should have their respiratory medications and inhaler technique assessed prior to discharge. On direct questioning, 98% of respiratory patients report using their inhaler correctly; however, on testing, only 8% showed the correct technique. 13 This problem can be exacerbated in the elderly, for whom issues such as visual acuity, manual dexterity and cognitive impairment can act as additional barriers to correct inhaler use. 14 However, correct use of inhalers is associated with improved outcomes for patients, including a reduction in the risk of exacerbations and hospital admission. 15 Repeated instruction is required to ensure that inhaler technique is optimised. 16

Second, all patients should receive a written plan for how to manage a further acute exacerbation of their COPD and should receive a discharge pack of ‘emergency’ drugs prior to discharge. Self-management plans in COPD teach patients how to carry out disease-specific elements of self-care. They appear to be associated with improved well-being and a reduced risk of hospitalisation. 17 Early treatment of COPD exacerbations is associated with a more rapid recovery from the acute episode, reduced risk of hospitalisation and better health-related quality of life. 18 Self-management strategies are a complex intervention and the optimum form and method of delivery of self-care education are not yet clear. The provision of self-management education and a discharge drug pack, as part of the bundle intervention, is intended to assist the patient in optimising self-management of subsequent exacerbations with the aim of reducing the risk of re-admission. However, it is recognised that this is an element of the care bundle with a less secure evidence base as well-conducted trials of self-management have highlighted that not all patients become successful self-managers. Therefore, not all individuals will experience improved outcomes. 19,20

The third discharge bundle element is that smoking status should be assessed together with a willingness to quit and, in the case of those patients indicating a wish for further assistance, a referral should be made to a stop smoking programme. Smoking remains the biggest preventable cause of death and disease in the UK and accounts for approximately 50% of health inequalities between socioeconomic groups. 21 In a study of factors predicting short- and medium-term mortality in hospitalised patients aged > 65 years, current smoking was the factor associated most strongly with risk of death during the follow-up period. 22 Exposure to cigarette smoke has also been associated with an increased risk of hospital re-admission within 1 year after discharge following an admission with an infective exacerbation of COPD. 23 Finally, two-thirds of smokers expressed a wish to stop smoking when asked if they wished to quit. 24 It is clinically effective and congruent with the bundles’ aim of reducing risk of death and hospital re-admission to include a clear focus on smoking cessation. Clinicians should, therefore, use every patient contact to explore the patient’s wishes about stopping smoking.

The fourth bundle element states that all patients should be assessed for their suitability for pulmonary rehabilitation prior to discharge. Systematic review of the evidence base for the benefits of pulmonary rehabilitation concludes that rehabilitation relieves dyspnoea and fatigue, improves emotional function and enhances patients’ sense of control over their condition. Pulmonary rehabilitation therefore forms an important part of the long-term management of stable COPD. 25 However, the provision of pulmonary rehabilitation in the period immediately following hospital discharge for an exacerbation has also been shown to improve patient well-being in addition to reducing risk of hospital re-admission. 26–28 Review of the enablers and barriers to physical activity in COPD patients identified hospital admission as an opportunity to work with patients to overcome practical and psychological factors preventing patients from increasing activity levels. 29 Therefore, clinicians should aim to actively recognise and address barriers to physical activity.

Finally, community follow-up within 2 weeks of discharge from hospital should be organised. When it is not possible to achieve this, consideration should be given to the establishment of a system whereby patients are contacted by telephone following their discharge from hospital and are offered the opportunity for support. Follow-up for patients following an exacerbation of COPD provides an opportunity to review patients’ medication and offers the opportunity to identify those patients experiencing an early deterioration following discharge. The best timing, mechanism and venue for this follow-up is not yet clear. However, respiratory follow-up of patients within 30 days of discharge is associated with a reduced risk of re-admission. 30 The same benefits may also be obtained through telephone follow-up by the hospital team when this is supported by a comprehensive package of care, including the opportunity for early reassessment in the event of a deterioration. 31 The discharge bundle is reflected by the acronym TAPSS (Box 2):

In full, the components of the discharge bundle are:

• Statement 1 – all patients should have their respiratory medications and inhaler technique assessed prior to discharge.

• Statement 2 – all patients should receive a written plan for how to manage a further acute exacerbation of their COPD and should receive a discharge pack of ‘emergency’ drugs prior to discharge.

• Statement 3 – smoking status should be assessed together with a willingness to quit and, in the case of those patients indicating a wish for further assistance, a referral should be made to a stop smoking programme.

• Statement 4 – all patients should be assessed for their suitability for pulmonary rehabilitation prior to discharge.

• Statement 5 – community follow-up within 2 weeks of discharge from hospital should be organised.

Partnership with the British Thoracic Society

This study was conducted in partnership with the BTS, which had previously undertaken a pilot evaluation of the introduction of COPD care bundles. 9 The pre-existing commitment by some implementation trusts to delivering care bundles and the roll-out of the BTS training programme precluded delivery of a study using a randomised controlled trial design. We, therefore, selected a controlled before-and-after study as the most robust study design to measure any association between care bundles and better costs and outcomes of AECOPD care. This study, therefore, included a group of acute hospital trusts that agreed to deliver the COPD care bundle intervention as well as a group of broadly comparable trusts that did not deliver the intervention during the study period. It involved three different levels of data collection and analysis using mixed-methods research to build a comprehensive data set with which to evaluate the effectiveness, efficiency and acceptability of the care bundle package.

In summary, this research is intended to provide independent evidence of the impact of COPD care bundles on hospital admissions and re-admissions. It also provides information on how a co-ordinated care package might improve quality of care, equity of access, patient and carer experience and service delivery for COPD patients within the acute setting, considering cost implications and implementation challenges. The research also explores potential enablers and inhibitors of the delivery of the COPD care bundles. Going forward, the research could also inform the development and delivery of care bundles for other health conditions.

Identification of sites

There were two types of participating sites in the study (implementation sites and comparator sites) and they were identified using similar methods. To maximise the number and diversity of hospitals invited to participate in the research, a range of approaches were used:

• advertising calls for interest on the BTS website53

• advertising calls for interest on the respiratory section of the NIHR Clinical Research Network (CRN) website54

• approaching respiratory specialists from NHS trusts at BTS scientific meetings

• calling for interest via NIHR CRN-nominated local respiratory research leads

• calling for interest via NIHR CRN delivery managers

• generating new clinical contacts from known ones using ‘snowballing’ techniques

• making ‘cold calls’ to major acute hospitals not otherwise contacted.

Further detail about the identification of sites is available in Chapters 6 and 7.

Recruitment of sites

Once a hospital had expressed interest in taking part in the research, we sent further information about the study. This comprised a link to the NIHR CRN Portfolio database,55 a link to the study website hosted by the University of Bristol,56 a research summary, the full study protocol and a copy of the BTS COPD care bundles pilot study report. 9 Next, the hospital’s status as either an implementor of COPD care bundles or a comparator delivering standard care was determined, and the site was asked to sign a formal agreement to be a participant in the evaluation. Following this, the relevant site-specific information was submitted via the Integrated Research Application System website and contact made with the site’s research and development team to ensure that all appropriate permissions and approvals were in place. This included the appointment of a local principal investigator (PI) at each site to take responsibility for oversight of data collection and patient care. Further detail about the recruitment of sites is available in Chapters 6 and 7.

Allocation to study level

The study design allowed for up to eight implementation sites to be assigned to level 2 for data collection and analysis purposes. This allocation depended on two conditions being met; first, delivery of both an admission and a discharge bundle for COPD care and, second, a willingness to report on the level 2 data requirements. Any remaining implementation sites were allocated to level 1 participation. Eight comparator sites were also allocated on the basis of a number of prespecified criteria (i.e. number of COPD admissions, 28-day re-admission rates and COPD mortality rates) to level 2 as means of obtaining eight implementation–comparator site pairs. Finally, six sites from those allocated to level 2 were purposively selected as level 3 case study sites. Further detail about the allocation criteria and process is available in Chapters 5, 6 and 7.

Identification of participants

Up to 10 individuals (i.e. patients admitted following an acute exacerbation of COPD or their carers) were identified by respiratory team members as being appropriate for participation in the study at a sample of level 3 case study locations. Individuals were selected at various stages along the COPD patient-care pathway, taking account of a number of factors such as their health status, level of cognition and ability to communicate easily with the research team. Further detail about the selection criteria and process is available in Chapter 7.

Recruitment of participants

All participants invited to join level 3 of the study were recruited using procedures set out by the International Conference on Harmonisation (ICH) good clinical practice (GCP) guidelines,57 with informed written consent being obtained prior to the start of any data collection. Patients and their carers were interviewed during the relevant admission period and were, whenever possible, interviewed again 30–90 days post discharge, either face to face or via the telephone. Staff involved in a patient’s care, be that in the acute or community setting, were also invited to interview in a similar manner and at similar time points. Further detail about the recruitment of participants is available in Chapter 7.

Level 1 data collection

All sites were asked to report a range of aggregated routine data including COPD admission rate, COPD re-admission rate at 28 and 90 days, overall re-admission rate at 28 days, in-hospital mortality for COPD admissions, length of stay for COPD admissions, total number of bed-days for COPD admissions, total number of COPD patients seen and discharged by ED and, at implementation sites, the total number of patients in whom the bundle was used. Further detail about level 1 data collection is available in Chapter 5.

Level 2 data collection

In addition to the level 1 data, level 2 sites were required to provide pseudo-anonymised patient-level data, including age, sex, ethnicity and some geographical variables, for all patients having an admission for an acute exacerbation of COPD. A selection of non-identifiable clinical information about individual patients was also given, including admission month and year; source of admission; ICD-10 diagnosis codes; Office of Population, Census and Surveys (OPCS) procedure codes; length of stay – total and by ward type; discharge destination; health-care resource group (HRG) codes; pseudo-anonymised consultant and general practitioner (GP) practice codes; emergency inpatient admissions for COPD and other causes; outpatients appointments; ED attendances; in-hospital mortality; 90-day mortality, including the number of days after discharge that death occurred by data linkage to death registry information; and, at implementation sites, the total number of patients in whom the bundle was used. Finally, level 2 sites reported on a series of process measures associated with the delivery of components of COPD care by returning information extracted from a randomly selected sample of 140 patient records per site. Further detail about level 2 data collection is available in Chapter 5.

Level 3 data collection

In addition to the level 1 and 2 data, level 3 sites provided data on the process of care bundle implementation, the context in which care bundles were delivered, the impact of care bundles on staff, patients and carers, and, when appropriate, the nature of usual COPD care. Data collection was carried out for the duration of the study through document analysis, non-participant observation over extended site visits and in-depth interviewing following both admission and discharge. It was guided by topic guides and observation schedules, and conducted at a selection of both implementation and comparator sites. Further detail about level 3 data collection is available in Chapter 7.

Quantitative analysis of effectiveness data

Level 1 data were used principally to calculate the mean change following the introduction of care bundles for each site for all outcome measures. This mean change was then compared between implementation and comparator sites using ordinary linear regression, with adjustment for the following measures from the first period: number of COPD admissions, 28-day re-admission rate and in-hospital mortality rate. Further detail about level 1 data analysis is available in Chapter 5.

Level 2 data were also used in a series of appropriate regression models, depending on the outcome type, to compare the difference in change between the implementation and comparator groups. These models included a ‘group–time’ interaction term to estimate the difference in change in outcome between the implementation and comparator sites before and after implementation of the care bundles. This approach reflects the fact that the samples in the ‘before’ and ‘after’ periods captured data from sets of predominantly different individuals. All models took appropriate account of the ‘paired’ design by including indicator variables to distinguish each pair of sites. This technique accommodated any between-site variation (i.e. clustering) in outcomes. In the case of patients admitted multiple times during the study period, when these could be identified, a sensitivity analysis was conducted using only data from the patient’s first admission, with the primary analysis elaborated as necessary to accommodate any correlation in outcome between a single patient’s episodes of care. Further detail about level 2 data analysis is available in Chapter 5.

Quantitative analysis of cost-effectiveness data

The economic impact of care bundle implementation at level 1 sites was evaluated using aggregated trust-level data to describe the cost of COPD care per admission for both implementation and comparator sites during their ‘before’ and ‘after’ time periods. A simple unit-costing methodology was deployed, based on a weighted average of non-elective COPD-related HRG codes, to estimate the incremental impact of the care bundle delivery on COPD-related NHS secondary care costs.

A further economic evaluation considering the 90-day period following the index admission for AECOPD was also undertaken in level 2 sites. This involved the estimation of per-patient secondary care NHS costs using a more in-depth HRG unit-costing methodology,59 where patient-specific resource use was valued using comprehensive, nationally representative sources (e.g. NHS Reference Costs 2014–2015,60 the British National Formulary61 and Unit Costs of Health and Social Care62). Information on procedures and investigations undertaken during an inpatient stay (e.g. X-rays, onward referrals and drugs prescribed) was also collected by reviewing medical records alongside routine data as part of the estimation of costs of admissions and re-admissions during the 90-day post-discharge period. Linked information, on 90-day mortality (including the number of days between discharge and death), was also collected.

The duration of the interactions between a subsample of admitted patients and clinical staff was recorded in a ‘time and motion’ study at level 3 sites to provide an estimate of the staff time involved in treating COPD patients, as a way of informing the level 2 cost-effectiveness analysis. Combining the various data on costs and mortality allowed an estimate of cost-effectiveness to be calculated as a ratio of the difference in NHS secondary care costs between intervention and comparator sites to the between-site differences in 90-day mortality. Uncertainty surrounding this estimate was quantified using cost-effectiveness acceptability curves (CEACs) and, when appropriate, a deterministic sensitivity analysis was conducted.

Information on post-discharge resource use was collected from a subsample of consented level 3 patients during telephone interviews. This information provided a descriptive analysis of differences between types of site in the use of primary care services, community care and informal care up to 90 days post discharge. Further detail about the cost-effectiveness data analysis is available in Chapter 6.

Qualitative analysis

The observational data from level 3 sites were collected in note form, developed as soon after the period of observation as possible, before being replicated as a Microsoft Word (Microsoft Corporation, Redmond, WA, USA) document and uploaded into the proprietary qualitative analysis package NVivo (QSR International, Warrington, UK). All interviews were digitally recorded, fully transcribed and anonymised before being uploaded into NVivo in readiness for coding and analysis. Following development of detailed, narrative case study descriptions of each level 3 site, both observational and interview data were examined using a cross-case thematic analysis. 63 This approach was used to draw out the key issues in the data, using a coding framework that was developed collaboratively by members of the research team. This process enabled both inductive and deductive analysis, focusing on the study research questions but also enabling the incorporation of emergent views and experiences expressed by interviewees. All data were analysed and interpreted by at least two qualitative researchers, in order to cross-reference findings and ensure both clarity and consistency. The analysis sought to identify similarities and differences between sites, highlighting those aspects that might be transferable to other hospitals implementing, or intending to implement, care bundles. Attention was also given to any overlaps or divergence between aspects of practice seen at implementation and comparator sites. Further detail about level 3 data analysis is available in Chapter 7.

Design

A controlled retrospective cohort design was used to compare outcomes of introducing care bundles with usual care for patients admitted to hospital with an acute exacerbation of COPD. Quantitative analyses compared trust-level monthly aggregated data, referred to as level 1, between 19 acute trusts that delivered the COPD care bundles intervention (i.e. implementation sites) and a broadly comparable group of 12 similar acute trusts that did not deliver the COPD care bundles intervention (i.e. comparator sites).

From these 31 sites, a sample of seven implementation and seven comparator sites were identified for more detailed quantitative study, referred to as level 2, including gathering of process data from routine hospital discharge data and follow-up of mortality. Initially, it was planned to match (1 : 1) implementation with comparator sites on current COPD admission, 28-day re-admission and mortality rates prior to bundle implementation. However, changes in various sites’ plans regarding bundle implementation, as well as their ability to provide individual-level data for the more detailed quantitative study, meant that the proposed matching was not always possible. Level 2 analyses, therefore, centred on all implementation and comparator sites that provided appropriate data.

All 31 participating sites were allocated an index date and quantitative data were collected for the 12 months before and after this date. The index date for implementation sites was based on when the admission and discharge bundles were implemented. As sites implemented the bundles according to their own time frame, we relied on local investigators to provide a ‘best estimate’ date of implementation. Each comparator site was allocated the index date of a comparable implementation site.

All sites were asked to provide a range of routinely collected, trust-level aggregate data, and level 1 analyses were based on these data. This included COPD 28-day re-admission rates, length of stay for COPD admissions, total number of bed-days for COPD admissions, COPD 90-day re-admission rates and in-hospital mortality. In implementation sites, the total number of patients each month in whom the bundles were used was also collected, when available.

As well as providing the above-mentioned aggregate data, sites agreeing to provide level 2 data were also asked to provide routinely collected, anonymised, patient-level data for the 12 months before and 15 months after their index date. We requested 15 months of post-index data to allow for 28- and 90-day re-admission rates to be calculated for months 1–12 after the index date. The data requested included all emergency COPD hospital inpatient admissions and, for those patients with a COPD admission, all their non-COPD admissions, acute care data, ED data (when available), 90-day mortality data and indication of whether or not patients received the admissions and/or discharge bundle. When sites providing level 2 data were unable to provide aggregate level 1 data, these were calculated by the research team from the individual-level data provided in level 2. One level 2 site, owing to technical issues, was unable to provide level 1 data for a substantial proportion of the study period and was able to provide level 2 data for only 4 months pre index and for the post-index data period. This site was retained for level 2 analyses making use of all available data, but was not included in level 1 analyses as a third of monthly data would be missing regardless of estimation from level 2 data.

Furthermore, for each site providing level 2 data, 140 patients were randomly identified in the post-index date period for case note extraction. Identification of the sample of patients to be studied was done at the University of Bristol after sites provided an anonymised post-index date listing of COPD admissions data. Patients were selected evenly over the 12-month period, based on their date of admission, to achieve a balance across the year and avoid seasonal variation having an impact on results. In order to satisfy the requirement for 140 patients, 11 patients were chosen from months 3, 6, 9 and 12 post index date and 12 patients were chosen from months 1, 2, 4, 5, 7, 8, 10 and 11 post index date. A list of anonymised patient IDs and dates of admission was given to each site to enable them to conduct case note extraction locally, using paper forms. These data were then input into an Access database at the University of Bristol.

As it was expected that not all case notes would be available, an additional list of patient IDs satisfying the same criteria as above was generated. This list was used to identify ‘replacement’ IDs should any site report being unable to identify patients in the original list. When replacement patient IDs were requested, the reason for the request (e.g. being unable to locate notes, confirmation that the patient does not have COPD) was logged. We attempted to maintain a balance of admissions over the year by assigning a replacement with a comparable admission date.

Study population

The target study population was all people aged ≥ 18 years who were admitted to hospital with an acute exacerbation of COPD as their primary cause of admission (ICD-10 diagnostic codes J41–J44). Elective admissions for COPD or admissions where COPD was not the primary cause were excluded.

Outcomes

The primary outcome of the study was COPD re-admission rate at 28 days. It was considered as part of both the level 1 data analysis (i.e. using monthly trust-level data) and the level 2 data analysis (i.e. using individual-level data). Secondary outcomes were:

• total number of COPD admissions

• in-hospital mortality for COPD admissions

• length of stay for COPD admissions

• total number of bed-days for COPD admissions

• COPD re-admission rate at 90 days

• overall re-admission rate at 28 days

• total number of COPD patients seen and discharged from ED

• total number of COPD patients in whom bundle used (implementation sites only)

• 90-day mortality rate.

All secondary outcomes were collected using level 2 and level 1 data, except for the 90-day mortality rate, which was collected only from level 2 sites. In addition, sites providing level 2 data also provided a number of process measures and we considered which individual elements of both care bundles were delivered. These are described in Chapter 1.

Case mix variables

For sites providing level 2 data, we had access to individual-level data allowing us to describe the patient population in terms of routinely collected data: age, sex, ethnicity, socioeconomic deprivation and comorbidities.

Age was calculated and provided by sites and used in analyses as a numeric variable. Sex and ethnicity were coded by sites according to the Hospital Episode Statistics data dictionary. 67 Ethnicity was initially categorised into six categories (‘White’, ‘Mixed’, ‘Asian’, ‘Black’, ‘Chinese’ and ‘Not known/not recorded’) but, owing to low frequencies in some categories, was ultimately recoded as ‘White’ and ‘Other’. Socioeconomic deprivation was determined by the Index of Multiple Deprivation (IMD) rank of the lower super output area (LSOA) of the postcode where the patient lived. When sites could not provide this measure of deprivation, they were advised to use the National Perinatal Epidemiology Unit tool68 for calculating IMD from postcodes. When IMD rank was still not provided by the site, the University of Bristol team assisted sites to identify the associated IMD rank from LSOA data. Using all ranks for England and Wales combined, quintile cut-off points were identified and applied to our sample. The IMD rank quintiles were used as a categorical variable in the analysis.

For each admission, all secondary diagnosis codes were used to calculate the Charlson Comorbidity Index (CCI) score. The score was used as a numeric measure in analysis. One site failed to provide secondary diagnosis codes for any of its recorded admissions; the CCI score in this case was set to missing.

Descriptive analyses

Characteristics of participating sites were summarised using frequencies and proportions for categorical variables and for continuous measures means and standard deviations (SDs) or medians and interquartile ranges (IQRs) where data were skewed.

Completeness of level 1 and level 2 data (outcomes and case-mix variables) was summarised by site and time period (pre- or post-index date) in terms of the number and proportion of months for which data were available (level 1) and the number and proportion of admissions for which data were available (level 2).

Prior to any comparative analyses, outcomes were first described using descriptive statistics. The pre- and post-index date 12-month means (SD) of each level 1 outcome were calculated separately for each site and the means across implementation and comparator sites were presented with 95% CIs. For each level 2 outcome, frequencies and proportions (or means and SDs or medians and IQRs) were used to summarise the pre- and post-index date outcomes. Pooled data across implementation and comparator sites are presented separately.

Comparative analyses

Level 1 data were used, principally, to calculate the mean change following the introduction of care bundles for each site for all outcome measures. This mean change was then compared between implementation and comparator sites using ordinary linear regression, with adjustment for the following measures from the first period: number of COPD admissions, 28-day re-admission rate and COPD mortality rate.

For individual-level patient data (level 2), we used a range of appropriate multilevel regression models depending on the outcome type to compare the change in outcomes after the index date between implementation and comparator groups allowing for clustering within patients and NHS trusts. Logistic regression was used for binary outcomes (re-admission outcomes, in-hospital mortality and 90-day mortality) and negative binomial regression was used for length of stay and the number of ED attendances, because of the large variance in outcome. Where logistic models failed to converge, Poisson models were used instead. These models included terms for group and time period as well as an interaction term between the two to estimate the difference in change in outcome between implementation and comparator sites before and after the introduction of the intervention. This approach is required because the samples in the ‘before’ and ‘after’ periods will be of (mostly) different individuals. A sensitivity analysis was also conducted of the primary outcome using only data from patients’ first admission to eliminate the correlation between repeated observations within patients.

Models were first run without adjusting for potential confounding. After this initial analysis, all models were then rerun adjusting for patient age, sex, socioeconomic status, CCI score and ethnicity. We were limited in the number of individual patient-level data available. We therefore opted a priori to include all demographic variables that were well completed and for which there was evidence in the literature that they were associated with COPD. As some sites were not able to provide some of these covariates, we first adjusted for the most complete covariates then added variables individually. To account for the fact that one site did not have 12 months’ worth of pre-index date data, we also ran a separate model additionally adjusting for month of year.

For each logistic regression analysis, odds ratios (ORs) and 95% CIs were presented, and for negative binomial and Poisson models the incidence rate ratios (IRRs) and 95% CIs were presented.

Patient characteristics were compared between implementation and comparator sites using t-tests for continuous measures and chi-squared tests for independence and trends for categorical variables.

The model-building approach for each outcome is described in Table 1.

Case note extraction analysis

The extent to which each site implemented each element of the bundle was recorded using case note extraction data and the results were summarised by group, using frequencies and proportions. Chi-squared tests were used to compare the proportion of patients receiving bundle elements between implementation and comparator sites.

Coding of the implementation of each bundle element is limited to data that could be reliably captured from a retrospective inspection of case notes. As such, some nuances of the bundles could not be captured; for example, when the time of certain activities was not recorded we could not verify their delivery within recommended time frames. This, however, is addressed in the qualitative analyses of level 3 data in Chapter 7. Details of how the delivery of each bundle element was coded are provided in Tables 2 and 3.

Matching process

We intended to match implementation and comparator sites (level 2) based on COPD admission rate, COPD death rate and COPD re-admission rate from 2012 (i.e. pre-index date for all sites). However, owing to later-stage dropouts and sites changing levels (level 2 sites changing to level 1 and vice versa), matching was not always possible. For this reason, sites were not treated as matched pairs. We have, however, taken note of whether sites were part of the same NHS trust and included ‘NHS trust’ as a level in our multilevel models; there were two level 2 sites from the same trust.

Deriving level 1 data from level 2 data

Level 2 sites were asked to provide monthly aggregate data (i.e. the same as that provided by level 1 sites) as well as more detailed patient-level data. Some level 2 sites, however, provided only the patient-level data. These data held most of the information necessary to derive the requested level 1 data. Whenever possible in these sites, level 1 data were derived from the level 2 data. It is impossible to determine, however, if our methods of deriving level 1 outcomes are identical to the results the sites would have provided themselves.

Sites forming part of the same trust

Two level 2 sites were part of the same NHS trust and might, therefore, have similar working practices and outcomes. The sites were in the same arm of the study and any potential within-trust clustering was accounted for using multilevel modelling in which ‘trust’ was included as a level.

Missing case note extraction data

Two level 2 sites from the same NHS trust did not have capacity to complete two full sets of the case note extraction audit forms (i.e. 140 sets of notes each). As a compromise, they performed the case note extraction on 70 sets of notes at each of the two sites. This reduced the total number of case report forms that we received. Seven other sites also failed to provide the full 140 case note extraction report forms owing to limited time and resources.

Sites not providing the full 12 months pre- and post-index date (plus 90 days) data set

The study protocol stated that all sites would provide data for the 12-month period prior to implementation of the intervention (i.e. pre-index date) and 12-month period following implementation (i.e. post-index date). Level 2 sites were also asked for an additional 3 months of data after the 12-month post-index date to enable the calculation of outcomes, which look forward 28 days and 90 days (i.e. 28-day re-admission variables, 90-day COPD re-admission and 90-day mortality). Of the 14 level 2 sites recruited, only seven provided data for the full time period requested. Data were missing as follows:

• One site (COMP08) migrated its hospital record systems 6 months into the study period and so could provide data only for 4 months pre index date. We used all available level 2 data for this site in the patient-level analyses and ran sensitivity analyses with and without them.

• Four sites (COMP04, COMP08, IMP04 and IMP11) provided sufficient post-index date data to calculate 28-day outcomes for the full period, but 90-day outcomes for these sites excluded the last 1–2 months of the study period.

• Two sites (IMP02 and IMP03) did not provide any data after the 12-month post index date. As such, 28-day re-admission outcomes for these sites exclude the last 28 days of follow-up. Similarly, 90-day re-admission and 90-day mortality outcomes for these sites exclude the last 90 days of follow-up.

The missing data mean that there is an imbalance in time of the year between pre- and post-index periods for these sites and unequal numbers of data per site. To account for the imbalance, month of the year was included in regression models (see model 6 described in Comparative analyses). Furthermore, a sensitivity analysis was performed for the primary outcome using the same number of post-index months in all sites, this period was 9 months of post-index date data for all sites.

Description of participating sites

As this is an observational study, the decision of whether or not to implement COPD care bundles, and, if implementing, when and which bundles to implement, was taken by the sites themselves based on their own needs, capacity and QI strategy. Thus, understanding the baseline characteristics of sites is important to putting clinical outcomes into context.

Thirty-one sites provided data: 19 implementation and 12 comparison sites. Of the 19 implementation sites, seven implemented both the admission and the discharge bundles, and it was these sites that contributed patient-level data for level 2 analyses. Although it is unlikely that all components of care bundles were implemented in full on any given date, all participating sites were able to provide an approximate date from which their bundles were implemented, and this became their index date. When sites implemented both types of care bundle, these were implemented between 1 January 2013 and 1 October 2015. Sites that opted to implement the discharge bundle only did so earlier (between 1 July 2011 and 1 June 2013). The comparator sites were allocated a range of index dates so as to be broadly comparable to the implementation sites. These ranged from 1 January 2013 to 1 October 2015. A summary of the key characteristics of the 31 participating sites is provided in Table 4.

Participating sites were located across England, with one site in Wales, and they comprised both large city hospitals and smaller district hospitals. Pre-index date level 1 data (i.e. monthly trust-level aggregate data calculated by sites themselves) allow us to see how the sites compared in terms of the number of COPD admissions they had (as an indicator of hospital size and the demographic of the patients they serve) and their 28-day COPD re-admission rate. When comparing sites on their mean monthly number of COPD admissions, a large amount of variation was observed. COMP03 had, on average, only 17.8 COPD admissions per month while IMP09 had 95.5 admissions per month. On average, implementation sites tended to have slightly more COPD admissions than comparator sites, although there was no evidence that the difference differed from zero [52.5 (SD 18.2) admissions per month in implementation sites vs. 48.0 (SD 21.0) admissions per month in comparator sites; p-value = 0.552]. Among the implementation sites, those that chose to implement both care bundles were similar in size to those that chose to implement only the discharge bundle [51.8 (SD 20.0) admissions per month in sites implementing both vs. 53.7 (SD 16.1) admissions per month in sites implementing only the discharge bundle; p = 0.830].

The mean monthly 28-day re-admission rates for COPD during the pre-index period showed considerable variability between sites, with re-admission rates varying between 4.4% at IMP04 and 43.6% at IMP18. On average, comparator sites had lower re-admission rates than implementation sites during the pre-index period [11.5% (SD 3.6%) vs. 15.1% (SD 9.2%); p = 0.137], although there was no evidence that the difference differed from zero.

Level 2 sites provided patient-level data, thereby allowing us to consider the demographic characteristics of patients with AECOPD observed in the study. The characteristics of patients having an emergency COPD admission during the pre-index date period, are described in Table 5. Patients at the implementation sites tended to be slightly younger and less likely to be recorded as ‘white’. Patients from that group were slightly more likely to be from the least socioeconomically deprived quintile.

Data completeness

For all outcomes except ED attendances, > 95% of site-months had non-missing outcome data. Twelve sites were unable to provide ED data such that non-missing data were available for only 61.3% of site-months. Table 29 in Appendix 1 summarises the completeness of level 1 data after all attempts were made to assist sites to collect the necessary information.

Completeness of follow-up data for level 2 outcomes has previously been described. When there were sufficient follow-up data, all level 2 outcomes (28-day re-admission for COPD, 28-day all-cause re-admission, length of stay, 90-day COPD re-admission, death in hospital, number of ED attendances and 90-day mortality) were calculated.

Level 1 outcomes

Level 1 outcomes were reported on a monthly basis for the 12 months before and after their index date. Pooled results across implementation and comparator sites, comparing the pre- and post-index date periods, are presented in Table 6. Regression analyses on monthly outcomes were conducted to estimate how comparator and implementation sites compare after adjusting for the number of COPD admissions, overall 28-day re-admission rate and in-hospital mortality rates in the pre-index date period.

When comparing outcomes between the pre- and post-index date periods, changes were very small in both implementation and comparator sites. There was no evidence that the change experienced in the implementation sites differed from that observed in the comparator sites.

Implementation sites were asked to report the number of admission and discharge bundles used each month. These data were often not recorded prospectively, so many sites were unable to report them. The case note extraction analysis from level 2 sites described in Case note extraction, however, gives an insight into how and when bundles were delivered at implementation sites.

Level 2 outcomes

Level 2 outcomes were available for 14 sites: seven implementation and seven comparator sites. Details of these outcomes for both the pre- and the post-index date periods are summarised by site type (i.e. implementation or comparator) in Table 7.

The results suggest small improvements over time for most outcomes in both implementation and comparator sites. To further understand these changes and to assess whether or not changes differed between implementation and comparator sites, regression analyses accounting for potential confounding were conducted and are described below for each outcome.

Primary outcome: re-admission with chronic obstructive pulmonary disease within 28 days

There were a total of 19,097 emergency hospital admissions for COPD during the full study period, of which 13.0% resulted in a re-admission for COPD within 28 days. In the pre-index date period, 11.6% of COPD admissions were re-admitted within 28 days in implementation sites. In comparator sites, the proportion was 14.7%. Post index date, this proportion fell to 10.8% in implementation sites and increased slightly to 14.7% in comparator sites.

To estimate changes over time in the implementation and comparator groups and to test whether or not these changes differed between the groups, multilevel logistic regression models were run, accounting for patients having multiple admissions and clustering within NHS trusts. In a model unadjusted for potential confounders (Table 8, model 1), it is observed that both groups had a reduction in the odds of re-admission after the index date, although the ORs were close to 1 and the 95% CI overlapped with 1. A group–time interaction term was included in the model and a likelihood ratio test was conducted to assess whether there was evidence of a difference in the pre–post changes between the two groups. The odds of a reduction in COPD 28-day re-admission from pre index to post index in the implementation group is 0.97 times that in the comparator group. The 95% CI (0.79 to 1.20) is wide, overlapping with 1, and the p-value for the likelihood ratio test was 0.804, suggesting no evidence of a difference between implementation and comparator sites.

To adjust for potential confounding, we considered a number of prespecified patient-level characteristics that might be important: age, sex, ethnicity, socioeconomic status and CCI score. This model (see Table 8, model 2) included 18,849 patients and, with the additional adjustment, results were very similar to those obtained in the unadjusted model.

Further adjustment for CCI score (see Table 8, model 3) resulted in fewer observations in the model (n = 17,459 admissions) because of missing data at a single comparator site. This model shows a 7% drop in the odds of re-admission post index date in the comparator sites, which is larger than that estimated in models 1 and 2, but the confidence interval remains wide. The results in the implementation sites were comparable to those obtained in model 2. After adjusting for CCI score, age, sex and ethnicity, the OR for the group–time interaction became slightly > 1, suggesting that the change in the comparator sites is larger than that in the implementation group. The confidence interval remained wide and included the null and the p-value for the likelihood ratio test for the interaction term was 0.884.

We next ran a model that adjusted for age, sex, ethnicity and IMD quintile as well as one that adjusted additionally for CCI score (see Table 8, models 4 and 5). The results were comparable to those from model 3 and similar results were seen when month of year was also adjusted for (see Table 8, model 6).

A sensitivity analysis restricted to the first admission a patient would have had (12,423 admissions) and unadjusted for potential confounders showed larger changes in implementation (OR 0.70, 95% CI 0.57 to 0.86) and comparator sites (OR 0.80, 95% CI 0.66 to 0.97) than in the model including all admissions (Table 8, model 1), but, as observed in the full analysis, there was no evidence that these changes differed between the two groups of sites (p-value for interaction = 0.643).

Secondary outcome: all-cause re-admissions within 28 days

The proportion of COPD admissions that were followed by an emergency re-admission within 28 days for any cause (including COPD) was 23.6%, which was twice the proportion for COPD-specific re-admissions. In the pre-index date period, the proportion was 22.3% in implementation sites and 25.6% in comparator sites, while in the post-index date period the proportions were 19.8% in implementation sites and 25.8% in comparator sites.

We adopted the same analytical approach to secondary outcomes as the primary outcome, presenting the results of the analyses in Table 9.

In a model unadjusted for potential confounders (see Table 9, model 1), we observed that there was a small increase in the odds of re-admission in comparator sites, although the 95% CI was wide and the result could be due to chance. In implementation sites, however, there was a reduction and the 95% CI excluded the null. There was weak evidence that the change in the implementation sites was greater than that observed in the comparator sites (OR 0.85, 95% CI 0.71 to 1.01; p = 0.060).

When first adjusting for age, sex and ethnicity, we obtained very similar results to the unadjusted model (see Table 9, model 2). In addition, adjusting for CCI score reduced the available sample size due to missing data in this covariate (see Table 9, model 3). In this model, the change in implementation sites was similar to that observed in earlier models, but the change in the comparator sites became smaller. Further adjustments (see Table 9, models 4–6) did not alter the interpretation of the results.

Secondary outcome: re-admission with chronic obstructive pulmonary disease within 90 days

Some sites provided insufficient data to allow 90-day outcomes to be assessed for the full 24-month study period. Thus, analyses of 90-day outcomes relate to the 18,383 admissions for COPD for which there were sufficient follow-up data to assess re-admission within 90 days. Of these, 23.8% resulted in an emergency re-admission for COPD within 90 days. In the pre-index date period, these were 21.6% in implementation sites and 25.7% in comparator sites. Post index date, these were 21.3% in implementation sites and 26.1% in comparator sites.

To estimate changes post index date in the implementation and comparator groups, we first attempted to use multilevel logistic regression models, but these failed to converge. We, therefore, used multilevel Poisson models accounting for patients having multiple admissions and clustering within hospitals. The results of the analyses are presented in Table 10.

In a model unadjusted for potential confounders (see Table 10, model 1), we observed that both groups had a reduction in the rate of re-admission after the index date, although the IRRs were close to 1 and the 95% CI overlapped with 1. There was no evidence that the change differed between the groups and adjustment for potential confounders had minimal reduction in any effect estimates (see Table 10, models 2–6).

Secondary outcome: length of stay for chronic obstructive pulmonary disease admissions

There were a total of 19,381 hospital admissions for COPD during the full study period, and the median length of stay for these admissions was 4 days (IQR 1–8 days). In the pre-index date period, the median length of stay was 4 days (IQR 2–9 days) in implementation sites and 4 days (IQR 1–7 days) in comparator sites. Post index date, these shifted slightly to 4 days (IQR 1–8 days) and 3 days (IQR 1–8 days), respectively.

To estimate differences over time in the implementation and comparator groups, and to test whether or not the changes differed between the groups, multilevel negative binomial regression models were run accounting for patients having multiple admissions and clustering within hospitals.

In a model unadjusted for potential confounders (Table 11, model 1), we observed that both groups experienced a drop in the incidence rate post index date, but in both cases the 95% CI overlapped with 1. Although the drop was greater in the implementation group, the upper limit of the 95% CI for the interaction term crossed 1, indicating that there was no evidence of a difference. Further adjustment for patient characteristics did little to change any of the effect estimates (see Table 11, models 1–6).

Secondary outcome: number of emergency department attendances

After each emergency COPD admission, we counted the number of ED attendances that patients experienced until their next COPD admission or the end of follow-up. Sites COMP08 and IMP01 were excluded from the analysis because of missing ED data for the study period. The number of attendances at ED associated with each admission, as well as the duration of follow-up, were modelled using multilevel negative binomial regression models, accounting for clustering within trusts and repeated measures within patients.

There were 15,646 COPD admissions for which there was subsequently at least 1 day of follow-up to measure ED attendances. The median number of subsequent ED attendances was 0 but it ranged from 0 to 35. For implementation and comparator sites in the pre-index date period, the median number of attendances was 0 (IQR 0–1). In an analysis unadjusted for potential confounders we observed that the rate of ED attendances increased post index date in the comparator sites (IRR 1.14, 95% CI 1.04 to 1.26) and decreased in the implementation sites (IRR 0.63, 95% CI 0.56 to 0.70), a difference which was unlikely to arise by chance (p < 0.001). As shown in Table 12, adjusting for potential confounders had minimal effect on these differences.

In a post hoc analysis restricted to events within the 28 days after an inpatient admission for COPD, similar changes in ED attendance were observed in the implementation sites (IRR 0.55, 95% CI 0.48 to 0.63) and changes in the comparator sites became smaller (IRR 1.06, 95% CI 0.95 to 1.18). Results of the same kind were found when the follow-up window was extended to 90 days (implementation sites: IRR 0.58, 95% CI 0.52 to 0.65; comparator sites: IRR 1.09, 95% CI 1.00 to 1.20).

Secondary outcome: in-hospital mortality for chronic obstructive pulmonary disease admissions

Of 19,343 admissions, there were a total of 758 in-hospital deaths during the full study period. In the pre-index date period, there were 192 in-hospital deaths (4.1% of admissions) in implementation sites and 198 in comparator sites (4.4% of admissions). In the post-index date period there were 158 in-hospital deaths in implementation sites (3.3% of admissions) and 210 in comparator sites (3.9% of admissions).

Regression analyses to estimate differences over time in the implementation and comparator groups and to test whether the changes differed between the groups after adjustment for potential confounding are summarised in Table 13.

In a model unadjusted for potential confounders (see Table 13, model 1), we observed that both groups had a reduction in the odds of in-hospital death after the index date. There was no evidence, however, that the reductions differed between the groups as the confidence interval for the interaction term overlapped with 1 and the p-value of the likelihood ratio test was 0.992. As in all other analyses, further adjustment for patient characteristics reduced the sample size for the models (see Table 13, models 2–6), but did not change our interpretation of the results.

Secondary outcome: 90-day mortality for chronic obstructive pulmonary disease admissions

As described above, not all sites were able to provide sufficient data to calculate 90-day outcomes for the full follow-up period. Thus, we examined 90-day mortality post discharge for the 17,664 hospital admissions where the patient was discharged and had sufficient follow-up data. There were 1160 deaths within 90 days of these admissions. In the pre-index date period, there were 285 deaths within 90 days of discharge in the implementation sites (6.4% of admissions) and 347 deaths in comparator sites (8.0% of admissions). Post index date, the number of fell to 204 in the implementation sites (5.2% of admissions) and to 324 in the comparator sites (6.5% of admissions).

To estimate differences over time in the implementation and comparator groups and to test whether or not the changes differed between the groups, we first attempted multilevel logistic regression models, but these failed to converge so multilevel Poisson regression models were run instead. The results of our analyses are presented in Table 14.

In unadjusted and adjusted models, we observed that there was no evidence of change post index date in 90-day mortality in either the comparator or the implementation groups as the IRRs were close to 1 and the 95% confidence intervals overlapped with 1 (see Table 14, models 1–6). There was also no evidence that treatment group modified the difference.

Excluding sites with limited pre-index date data

Comparator site COMP08 provided limited pre-index date data owing to technical issues, and our analysis of the primary outcome was rerun excluding this site. The results of this analysis are available in Appendix 1 (see Table 31) and suggest that COPD re-admission at 28 days increased slightly in the post-index date period for comparator sites, although the confidence interval of the OR overlapped with 1 (as in the model including COMP08). The interaction effect became slightly larger, but there remained no evidence that the change post index date differed between implementation and comparator sites.

Restricting the post-index date period to the minimum duration consistent across all sites

The case where we used the minimal number of post-index date data has very similar results for all six model adjustments in both primary and sensitivity analyses. The results are presented in Appendix 1 and are very similar to those obtained when all available data were used.

Case note extraction

All level 2 sites were asked to examine the case notes of 140 randomly chosen COPD post-index date admissions in order to record the delivery of different elements of the admission and discharge bundle. As described earlier in this chapter, not all sites were able to provide 140 sets of these notes owing to local time constraints. One site (IMP20) examined case notes for 140 patients but we were unable to use the data provided because of errors in the anonymised IDs that the site had generated. Detail available in Appendix 1 describes the number of case notes returned, by site, as well as indicating whether or not the case notes had recorded patients as receiving an admission or discharge bundle.

Site COMP01 was unique among comparator sites in that, for a small number of patients, it was recorded that care bundles had been provided. Contrary to expectation, sites that were nominally implementing admission and discharge bundles did not always report having delivered these bundles.

To understand who received care bundles at the implementation sites, we considered the patient and admission characteristics of sites where admission and discharge bundles were or were not delivered (see Table 31, Appendix 1). There was no pattern of admission bundles being less likely to be delivered on a particular point of the week or at a particular time of day. There tended, however, to be more men in the group receiving bundles than in the group that did not receive the bundles. Similar trends were observed for the discharge bundle, although here the patients who did not receive a care bundle tended to be slightly older than those who received it. As well as looking at whether each site reported delivering care bundles, we also considered whether or not different individual elements of the bundle were delivered, as evidenced in the case notes. For the admission care bundle, our results are summarised in Table 15.

The first element of the admission bundle relates to a correct diagnosis of AECOPD, which is assessed through having a chest X-ray and an ECG result within 4 hours. These elements were delivered for most patients in both implementation and comparator sites, although the proportion receiving the element was higher in the implementation sites. The second element of an admission care bundle relates to recognising and responding to respiratory acidosis and the data tell us that most patients received this care. There was little difference between comparator and implementation sites.

Of the patients requiring supplemental oxygen, most received the correct prescription, and this occurred more frequently at implementation sites. Similarly, in those patients for whom antibiotics and steroids were deemed necessary, most received them within the specified 4 hours of admission. Patients in comparator sites were more likely to get their antibiotic prescription within the recommended 4-hour time-frame than those in implementation sites. Nebulisers were supposed to be delivered within 1 hour and the data show that this did not occur often in either group, although compliance on this bundle element was more common in the comparator sites. Few patients were reported to have had a review with a respiratory specialist, although this was twice as common in implementation sites as in comparator sites.

Using the same approach, we also considered the different elements of the discharge care bundle and examined how often these were delivered. Table 16 presents the results for both groups.

Reviews of respiratory medicines were performed on the majority of patients prior to discharge, although this was more common in implementation sites. Inhaler technique, however, was not assessed as often, although it was still more common at implementation sites.

To deal with future exacerbations, the discharge care bundle states that patients should be provided with both a written plan and an emergency pack of medications. This did not occur for most patients, although it was more common at implementation sites. Many of the patients studied never smoked or were ex-smokers and so did not require smoking cessation advice. When appropriate, however, it was discussed with the majority of patients at implementation sites. Nearly all patients were assessed in terms of their suitability for pulmonary rehabilitation, although this was more common at implementation sites. Community follow-up was provided for most patients, but was more common in implementation sites.

We found that only 18 patients (3.5% of those for whom non-missing data were available) did not receive any of the elements of an admission care bundle (4.9% in comparator sites and 2.2% in implementation sites) and only 30 patients (5.8%) received all five elements (3.7% in comparator sites and 7.6% in implementation sites). The average number of admission care bundle elements received was 2.2 (SD 1.1) in comparator sites and 2.6 (SD 1.1) in implementation sites.

As with the admission bundle, it was very rare for patients not to receive any of the discharge care bundle elements (11.1%). Receiving all elements of the discharge care bundle was not the norm but was more common in the implementation sites (28.3%) than in the comparator sites (0.8%). The average number of discharge care bundle elements delivered was 1.8 (SD 1.3) in the comparator sites and 2.8 (SD 1.7) in the implementation sites.

Classifying sites by their level of COPD care bundle implementation (i.e. mean number of admission/discharge bundles delivered) showed that only IMP05 delivered an average of more than three admission care bundle elements per patient. All other sites, including comparator sites, delivered a mean of > 1 and ≤ 3.

With regard to discharge care bundles, IMP02, IMP03 and IMP11 all had high levels of compliance, with an average of > 3 discharge bundle elements delivered per patient. All other sites delivered 1–3 elements per patient, except COMP04, which had delivered < 1.

There were no clear trends in the proportion of patients receiving all bundle elements according to the month of admission subsequent to the index date. This was the case for both the admission and the discharge bundles.

Level 1 sites

Level 2 sites

Level 1

A total of 30 level 1 sites provided data, of which 11 were comparator sites and 19 were implementation sites. Mean costs per month were higher at implementation sites both before and after the introduction of care bundles at implementation sites (Table 18 and Figure 2).

FIGURE 2.

Costs of admissions and re-admissions at 90 days. (a) Comparator sites; and (b) implementation sites.

The difference between costs at implementation sites minus costs at comparator sites in the ‘before’ period, and costs at implementation sites minus costs at comparator sites in the ‘after’ period provides a simple indication of how costs at implementation sites differed from comparator sites when account is taken of the introduction of care bundles. An equivalent formulation is to calculate the difference found from subtracting the pre-bundle costs at comparator sites from post-bundle costs at comparator sites and pre-bundle costs at implementation sites from post-bundle costs at implementation sites. For admissions only, this difference in differences is £155, for admissions plus 28-day re-admissions it is £602, and for admissions plus 90-day re-admissions it is –£241. This indicates that there were no large differences between sites and increases in post-implementation costs were slightly higher at implementation sites for two of the three measurement points.

Figure 2 shows monthly mean level 1 costs, calculated as the cost of admissions plus re-admissions at 90 days, at implementation and comparator sites. Month 13, indicated by the vertical bars, contains the index date for the introduction of care bundles and, therefore, defines the ‘before’ and ‘after’ periods. Figure 2 illustrates the inter-month volatility in costs that appears to be associated with seasonal admission patterns, and indicates the higher mean costs at implementation sites both before and after the introduction of care bundles.

Implications of level 1 trust-level cost-effectiveness results

These high-level data indicate an absence of any obvious divergent trend in costs between implementation and comparator sites. Mean costs were higher at implementation sites at baseline (month zero), following the introduction of care bundles (month 13), and when measured as the difference-in-differences between each site type for admissions only, for admissions plus 28-day re-admissions, but not for admissions plus 90-day re-admissions.

This level 1 descriptive analysis is limited as a high-level, trust-based analysis with no adjustment for potential confounding factors. Adjusted, patient-level analysis is the subject of the level 2 analysis presented in the next section.

Level 2

Results of cost-effectiveness analysis

Table 25 presents results for the unadjusted and adjusted models estimated using net benefit regression. Net benefit is observed to decline with increasing values of the threshold. These results are similar to those of the SUR models reported in Appendix 2.

TABLE 25 - Cost-effectiveness results – available case analysis using net benefit regression

λ, cost-effectiveness threshold value; ICER, incremental cost-effectiveness ratio; CE, cost-effectiveness.

Note that for net benefit regression, the interaction measures net benefit and is reported above for a threshold value of £20,000.

Threshold values represent cost per death avoided at 90 days.

Interactions measure the difference in outcomes (either cost or 90-day survival) between outcomes at implementation sites minus outcomes at comparator sites in the ‘pre’ period and outcomes at implementation sites minus outcomes at comparator sites in the ‘after’ period.

Models estimated	Net benefit regression, unadjusted (n = 8553)			Net benefit regression, adjusted for month in year and mixed effect for trust cluster (n = 8553)			Net benefit regression, adjusted for month in year, mixed effect for trust cluster, and all baseline covariates (age, sex, ethnicity and deprivation)(N = 8121)
	Comparator mean	Implementation mean	Interaction (95% CI)a	Comparator mean	Implementation mean	Interaction (95% CI)b	Comparator mean	Implementation mean	Interaction (95% CI)a
Net monetary benefit at λ = £20,000b
Monetary benefit in ‘pre’ period	£11,926	£11,242	£884 (£117 to £1650)	£11,826	£11,361	£764 (£2 to £1527)	£11,846	£11,336	£798 (£15 to £1581)
Monetary benefit in ‘post’ period	£13,411	£13,611		£13,400	£13,599		£13,396	£13,684
Cost-effectiveness statisticsc
NMB at λ = £5000 (95% CI)			£792 (£190 to £1395)			£699 (£99 to £1300)			£784 (£167 to £1402)
Probability cost-effective at λ = £5000			1.00			0.99			0.99
NMB at λ = £10,000 (95% CI)			£823 (£186 to £1460)			£721 (£87 to £1355)			£789 (£137 to £1441)
Probability cost-effective at λ = £10,000			0.99			0.99			0.99
NMB at λ = £30,000 (95% CI)			£945 (–£1 to £1891)			£809 (–£133 to £1751)			£808 (–£158 to £1774)
Probability cost-effective at λ = £30,000			0.97			0.95			0.95
NMB at λ = £50,000 (95% CI)			£1067 (–£309 to £2442)			£899 (–£473 to £2271)			£829 (–£576 to £2234)
Probability cost-effective at λ = £50,000			0.94			0.90			0.88

The available case analyses for all model specifications (see also the SUR and GLM models reported in Appendix 2) are broadly similar: the probability of cost-effectiveness declines with increases in the cost-effectiveness threshold (Figure 3). This is because care bundles are estimated to be less expensive and very slightly less ‘effective’ (where effect is measured using 90-day survival), once adjustments are made for month-in-year, site and baseline covariates. Unadjusted 90-day mortality was slightly better at implementation than at comparator sites. Given the emphasis on net benefit regression, evaluation of model fit and the use of penalised likelihood criteria played a less important role in model selection than originally anticipated. A plot of residual versus fitted plots from the available fully case-adjusted, net benefit regression model is presented in Appendix 2. Both the AIC and the BIC were slightly lower under the fully adjusted net benefit models (AIC = 170,741; BIC = 170,909) than under the fully adjusted SUR models (AIC = 169,899; BIC = 170,261).

FIGURE 3.

The CEACs for available case net benefit models.

We also ran two placebo regressions that estimated SUR and net benefit models on fully adjusted available case data. In neither model were coefficients on interactions between site type and time period (recoded to mimic the effect of care bundles being introduced half-way through the pre-year, rather than at the end of that year) significant. Moreover, estimated cost-effectiveness measured using net benefit was smaller than in the base case [net benefit model at a threshold of £20,000 = £233 (95% CI –£845 to £1311) compared with £798 (95% CI £15 to £1581) in the fully adjusted base case]. This offers a degree of reassurance that an ‘effect’ of care bundles on cost-effectiveness is not obvious if they are modelled as having been introduced 6 months before their actual introduction and is some evidence that the parallel trends assumption may be reasonable. However, the nature of placebo tests means that these results are necessarily suggestive rather than definitive.

Available case estimates are likely to be both biased (not least because of the exclusion of entire sites but also the exclusion of individual patient records in some cases) and inefficient (by excluding responses from over 31% of all individuals included in the sample). Table 26 presents results from the unadjusted, partially adjusted and adjusted models estimated with net benefit regression on imputed data.

TABLE 26 - Cost-effectiveness results – imputed cases using net benefit regression

λ, cost-effectiveness threshold value; CE, cost-effectiveness.

Threshold values represent cost per death avoided at 90 days after the index admission.

Models estimated	Net benefit regression, unadjusted model (n = 12,532), imputed observations			Net benefit regression, adjusted for month in year and trust site as a mixed effect (n = 12,532), imputed observations			Net benefit regression, adjusted for month in year, trust site as a covariate and baseline variables (n = 12,532), imputed observations
	Comparator mean	Implementation mean	Interaction (95% CI)	Comparator mean	Implementation mean	Interaction (95% CI)	Comparator mean	Implementation mean	Interaction (95% CI)a
Monetary benefit and net monetary benefit at λ = £20,000
Monetary benefit in ‘pre’ period	£10,580	£12,270	£282 (–£524 to £1088)	£9942	£12,380	–£1099 (–£1902 to –£296)	£9997	£12,356	–£1089 (–£1891 to –£297)
Monetary benefit in ‘post’ period	£12,148	£14,119		£12,867	£14,206		£12,898	£14,168
Cost-effectiveness statisticsa
NMB at λ = £5000			£263 (–£452 to £977)			–£1019 (–£1773 to –£305)			–£1013 (–£1727 to –£299)
Probability cost-effective at λ = £5000			0.76			0.00			0.00
NMB at λ = £10,000			£269 (–£464 to £1002)			–£1046 (–£1778 to –£315)			–£1039 (–£1770 to –£308)
Probability cost-effective at λ = £10,000			0.75			0.00			0.00
NMB at λ = £30,000			£294 (–£622 to £1210)			–£1150 (–£2063 to –£237)			–£1137 (–£2048 to –£227)
Probability cost-effective at λ = £30,000			0.74			0.01			0.01
NMB at λ = £50,000			£319 (–£884 to £1523)			–£1249 (–£2452 to –£45)			–£1231 (–£2428 to –£35)
Probability cost-effective at λ = £50,000			0.70			0.02			0.02

Imputed regressions again exhibit a pattern of the probability of cost-effectiveness declining with increases in the cost-effectiveness threshold. Figure 4 shows the CEAC for the unadjusted model only, as the CEACs for the adjusted models are indistinguishable from the horizontal axis.

FIGURE 4.

The CEAC for unadjusted imputed net benefit regression model. The CEAC is presented for the unadjusted model only, CEACs for partially and fully adjusted imputed models are practically indistinguishable from the horizontal axis.

The probabilities of care bundles being cost-effective are drastically lower under the unadjusted imputed models than in any of the unadjusted available case models. This suggests that omission of entire sites and individual patient responses imparted some bias to the available case analysis, assuming that the imputation model has not itself introduced further bias.

The probability of cost-effectiveness attenuates once adjustments are made for time-of-year effects, site mixed effects and baseline covariates. Again, assuming that the imputation model is less biased than available case models, this suggests that, once all sites are included in the analysis and adjustment is made for available covariates, there is little probability of care bundles being cost-effective at any cost-effectiveness threshold. These results, therefore, suggest that the available case analyses, and unadjusted imputed analyses, may be biased by both (1) the exclusion of missing observations in the case of the former and (2) the exclusion of covariates in the case of the latter. A sensitivity analysis excluding outpatient costs from the fully adjusted, imputed analysis was undertaken. This reduced the estimated probability of care bundles being cost-effective but overall was similar to the base-case analysis.

Implications of level 2 patient-level cost-effectiveness analysis

A large sample of patient-level data was available to the level 2 cost-effectiveness analysis. This detailed analysis complemented the hospital-level comparison of costs at level 1. The introduction of care bundles was associated with lower costs, but did not appear to improve the primary outcome, following the index admission, used to measure ‘effect’ in the cost-effectiveness analysis once adjustments were made for site and baseline covariates.

Elements of care bundles

The results of the descriptive comparison between site types in the costs of elements of care bundles are reported in Table 27. Note that the sum of costs reported is on an element-by-element available case base, and so formal inference is not appropriate because the number of included individuals differs between elements. Appendix 2 provides further detail on element-by-element costs.

Most costs are attributable to the admission care bundle. It is notable that comparator sites incurred many of the costs associated with providing elements of care bundles, although these costs are lower than at implementation sites. This is consistent with the evidence produced elsewhere (i.e. Chapters 5 and 7) that comparator sites engaged in many of the ‘bundle’ activities undertaken by implementation sites.

Results of level 3 analysis

Implications of level 3 patient observation and interviews on cost-effectiveness analysis

The small sample of individuals observed and interviewed as part of the level 3 work constrains the inferences that may be drawn about resource use by this group of COPD patients. Overall, there are no grounds for concluding that there is evidence of gross differences in doctor or other clinician engagement between implementation and comparator sites, or in relation to the intensity of resource use post discharge. Scrutiny of the experiences of larger samples of patients, ideally following randomisation to types of care bundle, would be necessary to clarify the relationship between clinician engagement during and after hospitalisation.

Summary

The value of in-depth qualitative research to examine quality and safety initiatives is well recognised, and the merits of interviews and non-participant observation for providing detailed insight into the practicalities of implementation of improvement initiatives have been demonstrated. 43 Drawing on such previous work, we sought to understand the everyday practice of staff implementing care bundles and providing usual care, and to gain detailed insight into staff and patient narratives associated with COPD care across multiple case study sites.

Of the 36 acute hospitals initially recruited to the study, six were purposively selected as case study sites. Following Yin,83 a case study approach enabled in-depth empirical inquiry of the contextual factors and circumstances of care bundle delivery. Qualitative methods used were non-participant observation of patient care within relevant settings within the case study hospitals; interviews with patients, carers and health-care professionals in the hospital sites; and patient, carer and health professional interviews in the community following patient discharge. A summary of the level 3 objectives and the data collected to achieve those objectives is provided in Table 41, Appendix 6.

Case study selection

For the case studies, four implementation sites (i.e. sites using COPD care bundles) and two comparator sites (i.e. sites providing ‘usual care’) were selected, enabling comparison both within and between sites. All sites were located in England. Details of the key characteristics of each case study site are presented in Table 42, Appendix 6.

Implementation sites were oversampled, as they were expected to provide a richer source of information about care bundle implementation. Comparator sites were selected to enable some comparison with ‘usual care’. Implementation sites were purposively sampled based on how established care bundles were, including the selection of a long-term ‘established’ care bundle implementation site and the selection of an ‘early’ implementation site which had recently introduced care bundles. This enabled evaluation of multiple approaches to, and stages of, care bundle implementation.

Of the four implementation sites selected, three were delivering both admission and discharge care bundles and one site was delivering the discharge bundle only. All had QI measures in place to varying degrees. Two of the sites had used Commissioning for Quality and Innovation (CQUINs) payment frameworks as part of QI to COPD care, and one site had a CQUIN in place for the delivery of the discharge care bundle during the data collection period. Sites were also purposively sampled on their location, with a balance of inner-city and suburban sites sampled to reflect the varying demands and population characteristics of differentially located hospitals.

Data collection

Across the six case study sites, data collection was conducted by two qualitative researchers using non-participant observation and interviews. Each researcher led data collection at a number of sites, following an agreed study protocol and using approved study documentation. In order to ensure a consistent approach to data collection, the researchers met on a weekly basis to share experiences of data collection and discuss any issues with a senior colleague. Between 8 and 10 days of observation were carried out at each case study site, covering a range of times of the day and days of the week. All activity was conducted Monday to Friday between 07.00 and 19.00. Figure 5 presents a summary of the qualitative data collection process at implementation and comparator sites.

FIGURE 5.

Schematic showing process of qualitative data collection.

Interviews

Semistructured interviews were carried out with patients, carers and acute and community staff, using a series of REC-approved topic guides. The aim of the interviews at implementation sites was to evaluate the process of COPD care bundle delivery, capture the patient and staff experience of care bundles, and scope the local context of care bundle delivery. Interviews were also carried out in comparator sites to assess similar dimensions of usual care. Interviews with patients, carers and acute staff were carried out face to face at the case study sites, and interviews with community staff and follow-up interviews with patients and carers were conducted over the telephone. Information sheets were provided to staff, patients and carers, and written informed consent was obtained for all interviews according to GCP guidelines. 65

The lead qualitative researcher at each site was responsible for the recruitment of staff, patient and carer interviewees during the period of non-participant observation. This was supported by the local PI, who was usually a respiratory consultant or specialist respiratory nurse, who identified prospective participants. A total of 114 interviews were carried out with 105 participants. A breakdown of the interviews completed at each case study site is available in Table 28.

Data analysis

Analysis took place alongside data collection, in order to allow the incorporation of insights into the ongoing data collection process. 85 Using qualitative data from multiple sources enabled us to build up ‘thick descriptions’63 of each case study site, and facilitated examination of divergence and overlap between the data sets.

There were two dimensions to the data analysis. Drawing on case study methods described by Yin,83 the data were analysed to provide detailed case study descriptions of care bundle implementation. Using thematic approaches, such as those described in Braun and Clarke,86 cross-case thematic analysis was carried out, to identify recurring issues across cases and to inform interpretation of the quantitative findings from other levels of the study. Analysis of relevant local and national documents associated with COPD care, care bundle implementation and QI also provided more in-depth understandings of the context for delivery of COPD care.

All qualitative members of the project team were involved in data analysis, including reading transcripts. Preliminary findings were shared with the PPI panel and the Study Steering Committee (SSC) for feedback. The lead qualitative researchers for a particular case study site developed detailed case study descriptions of that site, drawing on the observations, field notes and interviews. For the thematic analysis, both researchers conducted a first round of coding, aided by the software NVivo (version 10). This led to the development of a detailed coding framework that incorporated both deductive (i.e. anticipated) and inductive (i.e. emergent) codes. This coding framework was then flexibly applied to all the qualitative data by the researchers, with a senior colleague checking a subset of the coding for coherence and completeness. Field notes taken during observation were also coded alongside interview transcripts, and developing themes and concepts were discussed and refined at regular team meetings throughout the research.

The concepts of barriers to, and facilitators of, COPD care bundle implementation, as described in the study protocol,51 were used to structure the thematic analysis, alongside theoretical frameworks of QI devised for use in the NHS. 87 These theoretical frameworks informed the analysis in terms of broader NHS strategies associated with QI, situating the case study analysis within the policy environment.

Analysis across the implementation case studies enabled the identification of factors that seemed to be consistently related to the successful (or otherwise) delivery of the care bundle components. It focused particularly on where, how and why the implementation of the intervention had, or had not, worked. Analysis of the challenges associated with the implementation of care bundles was also informed by Institute for Healthcare Improvement competency frameworks which, following Lennox et al. ,42 were used to assess the ways in which such challenges are associated with the knowledge and skills of the staff implementing the care bundles.

Summary

First, the context of care bundle delivery will be presented, drawing on the case study descriptions, and relevant local and national documents. Second, the findings of the cross-case thematic analysis will be presented, focusing on the barriers to, and facilitators of, care bundle implementation, comparisons with usual care and the patient/carer experience.

Context of care bundle delivery in case studies

Case study descriptions were compiled for each of the six case study sites. These descriptions were compiled using interview data, observational data and background information gathered from site recruitment and set-up. Brief summaries of the case studies are included below.

Themes across case studies

In this next section, we present the findings of the cross-case thematic analysis, focusing on barriers to, and facilitators of, care bundle implementation, comparisons with usual care, and patient and care experience of care bundles for COPD.

Strengths and limitations

There are a number of strengths to the level 3 qualitative case studies. As part of a mixed-methods evaluation, multiple qualitative methods were used in a range of hospital case study sites, with a diverse sample of participants including acute and community staff, carers and patients. Interview and observational methods enabled exploration both of perceptions of care bundle implementation and the actual practice of implementation. This has given an understanding of the overlaps and divergence between what participants say they do and what they actually do. As the majority of data collection took place in an acute setting (during the admission of a patient), the case studies provide ‘in the moment’ insights into patient care rather than relying solely on retrospective accounts of care bundle implementation.

Building in a longitudinal design to the patient interviews, from inpatient care to post-discharge follow-up, is also a strength, as it provides detail about longer-term patient experience and gives insight into the role that care bundles might play in securing effective and timely follow-up care for patients. Despite the relatively small number of follow-up interviews undertaken, the data provided rich descriptions of follow-up care received via GPs and community teams, and gave an understanding of issues (such as self-management at home) as well as detailed reflections on patients’ admission experiences.

In terms of limitations, it was clear that acute hospitals proved a challenging environment in which to recruit study participants and gather data. The primary challenge was working with very unwell patients, who were often unable to give detailed accounts of their experiences. However, the qualitative data set has successfully captured a variety of observations and experiences of acute care for this specific patient group.

A key limitation is the relatively small number of follow-up interviews with patients and carers because a high proportion of those people who gave an interview following admission had died, no longer had capacity to be involved in the study, had been re-admitted, were not contactable or did not consent to a follow-up. This clearly demonstrates the challenge of working with a patient population who have a complex and unpredictable disease associated with frequent hospital admissions and social isolation.

An additional limitation is that non-clinical service managers, commissioners and trust representatives were not interviewed as part of the study. Interviews with these service managers could have offered more insight into the strategic decision-making around QI initiatives and the governance and funding structures in which they are embedded. However, this was addressed informally during some interviews with senior clinicians and poses an opportunity for future research.

We included only six hospital sites in England in level 3 data collection and restricted the hours of working to weekdays between 07.00 and 19.00. Carrying out observations at the weekend, overnight, over a longer period of time or at other sites may have produced different findings. Furthermore, because all of the level 3 data were collected between the months of March and October, we may have missed the period of highest admissions as there is a seasonality to COPD exacerbations. There is also a temporality to the implementation of QI practices, and the majority of this data set provides only a snapshot of these processes. Access to more longitudinal data would provide better insights into the life cycle of QI and care bundle implementation.

In addition, the hospitals that enrolled as case study sites were those that welcomed researchers to observe their patient care. It is possible, therefore, that this self-selecting group felt that they were performing well enough to survive scrutiny. Sites with concerns about the care they were providing, or their approach to the implementation of care bundles, may not have been willing to be involved in the study.

There was also a reliance on clinical staff to act as gatekeepers to patients, carers and observational encounters. As a result, it is possible that the clinical interactions observed by the researchers were ones in which staff were most comfortable, and that the quality of these interactions may have varied considerably outside the sample of staff actually observed.

Overall, there are a number of key messages from the level 3 data:

• Staff perceptions of care bundles were largely positive as a way of standardising working practices and patient care, supporting a clear care pathway for patients, facilitating communication between different teams and individuals responsible for patient care and identifying necessary support required by patients following discharge.

• Embedding reliable, sustainable QI required managerial support, resourcing and regular education and training. Monitoring was also necessary to measure the effectiveness of implementation.

• Greater attention appeared to be focused on the discharge bundle, while the admission process is more complex and was not necessarily in the hands of the respiratory team, making it more difficult to implement and monitor.

• Both patients and their carers require support during admission and following discharge. Community services are an important part of avoiding re-admission, although pressure on these services means that patients can be waiting a long time to access them.

• Communication between multidisciplinary teams involved in COPD patient care can be facilitated by the use of a care bundle, but can also be facilitated by monthly meetings, a thorough discharge checklist and effective follow-up.

• Organisational pressures around patient numbers, resources and staffing mean that it is not always possible for patients to receive the quality and amount of care that health-care professionals would like, particularly in relation to follow-up.

Barriers to care bundle implementation

The reason why levels of implementation of care bundles were poor is likely to be multifactorial. It reflects the challenge of making changes to care in the NHS in the face of increasing operational pressures and against a background of problems with staffing and health-care resource. The most frequent comment made by staff interviewed as part of this study was that the under-resourcing of health and social care made care bundle implementation difficult. Staff felt that they were ‘firefighting’ much of the time, and as a result had less time for QI initiatives. In addition, staffing of hospital emergency zones is dynamic and, consequently, there are problems maintaining adequate levels of knowledge and skill among staff to ensure sustainable change in the delivery of patient care.

Implementation of complex interventions rarely happens on a single date, so using a single date to define ‘before’ and ‘after’ data is likely to significantly underestimate the eventual impact, as a new way of working can take weeks or months to embed. Implementation sites sometimes found it difficult to define an index date as implementation was gradual. It might mean in some cases that the pre-index date period included limited bundling and, therefore, diluting the before-and-after differences. However, using the data collected from case notes extracted after the index date, no clear trends were found in the proportion of patients receiving all bundle elements according to the month of admission since index date. This was the case for both the admission and the discharge bundles.

It was clear that patients identified at the front door of hospitals were often incorrectly diagnosed. Staff commented that, if a patient was admitted with a chest infection and had smoked at some point in their life, then the assumption was that this was an infective exacerbation of COPD, even when there was no evidence of confirmation of the diagnosis by spirometry. This misdiagnosis of AECOPD had a negative effect on the workload of staff responsible for implementing care bundles, as their time was frequently taken up attending to patients who turned out not to have COPD. This, inevitably, had an impact on the amount of time available for them to deal with patients who might have benefited most from staff time. However, staff also highlighted the fact that screening of patients for COPD care bundles permitted them to identify people for whom there was diagnostic uncertainty and this, in turn, could be flagged to appropriate members of the medical team.

Facilitators of care bundle implementation

Two of the sites selected as case studies revealed that they had previously received CQUIN payments associated with care bundle implementation. Although these payment frameworks were no longer in place at the time of data collection, staff clearly felt that the systems developed to monitor and deliver the now-defunct CQUIN were still paying dividends in terms of care bundle implementation and had contributed to their longer-term sustainability.

Challenges for implementing and demonstrating improvement in health-care settings

A key issue in the literature on quality and safety in health care is that many improvement initiatives fail to exceed the overall ‘rising tide’ and so have difficulty demonstrating added value. 43

Context is important and is likely to have an impact on the implementation and the likelihood of the intervention ‘outpacing the secular trend’; both the national context into which it was introduced and the local context/history of individual settings where implementation has taken place. It is crucial to understand the extent to which an intervention or programme is different from what is already provided – is it novel or is it a ‘bundling’ of existing procedures? Settings may already have been significantly exposed to the ‘ingredients’ in a bundle of procedures prior to the implementation of the ‘intervention’. Participants in a programme or intervention may give accounts of an incremental history of improving practice, happening before and during the implementation of the intervention. Substantial improvement may be occurring outside the programme or intervention throughout the period that it was running.

The language around QI and implementation of care bundles implies that centres introducing care bundles are by nature ‘dynamic’, whereas centres providing usual care are ‘static’. This was found not to be the case. Both implementation and comparator sites are functioning within the same political and clinical context.

There are a number of national initiatives around improving care of patients at the point of entry to hospital, and an ongoing emphasis on the needs of patients with COPD through the Royal College of Physicians, National COPD Audit, which is running in all acute trusts in the UK. Thus, even in the comparator sites, changes were being made to care to improve the way in which it was delivered.

In fact, in two of the comparator sites that had chosen not to use ‘care bundles’, patients were reviewed by specialist staff at the point of discharge with ‘discharge checklists’, which incorporated all the elements of the BTS care bundle. This is ‘care bundling by another name’ and is likely to have reduced the size of any difference in outcomes between implementation and comparator sites. In one comparator site, a checklist approach to support patient discharge had been selected as a more informal way of introducing the initiatives described in the care bundle. This was because conflict between respiratory, ED and acute medical admission staff meant that it was not acceptable locally to introduce a care bundle, as staff did not wish to participate in QI activities but, nevertheless, the respiratory specialist driving the programme wished to standardise care and chose this more informal method of achieving the same end.

The national context may have an impact on how participants view an intervention. If there have previously been ‘top-down’ drivers towards implementation of particular procedures, there might be resistance to what are perceived as similarly ‘top-down’ subsequent initiatives within local contexts. If something is perceived as being imposed from ‘outside’, then there may be a lack of professional ownership and engagement. Settings may feel suspicious about the potential for data to be used for performance management and ‘shaming’.

There may be variations in local responses to a programme or intervention, in terms of the extent to which staff within settings attribute their behaviour to the intervention. In the ‘Matching Michigan’ study,43 there were three types of response:

1. transformed – as the intervention was seen to have introduced radical improvement in care

2. boosted – as the intervention was seen to have reinforced existing good practice or supported further improvements

3. low impact – as staff behaviours were not attributed to the intervention but to other influences.

In transformed settings, an intervention may be viewed by staff locally as having provided the tools they needed to make change where prior change efforts had failed. In boosted settings, a programme or intervention may be absorbed into a local narrative of cumulative improvement. In low-impact settings, staff may be unsure of the ‘added value’ of the intervention and may be wearied by previous change initiatives that are seen to have failed.

Senior figures may be important in influencing responses to programmes and interventions. ‘Compliance’ with an intervention may be shaped by which senior staff are present and the extent to which they have ‘bought into’ the intervention. The commitment, characteristics and skills of local leads can be pivotal. Transforming or boosting of efforts is most likely to occur when those locally responsible for leading implementation believe in the value of the programme or intervention and use multiple tactics to facilitate others’ engagement. If local leads are sceptical or resistant, then implementation is more likely to falter or fail.

Staff interviewed as part of the case study work highlighted that there was a benefit to be obtained from strong clinical leadership around the implementation of care bundles. This motivated staff and focused them on care bundle implementation. However, where leadership was concentrated in the hands of a single individual, there were also disadvantages. The need to pass decisions by a single person often led to delays in bundle implementation, and in some of the sites conflict between individual clinicians delayed bundle implementation.

Patient experience associated with the use of care bundles

Patients reported that they found hospital admissions frightening and disorientating. During interviews, patients were sometimes unable to remember details of the early part of their admission. Consequently, they were not able to provide a considered judgement on the benefits of the admission bundles.

Patients did, however, report a benefit from receiving a discharge bundle. At the point of discharge many were anxious about their medications and confused about how to use their inhalers. Staff administering care bundles spent between 15 and 60 minutes with the patients, depending on the operational context at the time the patient was being discharged. Discharge bundles were generally delivered at the bedside by a single staff member. Patients appreciated this as it allowed them to question a knowledgeable staff member about their medication regime and use of inhalers. The qualitative researchers noted that inhalers were frequently changed at the time the discharge because patients were found to be unable to use their inhalers effectively.

Benefits of care bundles as perceived by staff

Generally speaking, staff perceptions of care bundles were positive. They were felt to enhance patient care and patient outcomes. Staff highlighted that they perceived that care bundles assisted in the standardisation of patient care and facilitated improved clinical practice, enabling staff to take ownership of different aspects of the patient’s care.

The need to arrange follow-up in the community was also felt to be beneficial as it promoted better communication with community services. GP interview revealed that a frequent problem is a lack of timely information on a patient’s clinical condition following discharge from hospital. They appreciated the contact from the specialist team to arrange follow-up in the community at the point of discharge, and felt that on some occasions this supported the primary care team in managing patients’ discharge more effectively, with the overall aim of keeping patients in their homes and reducing the risk of re-admission.

Staff also reported that the introduction of care bundles had significantly improved uptake of smoking cessation and pulmonary rehabilitation courses. This is likely to lead to long-term benefits in terms of improved health and quality of life. However, it has also created problems. In one hospital where there had been a 30–70% increase in referrals to pulmonary rehabilitation, waiting lists had occurred for the first time and had reached 3 months in length. Owing to financial constraints within the health service, funding to expand the pulmonary rehabilitation course was not available and there was concern that, ultimately, the presence of a waiting list would discourage patients from completing the course.

Strengths

These analyses benefit from the collection of data from a wide range of hospitals across England and Wales, as well as the collection of detailed case note information allowing for the assessment of care bundle delivery in the real-world setting.

As part of the mixed-methods evaluation, multiple qualitative methods were used in a range of hospital case study sites, with a diverse sample of relevant participants, including acute and community staff, carers and patients. The study has drawn on interview and observational methods in the context of an acute medical admission to explore both perceptions of care bundle implementation and the actual practice of implementation. The data provide ‘in the moment’ insights into patient care rather than relying solely on retrospective accounts of care bundle implementation. The longitudinal nature of follow-up of patients during admission, discharge and afterwards provided rich descriptions of follow-up care received via GPs and community teams and provided insights into self-management as well as reflections on a patient’s admission.

The acute admissions setting was a challenging environment in which to recruit participants and gather data. Patients were often very unwell and unable to give detailed accounts of their experiences. However, the fact that data were collected in this way offered insights into the complexity of delivery of COPD patient care. The mixed-methods approach helps to illustrate not just what has happened to patients during their care pathway, but also why their care is delivered in a certain way. These data offer the opportunity to better understand the challenges of care bundle delivery and more widely of similar QI initiatives in the NHS.

There are also strengths in the methodology applied in the quantitative data analyses. Large-scale studies in the NHS generally rely on Hospital Episode Statistics. This is a data set that is collected mainly for costing and planning health care and it does not always reliably reflect outcomes of health care. In our study, we also have the benefit of patient-level data, abstracted from a randomly selected set of patient notes. This has allowed us to corroborate our findings derived from the larger level 2 patient-level data set and the level 1 aggregate data set covering a wider range of sites. The patient-level data obtained in level 2 data extraction allow us to estimate the extent to which care bundles were implemented in practice rather than relying on clinician estimates of the number of bundles delivered.

Limitations

Owing to the implementation of care bundles in hospitals in England prior to the start of this study, it was not possible to undertake a randomised controlled trial to evaluate the impact of care bundles. Although we have adjusted for available confounders in our models, and imputed missing data where appropriate, the observational nature of the study design means that there is still a degree of residual confounding caused by factors not accounted for in the quantitative analysis, and this cannot be discounted as a source of any observed differences between sites.

Sites were recruited to the study by a number of means, including publicity materials disseminated through the BTS and via the NIHR CRN. Some sites had been part of an earlier pilot of care bundles undertaken by the BTS. The main form of contact was an invitation to a senior respiratory clinician to participate. Given the nature of this recruitment, it is possible that the sites that participated in this study, whether implementation or comparator sites, were those where the respiratory team were interested in care bundles or in research into care for patients with COPD. Therefore, recruited sites, teams and individuals may well have already been among the ‘better’ performers as regards delivery of COPD care or QI programmes. This could have introduced an element of recruitment bias, and it is possible that the apparent lack of differences between outcomes of the two groups reflects this selection of ‘enthusiast-led’ sites. In addition, if the sites and teams were among the better performers, then there may be an element of regression to the mean, which would tend to attenuate the effects of care bundles.

As this was an observational study, staff at the sites were free to decide for themselves whether or not to implement the COPD care bundles and, if so, which elements and when. These decisions will have been made based on local circumstances and, occasionally, site plans would change after agreement to take part in the study. For example, during the study recruitment stage, sites occasionally moved from level 1 to 2 (or vice versa) or staff changed their minds as to whether or not to implement COPD care bundles as originally stated. These changes made it impossible to match level 2 implementation sites to comparator sites as originally planned. We were, therefore, confined to using all available level 2 sites, regardless of whether they might be considered closely matched to their comparators.

In assessing the impact of a complex change in clinical practice, it can be difficult to accurately define an index date as changes can be rolled out slowly. This approach (to use as index date the date on which bundles were fully implemented) attempts to address this. However, it is likely that elements of the intervention were being delivered in the pre-index date period.

Collecting data directly from several NHS trusts was not straightforward. Although there were high levels of engagement from many at the sites, data extraction from hospital systems was a task generally delegated to NHS team members with many other competing obligations. Thus, retrieving data took much longer than anticipated, as did resolving queries on the data.

These delays also meant that, in many cases, we had to use available data rather than request further data to improve completeness.

Finally, in collecting information from multiple sites and using a variety of local staff, we cannot exclude the possibility that study data were not collected in the same manner at each site. Guidance was provided to sites and all queries were responded to, but there may still have been variations in hospital-level practices in recording and extracting data.

In the analysis of resource utilisation, secondary care costs and cost-effectiveness, the level 1 descriptive analysis is limited as it was a high-level trust-based analysis with no adjustment for potential confounding factors. A limitation of the more detailed level 2 analysis was that a number of the planned models could not be estimated, either because of sparsity in models with many indicator variables or because of convergence issues when estimating mixed-effects models. It was difficult to say which model definitively served as the best representation of the relationship between care bundles and the cost and survival outcomes. However, those models that were estimated produced reasonably similar results.

The small sample of individuals observed and interviewed as part of the level 3 qualitative work constrain the inferences that may be drawn about resource use by this group of COPD patients. Overall, there are no grounds for concluding that there is evidence of gross differences in clinician engagement between implementation and comparator sites, or in relation to intensity of resource use post discharge. Scrutiny of the experiences of larger samples of patients, ideally following randomisation to types of care bundle, would be necessary to clarify the relationships between clinician engagement during and subsequent to hospitalisation.

There are drawbacks in terms of generalisability associated with the limited number of sites included at level 3. Carrying out observations at different times of the year, the weekend, overnight, over a longer time period and at other (or more) sites may have produced different findings. There were also a relatively small number of patient follow-up interviews.

Finally, the relatively short duration of the study limits our ability to draw a conclusion about whether or not bundles might have been more effective at improving patient outcomes if they had been more reliably implemented. Given more time, it is possible that implementation sites might have reached a point where 95% of patients received an appropriate care bundle. This is considered the minimum level of compliance with a particular action where its implementation could be deemed reliable.