A randomised 2 x 2 trial of community versus hospital pulmonary rehabilitation for chronic obstructive pulmonary disease followed by telephone or conventional follow-up

Article history

The research reported in this issue of the journal was commissioned by the HTA programme as project number 01/15/10. The contractual start date was in November 2002. The draft report began editorial review in May 2008 and was accepted for publication in June 2009. As the funder, by devising a commissioning brief, the HTA programme specified the research question and study design. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

None

Copyright statement

© 2010 Queen’s Printer and Controller of HMSO. This journal may be freely reproduced for the purposes of private research and study and may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2009 Queen’s Printer and Controller of HMSO

Background to the study

Chronic obstructive pulmonary disease (COPD) is a common condition imposing significant burdens on large numbers of patients, with important implications for health-care providers from a clinical and cost point of view. 1 Pulmonary damage is largely irreversible so pharmacological treatment is necessarily limited in its efficacy. As a consequence of breathlessness, people with COPD tend to limit their exercise and become deconditioned. It is now firmly established that a pulmonary rehabilitation programme of structured exercise (usually accompanied by an educative programme) yields significant health benefits. A Cochrane Review concluded in 2006 that ‘Rehabilitation relieves dyspnoea and fatigue, improves emotional function and enhances patients’ sense of control over their condition. These improvements are moderately large and clinically significant.’2

Early studies of pulmonary rehabilitation sometimes involved very intensive programmes, for instance a Canadian study used an 8-week inpatient programme followed by a 16-week supervised outpatient programme. 3 Subsequent trials have taken place in a variety of settings: hospital inpatient, hospital outpatient and community. The term ‘community’ may refer to group rehabilitation of a similar nature to that carried out in a hospital, but it is also used to describe home care, with one-to-one supervised exercise. Home rehabilitation may well be useful for very disabled patients who are unable to access services outside the home, but it is likely to be labour intensive and expensive, limiting its utility to a minority of those with significant impairment. Here we use the term ‘community rehabilitation’ to refer to supervised group rehabilitation in non-hospital venues.

Although there are undoubtedly great beneficial acute effects of pulmonary rehabilitation, these unfortunately decline at the end of the rehabilitation period. 4,5 The American Thoracic Society (ATS)/European Respiratory Society (ERS) statement on pulmonary rehabilitation notes that: ‘Pulmonary rehabilitation is a service that complies with the general definition of rehabilitation and achieves its therapeutic aims through a permanent alteration of lifestyle.’6 We and others wondered whether community rehabilitation would not only enhance adherence to exercise programmes by facilitating ease and convenience of participation, but also enhance its efficacy by linking it effectively to a lifestyle change rather than being a treatment that others apply in a hospital. 7,8

This theme of moving services to community settings where possible has been adopted as official policy in the UK. 9 Community programmes have begun to be increasingly established. A cautionary note is sounded by the ATS/ERS joint statement: ‘Properly conducted pulmonary rehabilitation offers clinical benefit in all settings that have so far been studied; however, few clinical trials offer direct comparison among various settings. The majority of studies describing the benefits of pulmonary rehabilitation are derived from hospital-based outpatient programs.’ In assessing the evidence base for community programmes, the National Institute for Health and Clinical Excellence (NICE) guidelines conclude, ‘The majority of studies have been performed in a hospital setting. There is limited data on effectiveness in community or home studies and there have been no comparative studies.’1

Aims of the study

It seemed timely therefore to examine further the evidence for the efficacy of pulmonary rehabilitation in a community setting acutely as well as chronically, and to compare the efficacy with that of a more traditional hospital-based approach.

As noted, the benefits of treatment decline with time after pulmonary rehabilitation has finished. This has led to a search for ways to prolong its benefit, including such intensive strategies as repeated courses of rehabilitation10 and weekly supervised rehabilitation. 11 Whilst these have shown promise, they are expensive and no data have been advanced to explore the cost-effectiveness of such approaches.

As we hypothesised that effecting a lifestyle change was key to producing lasting benefit, we wondered whether simple telephone encouragement without direct health-care contact would be beneficial.

From a UK perspective it was important to have a trial that examined service delivery within the NHS. As models of health-care delivery vary widely in other countries and hence will lead to different preconceptions and expectations, it is not certain that models examined elsewhere can be generalised to the UK system.

This study therefore undertook a 2 × 2 factorial trial of:

• pulmonary rehabilitation in hospital or community settings

• telephone follow-up or ‘standard follow-up’.

Acute effects of rehabilitation together with 18-month follow-up are included, with data on exercise capacity [using the endurance shuttle walking test (ESWT)] and quality of life (using generic and disease-specific tools). Cost–benefit analysis is also included.

Research question

At the first meeting with the Data Monitoring and Ethics Committee (DMEC) it was agreed that clarification of the hypothesis was indicated, particularly in relation to the specification of co-primary outcomes, and indeed the effect of this on the power calculation. It was noted that whilst there were data to enable the power calculation in relation to the initial changes in walking distance, it was not possible to predict the statistical variation relating to follow-up, and thus this aspect of the study should be excluded from the power calculation.

The redrafted hypotheses underlying the study were agreed and are:

1. The percentage change in the endurance shuttle walk post rehabilitation, compared with the baseline measurement, will be greater in the community-based than in the hospital-based group.

2. Community-based rehabilitation will increase the persistence of the changes observed in hypothesis 1.

3. Telephone follow-up will increase the persistence of the changes observed in hypothesis 1.

Changes to original protocol

Prior to the study we suggested that leisure centres be used for community rehabilitation, which would allow research participants to continue with programmes. This proved impossible as we had little co-operation from suitable leisure centres, and there was limited availability for continuation programmes on a city-wide basis.

In an ongoing community pulmonary rehabilitation programme in one suburb of Sheffield it had been possible to arrange transport with a voluntary body on a regular basis. This proved impossible to organise for our multicentre requirements, so we provided introductions to the voluntary body or provided personalised transport information if necessary from the local bus company. We felt that this was appropriate in view of the pragmatic nature of the intervention.

The self-completion version of the Chronic Respiratory Questionnaire (CRQ) was withdrawn immediately prior to the study, necessitating creation of our own self-completion version for all follow-up visits. The original interviewer-led questionnaire was used at visit 1, which is when subjects choose five items causing breathlessness that are most important to themselves. These items are then used in one section of the questionnaire at follow-up visits. The self-completed version (Appendix 2) adds a simple transcription of the potential responses from the interviewer-led questionnaire into a paper version. Research participants found it as easy to complete as the other questionnaires, and it facilitated collection of health-related quality of life (HRQoL) data by post if individuals refused to attend for evaluation at follow-up visits.

A planned follow-up visit for assessment 3 months post rehabilitation was never instigated as it may have acted as an impetus for adherence to the exercise regime in all patients, thus clouding the difference between usual follow-up and telephone encouragement groups.

The DMEC asked that the effect of the telephone encouragement on walking distance be promoted to a co-primary outcome together with the site of provision of pulmonary rehabilitation, and that the hypotheses be more clearly stated. Thus, whilst differences between the groups engendered by the site of rehabilitation might show a difference, we expected that any difference might be more marked at 18 months post rehabilitation.

At the first meeting of the DMEC they questioned the data used for power calculations and asked us to recalculate. This is addressed in detail in Sample size.

Recruitment was not as rapid as anticipated, and there were many dropouts between randomisation and the start of treatment. A short extension period for the study was granted.

At the time of the funding bid the incremental shuttle walking test (ISWT) was felt to be more appropriate than the 6-minute walking test for the primary outcome measurement, as it included an external source of pacing and was thus a better comparator for longitudinal testing. However, by the start of the study there was a new test available, the ESWT,12,13 which allows people to walk at a set pace equivalent to 85% of their maximal oxygen uptake, which should take approximately 5 minutes. Subsequent walks are at the same pace, and it is therefore much more appropriate for testing changes engendered by pulmonary rehabilitation as there is the possibility of a greater magnitude of change. It is possible to collect data on both distance and time walked. We used percentage change in ESWT distance as our primary outcome variable for the acute phase of the study but, for completeness, additionally reported absolute ESWT distance and time.

In some areas, pulmonary rehabilitation is not widely offered to people over the age of 70. We will make data gained in this study available to researchers in this field to allow further exploratory analysis of the data, and this will be reported elsewhere.

Ethics and governance

The study was approved by relevant local research ethics committees and by the research governance processes of the Sheffield Teaching Hospitals NHS Foundation Trust. An independent DMEC scrutinised trial design and conduct to ensure probity. Its members were Peter Calverley, Professor of Respiratory and Rehabilitation Medicine, University of Liverpool; Keith Abrams, Professor of Medical Statistics, University of Leicester; and Anthony Burton, a general practitioner in Doncaster.

Participants

People with COPD were recruited from respiratory clinics within the Sheffield Teaching Hospitals NHS Foundation Trust, community sources, physiotherapy clinics and self-referrals. All were seen by a respiratory physician to confirm the diagnosis of COPD according to the Global initiative for Chronic Obstructive Lung Disease (GOLD) criteria before being enrolled in the study, and to have their medication optimised. Informed consent was gained at the time of assessment for suitability for group pulmonary rehabilitation therapy by a study physiotherapist. This assessment, together with any quantitative measurement required by the study, was made in a hospital setting. This ensured that no bias was introduced.

The database

A bespoke database was held on a secure server which was backed up daily. Custom windows allowed study members to see only data that were relevant to them, only the study co-ordinator and database manager being allowed to see all fields or to amend data. Research participants were entered individually by entry of their hospital number and demographics were then imported from the hospital patient administration system (PAS). In the subsequent conduct of this long study, letters were always sent to the most recent address on PAS, and it was simple to check whether the hospital had been informed of the death of the participant before we made any telephone calls or appointments. Randomisation codes were held within the database, being allocated automatically once physician and physiotherapist had agreed the patient was suitable for rehabilitation. This system worked well.

The database automatically generated follow-up schedules and data collection sheets, and in turn information from these were promptly entered into the database. Numerical limiters and drop-down menus ensured consistency of data entry with numerical calculations from primary data being carried out within the database.

Study design

Sample size

The original primary outcome (later co-primary) was the percentage change in exercise capacity relative to baseline. Our power calculation was based on the percentage change in endurance shuttle walk distance, [(post rehabilitation – pre rehabilitation)/pre rehabilitation] × 100. From a study of 20 COPD patients the mean percentage change in distance walked relative to baseline was 188% [standard deviation (SD) 343%]. Assuming similar levels of variability, if a difference in mean percentage change of 100% between the community and hospital groups is considered to be of clinical and practical importance, then to have an 80% power of detecting this difference in means as statistically significant at the 5% (two-sided) level would require 186 research participants per group (372 in total). If 20% of participants are lost to follow-up then we needed to recruit and randomise 234 per group (468 in total). Table 1 shows the sample sizes required to detect various differences in percentage change in distance walked relative to baseline between the hospital and community groups.

TABLE 1 - Sample sizes required to detect various differences in percentage change in distance walked relative to baseline in between the hospital and community groups

80% power and 5% two-sided significance.

Mean difference in change scores	n per group	Total
200%	48	96
175%	62	124
150%	84	168
125%	120	240
100%	186	372
75%	330	660
50%	740	1480

This calculation used a historical group of patients with a very high variability in improvement. Early in the course of the trial, the DMEC suggested that it would be appropriate to update power calculations based on the 58 participants who had already completed a post-rehabilitation assessment. The sample size adjustment was carried out in accordance with ICH (International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use) guidance: ‘In long term trials there will usually be an opportunity to check the assumptions which underlay the original design and sample size calculations. This may be particularly important if the trial specifications have been made on preliminary and/or uncertain information. An interim check conducted on the blinded data may reveal that overall response variances, event rates or survival experience are not as anticipated. A revised sample size may then be calculated using suitably modified assumptions, and should be justified and documented in a protocol amendment and in the clinical study report.’17

Among these 58 people the mean distance walked at baseline (pre rehabilitation) was 307 m (SD 166 m) and 498 m (SD 387 m) at the post-rehabilitation assessment. This is an absolute increase of 191 m (SD 341 m) or a mean percentage change in distance walked relative to baseline of 75.7% (SD 118.8%). Thus, the updated estimate of the variability of the outcome is 120%. Assuming an SD of 120%, if a difference in mean percentage change of 60% between the community and hospital groups is considered to be of clinical and practical importance, to have an 80% power of detecting this difference in means as statistically significant at the 5% (two-sided) level would require 64 patients per group (128 in total), with valid distance walked data pre and post rehabilitation (Figure 1).

FIGURE 1.

Power size calculation. The figure shows the changing power to detect a significant difference with increasing sample size. Each curve represents a differing minimally detectable change in pre- vs post-ESWT performance, assuming a standard deviation of 120%.

Extrapolating from the early part of the study we expected to recruit 240 people, and for 60% to have evaluable post-rehabilitation data; so that post rehabilitation there would be 144 research participants or 72 per group. This was discussed in depth with the funders and agreed. Using the revised standard deviation of 120% and original effect size of 100%, with 70 participants per group the study would have over 99% power to detect this difference as statistically significant at the 5% (two-sided) level if it really existed.

Effect on acute outcome of treatment by individual physiotherapy team

During the conduct of the trial, three teams of physiotherapists delivered pulmonary rehabilitation sequentially. Each delivered care according to the trial protocol, and had been trained by the same trainers. Although each team carried out both hospital and community rehabilitation according to randomisation, we checked to ensure that there was no difference in group effect for each physiotherapy team. There was no interaction between physiotherapist and group effect, in each case being a slight trend to greater improvement in the hospital group. Although the relative efficacy by site was similar, we were surprised to note a distinct trend to different absolute efficacy for different physiotherapy groups, which fell just short of conventional statistical significance (p = 0.054) after correction for pre-rehabilitation walking difference. We could detect no difference in baseline demographics to explain this. Figure 3 shows that the difference was potentially large, e.g. the 95% CI for difference in mean endurance shuttle walking time post rehabilitation for those supervised by groups B and C was –0.007 to 5.792. This should be compared with a mean improvement of 4.2 minutes for the pooled data of all patients, i.e. the differential efficacy between pulmonary rehabilitation run by different groups could be as large as the overall mean rehabilitation effect.

FIGURE 3.

Acute effect of pulmonary rehabilitation by treating physiotherapy team. The graph shows a post hoc analysis of post-rehabilitation endurance shuttle walking test time achieved by patients treated by physiotherapist groups A, B and C compared with pooled pre-rehabilitation times. There was no significant difference in baseline characteristics of patients treated by each team.

Results

Figure 4 shows the number of patients who were randomised and the number of patients who had valid follow-up data. Two hundred and forty patients were randomised: 111 to community and 129 to hospital rehabilitation. Tables 7 and 8 show the baseline clinical, demographic and HRQoL characteristics of the two groups. In addition, Table 7 shows the percentage of pulmonary rehabilitation sessions attended per group. The groups appear well matched with respect to these characteristics.

FIGURE 4.

Patient numbers available for analysis – CONSORT flow chart.

Figure 4 shows that only 68% and 66% of patients in the community and hospital rehabilitation groups respectively had follow-up data for analysis. Tables 9–11 describe and compare the baseline clinical, demographic and HRQoL characteristics of these four groups (i.e. patients with and without follow-up data). To examine the problem of non-response/attrition bias or loss to follow-up bias we fitted a multiple linear/logistic regression model to see if the baseline characteristics of the COPD patients varied differently between the hospital and community groups according to whether or not the patient completed a post-rehabilitation assessment. The four groups appear to have broadly similar characteristics at baseline, and interaction tests found no evidence that baseline characteristics [age, ESWT, gender, BMI, forced expiratory volume in 1 second (FEV₁), HRQoL] varied significantly by treatment group or responder status.

One hundred and sixty-one research participants had valid pre- and post-rehabilitation data and were used for the analysis of the primary outcome change (pre versus post ESWT). Table 12 shows the baseline clinical and demographic characteristics of these 161 patients, together with the percentage of pulmonary rehabilitation sessions attended. Again, the two groups (76 community and 85 hospital participants) have similar baseline characteristics.

Exploratory regression analysis failed to show any baseline demographics (age, sex) or clinical parameters (FEV₁, baseline walking distance, BMI, MUAMA) that predicted those likely to accept rehabilitation and produce evaluable data, and allowed them to be distinguished from the group who dropped out between recruitment and first evaluation.

Primary outcome was percentage change in exercise capacity as assessed by the ESWT. Results can be expressed in two forms; the actual distance walked, or the duration. In addition to percentage change in ESWT distance, we also report absolute time and distance for ESWT for completeness to fully describe our research participants.

The primary efficacy response variable was the percentage change relative to baseline, i.e. [(end of rehabilitation – baseline)/baseline] × 100 exercise capacity.

Table 13 shows that people in the hospital rehabilitation group increased the distance they could walk at the post-rehabilitation follow-up by 283 m (SD 360 m), an increase relative to baseline of 109% (SD 137%). Patients in the community rehabilitation group increased the distance they could walk at the post-rehabilitation follow-up by 216 m (SD 340 m), an increase relative to baseline of 91% (SD 133%).

There was no statistically significant difference between the rehabilitation groups in percentage change in distance walked relative to baseline, which was 17.8% (95% CI –24.3 to 59.9, p = 0.405, n = 161).

Table 13 also shows that the hospital rehabilitation group increased the time they could walk at the post-rehabilitation follow-up by 5.0 (SD 5.9) minutes, an increase relative to baseline of 89.7% (SD 108.6%). Patients in the community rehabilitation group increased the time they could walk at the post-rehabilitation follow-up by 3.9 (SD 5.7) minutes, an increase relative to baseline of 76.2% (SD 110.2%). The observed difference between the two groups in the change in time walked relative to baseline was 13.5% (95% CI –20.8 to 47.7, p = 0.438, n = 160) in favour of hospital rehabilitation, although this difference was not statistically significant.

There was no statistical difference between attendance at pulmonary rehabilitation sessions between the community and hospital venues.

The preferred method of analysis is using post-treatment means adjusted for baseline using ANCOVA, as this is superior to the analysis of post-treatment means and the mean changes. Table 14 shows the results of ANCOVA by post-rehabilitation ESWT outcomes (distance and time walked). There was no evidence of a statistically significant difference between the rehabilitation groups on either the simple unadjusted analysis (comparison of post-rehabilitation mean distance walked) or the adjusted analysis (adjusted for baseline value of the outcome). The adjusted mean difference in distance walked was 67.8 m (95% CI –41.0 to 176.6, p = 0.22, n = 161). There was no evidence of a statistically significant difference between the rehabilitation groups on either the simple unadjusted analysis (comparison of post-rehabilitation mean time walked) or the adjusted analysis (adjusted for baseline value of the outcome), the adjusted mean difference in time walked being 1.1 minutes (95% CI –0.7 to 2.9, p = 0.25, n = 160).

Table 15 shows the results of the analysis of the secondary HRQoL outcomes. There was no evidence of a statistically significant difference between the rehabilitation groups on either the simple unadjusted analysis (comparison of post-rehabilitation mean HRQoL scores) or the adjusted analysis (adjusted for baseline value of the outcome).

Figure 5 shows how we can interpret the observed study results in relation to the mean difference and 95% confidence limits, and how these appear in relation to the null (zero value) of no treatment difference and the pre-specified minimum important difference in outcomes. 23

FIGURE 5.

Use of confidence intervals to help distinguish statistical significance from clinical importance.

The CONSORT flow diagram of Figure 4 shows that 238 COPD (111 community, 127 hospital) patients had pre-rehabilitation (baseline) ESWT distance walked data. However at post rehabilitation, only 161/238 (67%) had valid ESWT distance walked data. Two methods of missing data imputation were used to impute the missing ESWT data for these 77 COPD patients: LOCF and regression. Table 16 shows the results of the missing data imputation analysis and compares them with the original non-imputed data. Figures 6 and 7 show the results graphically alongside the original data. The missing data methods tend to reduce the treatment effect and move the data towards the null value of no difference in treatments. Figure 6 also shows the minimum clinically important difference (MID) in percentage change in distance walked relative to baseline on the ESWT between the rehabilitation groups. The ESWT does not have a clearly defined MID. Our original power calculations assumed 100% difference as being clinically important. Subsequent examination of our own data suggests that a more conservative estimate of 60% may be more appropriate (see Appendix 4). We therefore use this difference of 60% or more in percentage change in distance walked relative to baseline on the ESWT between the rehabilitation groups when inspecting our results. For the observed data the upper limit of the 95% CI is just inside the 60% MID. This shows that not only is the difference not statistically significant overall (as the CI includes zero) but also there is unlikely to be a clinical important difference [see Figure 5 condition (e)]. The two missing data imputation methods further support the view that the difference is not statistically significant and not clinically important and that the two forms of rehabilitation, hospital and community, are broadly similar or equivalent. However, if the MID changes, for example it is reduced to 40%, then the interpretation of the results clearly changes as well.

FIGURE 6.

Percentage change in endurance shuttle walking test distance walked relative to baseline with regression and LOCF imputed missing data. LOCF, last observation carried forward.

FIGURE 7.

Percentage change in endurance shuttle walking test distance time relative to baseline with regression and LOCF imputed missing data. LOCF, last observation carried forward.

Table 15 shows there were no statistically significant differences in any measure of HRQoL between groups.

A mean difference of five or more points between the groups on the SF-36 dimensions is regarded as the MID. 24 The broad CIs in some domains (which include a five-point difference) admit the possibility that a clinical difference might be detected in a more highly powered study [condition (d) in Figure 5].

A mean difference of 2.0 or more points between the groups on the CRQ total or 0.5 on each of its domains is regarded as the MID. 41 This is just included within the 95% CI for two domains, which cannot exclude the possibility a clinical difference might be detected in a more highly powered study, although this is unlikely.

A mean difference of 0.04 or more points between the groups on the SF-6D utility measures is regarded as the MID. 25 This is just included within the 95% CI, which cannot exclude the possibility a clinical difference might be detected in a more highly powered study, although this is unlikely.

A mean difference of 0.07 or more points between the groups on the EQ-5D utility measures is regarded as the MID. 25 This is just included within the 95% CI, which cannot exclude the possibility a clinical difference might be detected in a more highly powered study, although this is unlikely.

Global health change

After all the HRQoL questionnaires, participants were asked post rehabilitation to rate their overall HRQoL as the same, better or worse, since before they started rehabilitation. Table 17 and Figure 8 show how this response varies by group. There was no statistical evidence that this outcome varied by rehabilitation group. Overall 58.2% (99/170) felt their overall HRQoL was better post rehabilitation, 88.8% (151/170) feeling that it was the same or better.

FIGURE 8.

Post-rehabilitation self-reported global health change by group (n = 170).

Summary of acute effects

Rehabilitation groups were well matched with respect to baseline demographic and clinical characteristics. There were no differences in treatment effects between hospital and community rehabilitation with respect to the primary efficacy response variable (the percentage change relative to baseline during the ESWT) or to secondary HRQoL outcomes. However, the CIs for some of these differences were wide. Although unlikely, potentially clinically important differences between the groups cannot be entirely excluded.

For our primary outcome measure of endurance shuttle walking distance, the minimum clinically important distance we defined a priori was outside the 95% CI. It is unlikely that this trial failed to detect a clinically important difference between the sites of pulmonary rehabilitation because of underpowering; any real difference is likely to be small.

Overall the post-rehabilitation outcomes were broadly similar between the hospital and community rehabilitation groups, although we cannot say they are exactly equivalent. The pulmonary rehabilitation programme overall, irrespective of setting, produces statistically significant improvements in outcomes post rehabilitation (endurance shuttle walk times, and HRQoL using generic and disease-specific tools).

Results

Figure 9 shows the CONSORT diagram for the follow-up of the study. As not all of the people completed the ESWT at all of the follow-up visits, this is shown separately. The diagram is complex; if someone declined to attend a follow-up visit, he or she was encouraged to complete the questionnaires at home. Declining one of the visits and/or failing to return the questionnaires did not preclude the participant from being invited to attend for subsequent follow-up visits. Some people who declined appointments attended the Respiratory Function Unit for other purposes and were also encouraged into completing as many of the follow-up tests as possible. Tables 18 and 19 show the mean pre-rehabilitation demographic characteristics (including the percentage of possible rehabilitation sessions attended) and HRQoL scores of the four groups in the 2 × 2 factorial design. The groups were well matched. Table 20 shows the HRQoL scores of the four groups in the 2 × 2 factorial design immediately post rehabilitation, which is now the baseline for the follow-up analysis.

FIGURE 9.

Participant numbers (with ESWT data) available for analysis – CONSORT flow chart.

Endurance shuttle walking test outcomes post rehabilitation

Figures 10 and 11 show how the ESWT distance walked varies over time (the three post-rehabilitation follow-up assessment visits). There appears to an initial increase post rehabilitation followed be a decline in distance walked over the remaining three post-rehabilitation follow-up visits.

FIGURE 10.

Mean distance walked on ESWT by treatment group over time.

FIGURE 11.

Mean distance walked on ESWT by treatment group over time utilising the factorial design.

Tables 21 and 22 shows there was no evidence of a rehabilitation group effect (p = 0.971). After allowing for the initial post-rehabilitation baseline distance walked, time (follow-up visit) and the factorial design (telephone follow-up group), the average difference in the post-rehabilitation follow-up distance walked on the ESWT between the hospital and community rehabilitation groups was 1.5 m (95% CI –82.1 to 97.2). Where the rehabilitation takes place (hospital or community) has no effect on the longer term (6, 12 and 18 months post rehabilitation) distance walked on the ESWT.

The longitudinal model suggests there was no evidence of a telephone follow-up group effect (p = 0.174). After allowing for the initial post-rehabilitation baseline distance walked, time (follow-up visit) and the factorial design (rehabilitation group), the average difference in the post-rehabilitation follow-up distance walked on the ESWT between the telephone and no-telephone groups was 56.9 m (95% CI –25.2 to 139). Telephone follow-up has no effect on the longer term (6, 12 and 18 months post rehabilitation) time walked on the ESWT in this sample of patients with COPD.

Figure 11 shows how the ESWT time walked varies over time (the three post-rehabilitation follow-up assessment visits). There appears to be an initial increase post rehabilitation followed by a decline in time walked over the remaining three post-rehabilitation follow-up visits.

Figures 12 and 13 and Tables 23 and 24 show there was no evidence of a rehabilitation group effect (p = 0.572) using the ESWT time walked outcome. After allowing for the initial post-rehabilitation baseline time walked, time (follow-up visit) and the factorial design (telephone follow-up group), the average difference in the post-rehabilitation follow-up time walked on the ESWT between the hospital and community rehabilitation groups was 0.3 minutes (95% CI –0.9 to 1.6). Where the initial rehabilitation takes place (either in the hospital or out in the community) has no effect on the longer term (6, 12 and 18 months post rehabilitation) time walked on the ESWT in this sample of patients with COPD.

FIGURE 12.

Endurance shuttle walk time at follow-up. The graphs show the endurance shuttle walking time in telephone vs no telephone groups during 18 months of follow-up. The statistics presented refer to corrected intergroup differences as explained in the text. (Data show mean and standard deviation.)

FIGURE 13.

Endurance shuttle walk time at follow-up. The graphs show the endurance shuttle walking time in hospital- vs community-treated groups during 18 months of follow-up. The statistics presented refer to corrected intergroup differences as explained in the text. (Data show mean and standard deviation.)

The longitudinal model suggests there was no evidence of a telephone follow-up group effect (p = 0.127). After allowing for the initial post-rehabilitation baseline time walked, time (follow-up visit) and the factorial design (rehabilitation group), the average difference in the post-rehabilitation follow-up time walked on the ESWT between the telephone and no-telephone groups was 1.0 minutes (95% CI –0.3 to 2.2). Telephone follow-up has no effect on the longer term (6, 12 and 18 months post rehabilitation) time walked on the ESWT in this sample of patients with COPD.

Long-term quality of life outcomes post rehabilitation

Tables 25–33 show the results of analysing the HRQoL outcomes post rehabilitation. Both generic and disease-specific quality of life measures were utilised. The summary SF-6D is important as the measure that is used in health economic analyses, and the results are shown in the Figure 14. There was no difference between hospital and community rehabilitation groups over time after allowing for the baseline post-rehabilitation value of the outcome, time (follow-up visit) and the factorial design (adjusted group difference 0.00, 95% CI –0.02 to 0.02, p = 0.827). There was no significant difference in persistent of improvement in the telephone follow-up versus usual follow-up group (adjusted group difference 0.02, 95% CI 0.00 to 0.04, p = 0.09). Inspection of the lower limit of the 95% CI indicates that it is exactly zero. This raises the possibility that a more highly powered study might find a statistically significant difference, although the magnitude of any likely difference is small compared with the minimum important difference.

FIGURE 14.

SF-6D The figure shows the SF-6D, the summary parameter derived from the SF-36v2 generic health related quality of life measure, during follow-up. The upper panel shows hospital- vs community-treated groups, the lower shows telephone encouragement vs standard follow-up groups. Statistics refer to adjusted mean differences derived from ANCOVA as described in the text. (Data show means and standard deviations.)

Results for EQ-5D and the physical and mental domains of the SF-36v2 are presented in Tables 25–28. There were no significant differences during follow-up of these measures in either community versus hospital or telephone versus no telephone follow-up.

Disease-specific CRQ likewise failed to show any significant difference between community and hospital groups over time (adjusted difference 0.02, 95% CI –0.91 to 0.94, p = 0.974) (see Tables 29–33). Table 29 demonstrates a significant difference in total CRQ in favour of telephone versus no telephone follow-up (adjusted difference 0.92, 95% CI 0.01 to 1.83, p = 0.047), although the mean difference is less than the minimum important difference. Examination of the four domains of the CRQ shows that this difference was driven largely by a highly significant difference in mastery (adjusted difference 0.4, 95% CI 0.1 to 0.6, p = 0.002) (see Table 31). The minimum important difference for a domain of the CRQ is 0.5. The mean change in mastery is of this order. There was no significant difference in the emotional domain (adjusted difference 0.2, 95% CI 0.0 to 0.5, p = 0.076), The lower 95% CI is exactly zero, admitting the possibility that a more highly powered study may have found a statistically significant though small difference. Fatigue and dyspnoea domains failed to show differences.

Telephone follow-up

Eighty-one patients had one or more telephone contacts, and 80 patients had at least one valid telephone contact where they responded to the initial health status question (see Appendix 3 for details of the telephone call proforma) on ‘How is your chest today, compared to how it usually is?’.

For these 80 patients the number of valid telephone contacts ranged between one and eight with a mean (and median) of five. Figure 15 shows the distribution of the number of valid patient telephone contacts.

FIGURE 15.

Dot plot of number of valid telephone contacts (n = 80).

As the number of telephone contacts varied from one to eight, for each patient we calculated a series of summary measures to describe the telephone contact. For example, for each individual patient we calculated the percentage of the valid telephone calls in which the patient reported that he or she was still doing the booklet exercises. If patients reported they were still doing the exercises at each telephone contact, then this summary value would be 100%.

Table 34 reports the descriptive statistics for these summary measures of the telephone contact data. For example, patients reported that on average for 61% of telephone contacts they were still doing the booklet exercises and for 69% of telephone contacts they reported they were still doing other exercises as well. At 10% of their telephone contacts, patients reported increasing their exercise times.

For 65% of the telephone contacts, patients reported their chest today as being the same or better (than usual).

Introduction

The use of pulmonary rehabilitation exercise programmes for people with COPD could have several important effects on costs and outcomes. The most obvious effect on costs might be a decrease in the demand for primary NHS services and costs associated with medication use, although an increase is also possible because of greater problem identification and referral. Increased demand on staff time for conducting the pulmonary rehabilitation sessions and the additional facilities hire and equipment costs have also to be measured. An economic evaluation was undertaken to capture these changes in resources and to evaluate cost-effectiveness.

Economic evaluation methods

The economic evaluation followed the technology appraisal guidelines used by NICE26 and, as such, takes into account the NHS (rather than societal) perspective.

Outcomes of community and hospital pulmonary rehabilitation sessions were modelled as quality-adjusted life-years (QALYs), and costs measured in pound sterling. The comparative costs and outcomes of community and hospital pulmonary rehabilitation were estimated, along with the incremental cost per QALY of providing community pulmonary rehabilitation.

We perform a cost–utility analysis of pulmonary rehabilitation given in the routine (hospital) setting as opposed to the community setting. We also evaluate the effect of telephone follow-up calls on the long-term cost-effectiveness of pulmonary rehabilitation. Additionally assuming that individuals may not have the choice of location where they attend the pulmonary rehabilitation sessions, we assess the impact of follow-up calls where location is constrained.

Costs

Analysis

The first comparison for the economic analysis was based on hospital- versus community-provided rehabilitation. A second analysis comparing routine follow-up with telephone follow-up in the hospital and community setting was also undertaken. This analysis evaluates whether telephone follow-up leads to prolonged benefits of pulmonary rehabilitation. In the final analyses, telephone follow-up is assessed where the setting is fixed to hospital or community in the delivery of the pulmonary rehabilitation sessions.

Although we found that both costs and effects were not statistically significantly different between setting or between routine and telephone follow-up, the costs and QALYs were combined in order to assess cost-effectiveness. This follows accepted practices in economic evaluation for handling decision uncertainty rather than relying on arbitrary rules of inference. 29,30 For the decision-maker, a p-value is insufficient to judge whether or not to fund a new service. What is more helpful is an estimate of the likelihood that a decision is going to be cost-effective. The main focus of the analysis is the plot data on the cost-effectiveness plane and their associated cost-effectiveness acceptability curves (CEACs). These plots were based on the bootstrapped sample means, generated from the cost–QALY pairs from the data. Interpretation of the CEAC was based around the probability of cost-effectiveness in the £20,000–30,000 per QALY range, which reflects the thresholds that are typically used by NICE to identify which interventions to fund.

A further set of analyses were undertaken to adjust the results for covariates in order to take into account differences in baseline characteristics and produce more accurate mean differences. Data were not imputed.

Results

Analysis of hospital versus community pulmonary rehabilitation at 18 months post rehabilitation

Analysis of cost-effectiveness is bivariate in nature and, in order to capture the covariance between costs and effects, is best undertaken on paired data (using cases where both cost and effects data are available). This requirement, together with the use of multiple data sources, typically leads to attrition. Of the 240 individuals who attended the baseline pre-rehabilitation assessment only 90 individuals provide full 18-month economic data.

For the sample of 90 cases of paired cost and effectiveness data at 18 months, Table 41 (Appendix 1) shows the resource use items by study arm. There are no statistically significant differences in the number of resource use items in the two groups.

When combined with unit costs (see Table 38), the overall mean cost per individual (including the cost of hospital or community rehabilitation) was £4511.21 (£3794.69 SD) and £3643.74 (£3314.43 SD) for the hospital and community rehabilitation groups respectively.

From Tables 41 and 42 (Appendix 1) we can see that the pulmonary rehabilitation sessions account for only a minority of the total costs (approximately 5%). Secondly, the difference in mean costs is largely determined by the differences in prescription costs.

The number of QALYs gained was greater in the hospital pulmonary rehabilitation group (Table 43), although not statistically significant. We find there that whilst costs are lower in the community group, outcomes are worse. However this does not take into account the sampling uncertainty associated with the costs and QALY pairs.

This uncertainty is best illustrated in the cost-effectiveness plane shown in Figure 16; at the centre of the cloud of points are the mean incremental cost and QALYs gained/lost for community-based pulmonary rehabilitation from Table 43 (Appendix 1) (–867.47 and –0.03) with a cost-effectiveness ratio of 28,819.75 (95% CI –270,737.45 to 35,905.61). This shows other combinations of costs and QALYs which are consistent with the data and produce sample means in all four quadrants (i.e. positive and negative costs and QALYs in every combination).

FIGURE 16.

Incremental costs and QALYs of community rehabilitation. The plot shows modelled estimates of the incremental cost difference for community vs hospital rehabilitation vs QALYs gained; for instance, plots to the lower left show models demonstrating a lower cost for community rehabilitation but a reduced gain in QALYs.

All this information can be summarised in the form of a CEAC, which is shown in Figure 17. This shows the probability that the intervention (community-based pulmonary rehabilitation) is cost-effective at various ‘threshold values’ of a QALY. If we are not willing to pay anything for an additional health gain, the intervention would have an 88% chance of being cost-effective; this reflects the fact that the majority of observations in Figure 16 are in the south-west quadrant. In the range of willingness to pay for an additional QALY of £20,000–£30,000, the probability of the intervention being cost-effective decreases to between 56% and 50%. The fact that the scatter of points lies across all the quadrants in the scatter plot shows us that there is considerable uncertainty around the parameter estimates.

FIGURE 17.

Cost-effectiveness acceptability curve for community rehabilitation. The figure shows the probability that community-based pulmonary rehabilitation will be more cost-effective than hospital-based pulmonary for different threshold values of a QALY. Typically, NICE assumes a threshold value of a QALY as £10,000–20,000. The y-axis shows the probability that the new treatment is cost-effective.

Costs were then adjusted using the following baseline data: resource use, gender and age. This resulted in the mean incremental cost saving associated with community pulmonary rehabilitation apparently reducing from £867 to £463, although in neither case was this a statistically significant difference. QALYs were adjusted using baseline utility and, again, age and gender. This had a negligible effect on the results, with no change in the incremental QALYs gained due to community pulmonary rehabilitation at the second decimal place. These adjustments result in diminishing the probability that community-based pulmonary rehabilitation is cost-effective.

Analysis of community versus hospital pulmonary rehabilitation with routine follow-up versus telephone follow-up

When we add the cost of follow-up telephone calls to the long-term (18-month) analysis we can see its impact on the long-term cost-effectiveness of pulmonary rehabilitation [Table 44 (Appendix 1) and Figure 18].

FIGURE 18.

Cost-effectiveness acceptability curves for four groups at 18 months. The figure shows the probability that each intervention group in the factorial design will prove most cost-effective for a threshold cost per QALY. NICE typically assumes a threshold cost per QALY of £10,000–20,000. The y-axis shows the proportion of simulations favouring each treatment.

We can see in Figure 18 that the community-based pulmonary rehabilitation with telephone follow-up group is shown to be most likely to be cost-effective. The prolonged effects of telephone follow-up calls after pulmonary rehabilitation seem to have a greater impact on the community group as shown by comparing the difference between the two hospital curves, with standard and telephone follow-up with the community curves with standard and telephone follow-up. There is however significant uncertainty in this analysis.

Analysis routine follow-up versus telephone follow-up where the setting is fixed

An additional set of analyses were undertaken to assess the cost-effectiveness of telephone follow-up when the location of rehabilitation cannot be changed. In these circumstances the trial can provide information on the value of telephone follow-up (as this was randomised within each arm).

Estimation of cost-effectiveness planes and CEACs indicate that, in the community setting, follow-up is cost-effective (87% chance at £20,000 per QALY), whilst hospital-based pulmonary rehabilitation follow-up is not cost-effective (31% chance at £20,000 per QALY). However, when each set of analyses are adjusted using the same baseline covariates as before, the results change (Table 45, Appendix 1). The changes are small for the community group, and consequently have a small impact on the associated CEAC; the probability of telephone follow-up being cost-effective at £20,000 per QALY reduces from 87% to 82% (Figure 19). However, the change for the hospital group is large, changing the probability of telephone follow-up being cost-effective at £20,000 per QALY from 31% to 72% (Figure 20).

FIGURE 19.

Cost-effectiveness acceptability curves – routine vs telephone follow-up in the community group. The figure shows the corrected probability that telephone follow-up will be cost-effective compared with standard follow-up in the group receiving pulmonary rehabilitation in a community setting for various threshold costs per QALY. NICE typically assumes a threshold cost per QALY of £10,000–20,000. The y-axis shows the probability that the new treatment is cost-effective.

FIGURE 20.

Cost-effectiveness acceptability curves – routine vs telephone follow-up in the hospital group. The figure shows the corrected probability that telephone follow-up will be cost-effective compared with standard follow-up in the group receiving pulmonary rehabilitation in a community setting for various threshold costs per QALY. NICE typically assumes a threshold cost per QALY of £10,000–20,000. The y-axis shows the probability that the new treatment is cost-effective.

Lack of power and sample size

The primary outcome for the purposes of sample size estimation was the percentage change post rehabilitation relative to baseline (pre rehabilitation) in distance walked on the ESWT. Overall we had 161 (85 hospital and 76 community) patients with pre- and post-rehabilitation ESWT data for analysis. Clearly, we recruited fewer patients than the original sample size estimate of 186 patients per group (372 patients in total). This was based on a treatment effect of 100% and an estimated standard deviation of 343%. During their first meeting, the independent DMEC remarked on the relatively high SD used in this calculation, and suggested that we recalculate the power of the study based on results of those patients studied to date (with blinding preserved). The revised sample calculation was scrutinised and accepted by the trial sponsors. A revised SD of 120% was estimated from the first 58 patients recruited to the trial. We also revised the MID effect downwards to a 60% difference. So, assuming an SD of 120%, if a difference in mean percentage change of 60% between the community and hospital groups is considered to be of clinical and practical importance, then to have an 80% power of detecting this difference in means as statistically significant at the 5% (two-sided) level would require 64 patients per group (128 in total). Using the revised SD of 120% and original effect size of 100%, with 70 patients per group, the study would have over 99% power to detect this difference as statistically significant at the 5% (two-sided) level if it really existed.

Sample size calculations were a priori because after we had carried out the study it was the observed data that determined the size and direction of the treatment effect and the width of any CI estimates for this treatment effect. The only parameter we needed to consider from the initial power calculation was the size of the effect we considered to be clinically meaningful, the MID, and how our observed treatment effect and associated CIs appear in relation to this MID. Statisticians have long faced the difficulty of trying to elicit the MID from researchers and clinicians, particularly for the objective of sample size calculations. Yet the same researchers, who find a difficulty in specifying a threshold for the MID a priori, often have no difficulty in doing so after the data have been collected and analysed and in declaring a post hoc result to be of clinical or practical importance or not. The concept of an a priori MID is not well defined. However, researchers should attempt to define the MID before the data are collected and analysed. If the MID is defined after the data are analysed then it is not meaningful. In summary, post hoc power calculations are not recommended. What investigators should do is calculate a CI for the observed treatment effect from the data collected. This CI should then be interpreted in relation to the planned or minimum clinically relevant difference used in the initial sample size calculation. 31

The actual standard deviation for the observed data turned out to be a little larger at 137% and 133% for the hospital and community rehabilitation groups respectively. Patients in the hospital rehabilitation group increased the distance they could walk at the post-rehabilitation follow-up by 283 m (SD 360 m), an increase relative to baseline of 109% (SD 137%). Patients in the community rehabilitation group increased the distance they could walk at the post-rehabilitation follow-up by 216 m (SD 340 m), an increase relative to baseline of 91% (SD 133%). The observed difference between the two groups in the change in distance walked relative to baseline was 17.8% (95% CI –24.3 to 59.9, p = 0.405, n = 161) in favour of hospital rehabilitation. However, the CI for this difference is wide, reflecting the sample size and the uncertainty in the outcome. The CI excludes the minimum we sought to examine in a priori power calculations. Hence, although there may be a small difference in outcome between the groups, if such a difference exists, it is smaller than the magnitude defined in our original power calculations or indeed in our revised more rigorous definition. Thus the study was in effect adequately powered though it failed to recruit as many participants as called for using the original power calculations. Overall the post-rehabilitation outcomes were broadly similar between the hospital and community rehabilitation groups, although we cannot say that they are absolutely equivalent.

Recruitment and selection

The major inclusion criterion was that patients should have MRC grade 3 breathlessness or worse that was predominantly due to COPD in the view of a respiratory physician, mirroring advice in the UK NICE guidelines. The trial was designed as a pragmatic or ‘real world’ trial and hence we tried to avoid overly prescriptive inclusion and exclusion clauses. Patients may have had modest degrees of additional heart failure, asthma or other diseases that contributed to their dyspnoea. We see this as a strength rather than a weakness, as it allows greater generalisability of data to the bulk of routine clinical practice rather than to a subset of patients.

Research participants were accepted from a variety of venues, including hospital outpatients, referral to physiotherapy and direct self-referral following publicity. They were all reviewed by a respiratory physician to check the diagnosis and optimise treatment, unless such a review had already taken place within the last year. Diagnosis and treatment of COPD was in line with the GOLD guidelines. 14 Patients were included with stage 1 disease (i.e. FEV₁ in the normal range but an obstructive FEV₁/FVC ratio) if the physician felt COPD was the predominant cause of breathlessness. Hospital records for this subset of patients were reviewed, and patients included if there was clear radiological emphysema and a low gas transfer, with no alternative cause of dyspnoea apparent.

A maximal incremental shuttle test was included early in our assessment. We had some reservations about carrying out a maximal exercise test in a community setting without full resuscitation facilities, so these assessments were carried out in a hospital setting. In fact during the trial we did not experience significant side effects during these assessments. Further research on safety of maximal exercise testing in such patients would be valuable as it often forms part of rehabilitation protocols, and it would be useful to know the level of risk attached.

We felt that if no adverse events occurred during the initial maximal test, it would be safe to exercise patients in the community subsequently. The majority of exercise in the programme was performed at a submaximal level, and indeed was designed to be exercise that could be adopted as part of the patient’s subsequent lifestyle.

In terms of pulmonary rehabilitation, this trial was large. The Cochrane Review of trials of pulmonary rehabilitation versus standard care2 includes 30 trials with a mean of 42 ± 32 randomised study participants in total compared with 161 in this trial. After initial randomisation and before attendance for rehabilitation we found a large dropout. Again, this accords with previous findings. In the Cochrane Review, 84.8 ± 17.6% of those randomised were evaluated. Our trial produced evaluability broadly in line (67.5%) with these. No doubt some of the small trials reported could preferentially recruit highly motivated patients, so our data is in line with these findings.

One large trial of community versus hospital rehabilitation in Australia initially planned a significant follow-up phase but this was abandoned because of the low numbers of patients (27%) who could be successfully followed up. 32

Clearly we too had a significant dropout rate during the trial, but we believe that the data are valuable in identifying the benefit that potentially can accrue in those followed up, and in quantitating and highlighting this important potential limitation to the widespread adoption of pulmonary rehabilitation in either setting. We do not report detailed qualitative data relating to the low uptake and follow-up, as these did not form part of the trial. However, for completeness, Appendix 4 includes a poster presentation of a small qualitative study performed whilst the participants in this trial attended follow-up sessions. Others have suggested that initial uptake is affected by enthusiasm of recruiting staff, particularly doctors,33 as well as the accessibility of the programme and patients’ perception of likely benefits. 34 Subsequently, a variety of factors affected adherence to the programme, such as problems related to the disease itself, and patients’ expectations of treatment benefit and the limitations of their own disease. 35 Some authors have suggested that depressed patients may be less likely to attend. 36 External factors such as weather may also play a role. 37

We believe that this is a point that has not been sufficiently emphasised and further research is needed. Whilst there is no doubt that pulmonary rehabilitation is effective and it is correctly recommended by NICE for widespread adoption, the large numbers of patients who are either unwilling or unable to take up the offer of a rehabilitation place, or who subsequently drop out, will by definition not benefit from this rehabilitation. Indeed, in population terms, measures taken to increase uptake of pulmonary rehabilitation overall will have much greater marginal benefit than fine tuning the details of the programme itself providing this adheres to accepted minimum standards. At the same time it should be noted that our data were treated as far as possible on an ITT analysis; all those with evaluable data were included as per their randomisation, whether or not they completed all the prescribed rehabilitation. Our data showed that the participants not returning after initial randomisation had no clear demographic differences from those who came back. There was a non-significant trend for non-attendees to be female and have lower MRC breathlessness scores. Perhaps these people perceived themselves as being less disabled, more active and/or less likely to benefit. Alternatively, it is known that women may place relatively low importance on their own health maintenance, and tend to adopt a supportive role. Some of these relatively less disabled women may have concentrated on supporting other family members rather than seeing their own disease as a priority. 38

Outcome measures

The co-primary outcome was the difference in improvement in ESWT between hospital and community pulmonary rehabilitation groups post rehabilitation, and the difference in ESWT during 18 months’ follow-up between those receiving telephone encouragement and those receiving standard care. However, it has been reasonably argued that, as treatment of COPD is essentially symptomatic, health-care status outcomes might be more relevant. 2 We selected an exercise measurement as a clear and conventional outcome. Such outcomes are robust and easily compared between studies and with evaluation of other treatment modalities. As the core treatment modality is exercise training, it is logical to explore changes in exercise capacity as the direct effect of this. Licensing authorities continue to place reliance on physiological outcomes in trials of pharmacotherapy, for instance. We selected the ESWT as a well-validated tool to be sensitive to the effects of pulmonary rehabilitation. 12,13 The test is standardised in such a way as to tend towards isotime exercise at baseline to allow sensitivity to subsequent change. Time or distance might be reported as outcomes. Here we report both to allow full examination of data and definition of patient characteristics throughout follow-up. A problem with this measure is that there is not a clearly defined minimum important difference. We arbitrarily defined this a priori as a 100% change in time (as per the power calculations), subsequently adopting a more rigorous definition of 60%. Examination of the order of magnitude of change in ESWT time that was likely to lead to self-reporting of quality of life as ‘better’ rather than ‘unchanged’ or ‘worse’ suggests that this was justified (see Appendix 4).

Nevertheless, it is important to consider whether a change in exercise capacity translates to a change in health-care status, and to attempt to quantify that change. We used a variety of outcome measures. The SF-36 is a widely validated generic measure of HRQoL. The SF-6D extracted from version 2 is important as it allows calculation of cost-effectiveness by calculating QALYs.

We also utilised the CRQ, which as its name implies is a validated tool for use on inpatients with respiratory disease, and as such is more responsive to change than a generic tool. 39 Finally, we also collected EQ-5D data. There has been interest in using this very simple marker and we felt it a valuable inclusion as it might be easily incorporated into routine care to allow audit of effectiveness of programmes in the future. 42

Safety

Current British Thoracic Society (BTS) guidelines1 state:

The prevalence of ischaemic heart disease in this population has not been well defined. The rate of critical incidents occurring during routine maximal exercise testing, even in patients with cardiac disease, is small. Modest physical exercise does not appear to produce myocardial repolarisation abnormalities, even in the presence of hypoxaemia, in patients with COPD. However, it is recommended that simple first aid medication (oxygen, nebulised bronchodilators, glyceryl trinitrin, etc.) should be available on site. Staff supervising exercise programmes should be trained in resuscitation (e.g. to Advanced Life Support (ALS) standard). The backup of a hospital arrest team is probably unnecessary.

We performed initial maximal ISWTs in a hospital setting because of a concern for safety. No adverse events were recorded during the exercise testing, although we empirically excluded anyone with desaturation to below 80% on pulse oximetry during this test, following discussion with our DMEC.

Patients were advised to take their medication to the rehabilitation sessions. We had no adverse events during rehabilitation that were thought to be attributable to the intervention. Further reporting of safety data from ongoing audit would be useful, in particular relating to the safety of maximal exercise testing and of such programmes in those with a requirement for long-term oxygen or who desaturate on exercise.

There are no clear recommendations regarding safety issues in the NICE commissioning guidelines for pulmonary rehabilitation. 40 The BTS’s conclusion that hospital cardiac arrest teams are ‘probably’ not necessary suggests that further data on safety would be useful.

Acute effects

As noted, there was a significant dropout after randomisation but before initial pre-rehabilitation visit. It should be noted that the protocol utilised was identical in hospital and community to ensure that any differences observed were due to the setting per se. This meant that we did not use complex exercise devices such as exercise bicycles, treadmills or ergometers. Such a programme could easily be adopted in a very large range of settings and local circumstances. We considered our analysis to be ITT, but study participants were required to have evaluable pre-rehabilitation data to be included in the analysis. There was no significant difference in demographic characteristics between those who dropped out of the study before the pre-rehabilitation assessment and those who continued in the study either in community or hospital arms (except for the trend for women who were less symptomatic to drop out as noted above). We believe therefore that the results obtained in those evaluable data are likely to be generalisable. If those who declined rehabilitation in the current study could be encouraged to attend and participate in a rehabilitation programme, there is no a priori reason to suspect there would be a difference in outcome between them and the patients evaluated.

Maintenance of effects

Inevitably, a number of patients did not complete the follow-up as intended. For comparison of hospital versus community outcomes, we included in analysis all patients who attended for pre-rehabilitation assessment whether or not complete data were subsequently available. If patients did not wish to attend the hospital for full testing, they were given the option of completing quality of life questionnaires at home, and missing data were extrapolated as described.

For analysis of telephone versus routine care, full data sets had to be available post rehabilitation. The analysis examined persistence of treatment effect by measuring change of exercise capacity and HRQoL from this point onwards.

As the study was a 2 × 2 factorial design, initial analysis included a test for interaction between the groups (hospital versus community, and telephone encouragement versus standard care). No such interaction of significance was found so we continued to analyse as in a factorial design.

Economic evaluation

There have been relatively few other studies that examine the cost-effectiveness of pulmonary rehabilitation and the need for further studies is emphasised in the ERS/ATS guidelines. 6 The importance of critically examining cost-effectiveness is illustrated by the finding that intensive self-management plans (which are being increasingly adopted) may not in fact be cost-effective. 54 Given that long-term effects of pulmonary rehabilitation are likely to rely on behavioural changes with some overlap with self-management programmes, this urges caution.

The clearest data on the cost-effectiveness of pulmonary rehabilitation come from an RCT in the UK. 55 This showed that there was a clear but non-significant trend towards cost-effectiveness of pulmonary rehabilitation versus standard care and concluded that it was at least cost neutral.

A trial in the US suggested that rehabilitation could be cost-effective,56 but this used comparison of data in the years before and after intervention rather than being an RCT. Information to date therefore suggests that pulmonary rehabilitation is cost-effective.

At 18 months, community-based pulmonary rehabilitation had a lower mean cost of £867 (p > 0.05) and produced 0.03 fewer QALYs (p > 0.05). When assessed in combination, the CEAC indicates that there is a 56% chance that community rehabilitation is cost-effective at £20,000 per QALY. When cost and QALY estimates are adjusted this figure reduces. In essence, the results show that both community and hospital rehabilitation have about a 50% chance of being cost-effective. This reflects both the small differences seen in outcomes and the large amount of variability seen in costs. The conclusion from these results is that service providers can choose which type of pulmonary rehabilitation service they feel is most appropriate for the skills and resources available, taking into account patient preferences.

Once the choice of setting has been made (or if a service has the setting forced upon them), the cost-effectiveness analysis suggests that telephone follow-up could be cost-effective. However, it should be noted that this is a secondary analysis of the economic data, and the results are also dependent on adjustment for covariates. In practice, the uncertainty around this value does not allow firm conclusions to be reached. Further research is required.

Owing to the sick patient group, length of follow-up and the need for patient completed data for the economic analysis, attrition was high (61%). This makes the results highly uncertain. As discussed, imputation of data shows that a large effect on quality of life is unlikely to be concealed by differential dropout, supporting the relevancy of these analyses.

One further area of uncertainty is the longer term cost and health impacts of the two forms of rehabilitation and follow-up. If the treatment is expected to have an impact beyond 18 months, then further work would be necessary to model these impacts.

When acting on the results of this analysis, services need to consider their own costs, as they may vary from those used in this analysis. Especially in early stages, Trusts may be able to make use of spare capacity to run services well below the average costs given in Table 37 (Appendix 1).

Limitations of this study

Patients were randomised to treatment groups, and analysis of the demographics of resulting groups shows that they were indeed well matched. However, there is no conceivable way to blind the treatment group either to research participants or to those carrying out treatment. Those supervising the rehabilitation programme did so for both randomisation groups to the same protocol, and they should have been identical. It is impossible to exclude some bias because of pre-existing views and prejudices about the differing study groups on the part of the therapists. The fact that outcomes were very similar suggests that this was unlikely to play a significant role. The use of separate treatment and assessment teams with the latter blinded to treatment sought to minimise bias in the assessment process itself.

The fact that patients would have an unblinded view is not relevant in this study. One of the significant factors that led to our performing the study was the hypothesis that patients might experience differing overall efficacy of a rehabilitation programme as a result of their differing preconceptions and perceptions. Any ‘bias’ caused by patient unblinding reflects the real effects in routine medical practice.

A major limitation of this study is the dropout rate between randomisation and initial assessment and subsequent dropout during the study. As discussed, our performance in these areas seems in line with other studies. This study was constructed as a pragmatic trial to examine practical ways to deliver care in health communities rather than in precisely examining a mechanistic hypothesis. The difficulty of recruitment and retention is a real clinical problem which must be addressed when planning future health-care delivery and predicting likely outcomes in terms of public health. This limits confidence in the data overall. However, we could not identify demographic or disease severity factors that predicted dropout, and data imputation produced more conservative results, i.e. it was less likely that significant differences existed between groups. Hence, our main conclusion of clinical similarity between groups is likely to remain valid despite this.

We excluded significantly hypoxaemic patients and did not provide oxygen for exercise. There is some evidence that oxygen provision during exercise may enhance the beneficial effect of pulmonary rehabilitation inpatients with COPD, although this is weak. 57 It remains a moot point as to whether it is reasonable to extend community provision to such hypoxic patients and whether a more complex programme requiring routine oxygen usage is reasonable. It might be argued that such patients are more safely and practically treated in a hospital setting but we have no data to address this.

Our analysis tried to be inclusive, so that we used all subjects with initial evaluable data on an ITT basis whether or not there were subsequent full data sets. This included continuing to collect HRQoL questionnaires from those refusing to attend the hospital for follow-up physiological measurements, and indeed to collect data for the long-term follow-up from people who had attended very few pulmonary rehabilitation treatment sessions.

Our major measure of exercise capacity, the ESWT is well validated and known to be sensitive to the change produced by pulmonary rehabilitation programmes. A drawback is the lack of clearly defined minimal clinically significant changes. However, the post-rehabilitation changes we witnessed overall were large and likely to be of significance. By contrast, even the extreme of the 95% CI for between-group difference was considerably smaller, and was less than the a priori value we utilised as likely to be significant in our power calculations.

As noted above, we used a simple model of telephone follow-up. This was intentionally adopted as a very cheap and practical intervention that could be easily adopted if found to be effective. However, we do not know if more intensive telephone intervention might have produced greater effects.

This report uses health economic data that are based on patient recall of contact with health-care professionals. It uses a ‘snapshot’ reference period of 4 weeks before each assessment attendance. This is likely to be the source of some inaccuracy. However, this was supplemented by an analysis of hospital attendances derived from local hospital databases. Hospital contacts will be a major cost driver, so the inaccuracies of patient recall will be limited.

Qualitative data collection was not funded as part of the study and hence we do not report any data on perceptions of the programme and of patient preference. However, for completeness, Appendix 4 includes some qualitative data collected separately from our research participants.

Implications for health-care provision

This study showed that there is no clear reason to favour hospital or community provision of pulmonary rehabilitation for patients with COPD in terms of efficacy in improving exercise capacity or HRQoL. However, clear evidence is presented that group-based community rehabilitation is safe, effective and capable of producing results similar to hospital outpatient programmes.

Telephone follow-up produced beneficial long-term effects in terms of mastery. This selective effect on a component of HRQoL may be important but did not translate through to improvement on generic HRQoL scores (which would be expected to be less sensitive) overall. However, although the economic analysis suggested that telephone follow-up could be cost-effective at the threshold range operated by NICE, the data are not robust and do not permit a firm conclusion to be reached.

The evaluation showed similarity in the cost-effectiveness of community provision as opposed to hospital provision. In developing pulmonary rehabilitation services for a health community, careful attention to local factors that may impact on outcome is needed.

We have shown that a lot of people apparently suitable for group pulmonary rehabilitation are not willing to participate in it. Service providers should be aware of this and should ensure that services are as simple as possible to access. They should also try to offer alternative modes of management for those not accessing group rehabilitation services.

Implications for research

This study randomised people to a venue, then some of them had to wait a long time for the sessions to start. This is very similar to the delays between referral and treatment in some busy pulmonary rehabilitation centres. We found that many people changed their minds about whether they wanted to take part in group exercise sessions in that waiting period, or that the symptoms of their respiratory or a concomitant disease had worsened. These people were thus shown as dropouts before the trial had started. Most studies start with people at the inception of pulmonary rehabilitation sessions, which may give a distorted view of overall efficacy.

We have shown that there are many people who are suitable for pulmonary rehabilitation but decline to utilise the treatment. Further research is required on factors affecting uptake of pulmonary rehabilitation.

Our research concluded that community rehabilitation is safe. We excluded those on long-term oxygen therapy, those who had marked exertional hypoxaemia and those with cardiac failure (as well as other significant comorbidity). Additional research is required to assess the safety and efficacy of community programmes in these more complex patient groups.

Our data suggest that there could be an important difference in the magnitude of treatment effects that different teams can produce, even when working to a standard protocol. Further research is indicated to verify if this is true and, if so, what factors (e.g. interpersonal skills) are important.

We have shown that a simple form of telephone communication can affect mastery. Further research is required to evaluate costs and benefits in other aspects of symptom control and to the efficacy of differing forms of telephone follow-up.

Contribution of authors

Rod Lawson was the chief investigator. He prepared the submission documents then provided the majority of the medical input into the study. He was principal author of the report.

Judith Waterhouse was the study co-ordinator. She suggested the joint application to the HTA. On becoming the part time study co-ordinator she took day-to-day responsibility for all aspects of the study, organising the scheduling of the pulmonary rehabilitation sessions, training the data collection personnel and performing collection herself when necessary. She also helped create the text of the documents and proof read all submissions.

Stephen Walters was the study statistician. He was involved with the project at the design stage (including sample size calculation), prepared the randomisation schedule and continued as an active member of the steering group. He also performed the statistical analyses of the data and participated in the interpretation of the data and the writing (and revising) of the report (particularly the results chapter).

Yemi Oluboyede was the health-care economist. She came into the project when it was ongoing and analysed the economic data and wrote Chapter 6.

Disclaimers

The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.

Appendix 2.1

Used at visit 1 in the 2 weeks prior to treatment.

(PDF download)

Appendix 2.2

Used at visit 2 in the week immediately post treatment.

Also used at visits 3, 4 and 5 (6-monthly intervals post treatment).

The EQ-5D and SF-36 version 2 are identical to those used at visit 1 and thus are not repeated here.

(PDF download)

01/15/10: A randomised 2 × 2 trial of community versus hospital rehabilitation, followed by telephone or conventional follow-up; impact on quality of life, exercise capacity and use of health-care resources