Continuous positive airway pressure devices for the treatment of obstructive sleep apnoea-hypopnoea syndrome: a systematic review and economic analysis

Primary outcomes

• Subjective sleepiness as assessed by the ESS

• Objective sleepiness as assessed by the MWT, Osler test, MSLT or equivalent measure.

Secondary outcomes

• Blood pressure (mean day and night blood pressure were assessed separately as the mechanisms and patterns of daytime and night-time blood pressure disturbance in OSAHS vary, and the relationship between daytime blood pressure and vascular risk has been more clearly described in other studies)

• Cardiovascular disease (e.g. myocardial infarction, stroke)

• Accidents (e.g. driving, occupational), although it was thought unlikely that such data would be found in randomised controlled trials (RCTs)

• Quality of life, where it was measured using a standardised scale

• Mood, anxiety and depression, where they were measured using a standardised scale

• Simulated driving performance

• Neuropsychological functioning

• AHI/desaturation rate

• Any complications or adverse effects of treatment.

Outcomes such as changes to sleep architecture (e.g. rapid eye movement sleep, slow-wave sleep, sleep efficiency) were not considered.

Intervention and comparators

Forty-six of the 48 included studies that used fixed pressure CPAP. The remaining two studies used autotitrating pressure CPAP. 67,77 Three studies used humidified CPAP73,78,79 and in two studies the use of a humidifier was optional. 80,81 All CPAP interventions were treated as a single class in the analysis.

There were three three-arm trials: CPAP versus oral placebo and dental device;82 CPAP versus conservative/usual care and dental device;70 and CPAP versus sham CPAP and supplemental oxygen. 73

CPAP was compared with sham CPAP (18 studies);56,58,62–68,73,77,79,83–88 oral placebo (nine studies);78,82,89–95 conservative/usual care (eight studies);59,69,70,96–100 dental devices (12 studies);70,72,80–82,101–107 and posture-related devices (three studies). 60,61,108

Where sham CPAP was used as placebo, the subtherapeutic pressure ranged from 0 to 4 cmH₂O. Where reported, the majority of studies (n = 12) used a pressure of 2 cmH₂O or less; two used a pressure of between 3 and 4 cmH₂O. 64,84 In the studies using oral placebo an inactive tablet was used and participants were told that the tablet was intended to improve their airway function. The information provided on usual care/conservative treatment as a comparator was limited, but generally included dietary advice, dietary advice or referral to weight loss programmes, or advice on sleep hygiene and sleep posture.

Where reported, there were two main types of dental devices used in the included studies, one-piece non-adjustable devices;70,81,103 and two-piece adjustable devices. 72,80,82,101,102,105 In four of these studies incremental mandibular advancement was used until symptoms abated or further advancement was uncomfortable. 72,80,82,102 In one study some participants used a one-piece and some used a two-piece device. 106

Studies which compared CPAP with some form of device to control sleeping position used a backpack with a soft ball inside to prevent a supine position while sleeping;108 a shoulder–head elevation pillow to maintain an upright position (60°) while sleeping;60 or a cervicomandibular collar to retain the head in a natural position and to prevent the jaw from opening during sleep. 61

Participants

The participants in the included studies were predominantly middle-aged, male and overweight or obese. The mean age in the CPAP and comparison groups at baseline ranged from 44 to 58 years. With the exception of one study,98 the majority of participants in the included studies were male; the proportion of female participants ranged from 0% to 48%. Based on the mean BMI (where reported) ten studies were of an overweight population (BMI 25–30 kg/m²) and 30 were of an obese population (BMI 30.1–40 kg/m²); the highest mean BMI at baseline was 40.1 kg/m². 88 Two studies were of patients who were being treated for another primary disease: type 2 diabetes67 and headache symptoms. 98 Two studies specifically recruited patients with hypertension. 65,68

Table 1 provides details of baseline disease severity for the individual studies. Based on mean baseline daytime sleepiness, as reported by participants using the ESS, the majority of studies were of participants experiencing moderate sleepiness (n = 27); five of the included studies were of participants with severe daytime sleepiness and two were of participants with mild sleepiness. Symptom severity, as defined by the ESS, was not available for 14 studies. Based on disease severity at baseline, defined by AHI [or 4% oxygen desaturation or the respiratory disturbance index (RDI)], the majority of studies (n = 26) investigated a population with severe OSAHS, 15 investigated a population with moderate disease, and three investigated a population with mild disease. One study recruited patients with OSAHS that was mild in the lateral sleep position and severe in the supine position. 108 Disease severity, as defined by AHI or equivalent, was not available for three studies.

Study design

All the included studies were RCTs. There were 26 crossover trials, two partial crossover trials (only one group was crossed over in the second sequence) and 20 parallel trials. Only the data from the first sequence before crossover were used from the partial crossover trials. 85,86 For one crossover trial the outcome data appeared to be from the first sequence and these data were treated as parallel data. 63 For some individual outcomes, only the data from the first sequence of the crossover trials were reported in the papers due to detection of a carryover effect, and these were treated as parallel data in the synthesis. Studies using oral placebo as a comparator were exclusively of crossover design, as were the trials in which the comparator was postural therapy. This was also the dominant study design for trials comparing dental devices with CPAP. Parallel trials were the dominant design used in trials comparing CPAP with sham CPAP or conservative/usual treatment.

Treatment duration varied. The majority of studies were between 4 and 12 weeks’ duration. There were six studies of less than 4 weeks’ duration58,73,79,85,91,108 and four of longer than 12 weeks’ duration. 80,81,99,100 Participants were assessed at the end of treatment.

Criteria

1. Was the method used to assign participants to treatment groups or the sequence of treatments really random (e.g. computer generated or random number table)?

2. Was treatment allocation concealed?

3. Were the groups similar at baseline in terms of ESS and AHI?

4. If not, were adjustments made for differences in the treatment groups?

5. Did the analysis include an intention-to-treat (ITT) analysis?

6. Were appropriate methods used to account for missing data in the ITT analysis?

7. What proportion of participants was lost to follow-up for the primary outcomes?

8. Was the study described as blind or double-blind?

9. Who was blinded?

10. Were the participants CPAP naïve?

11. Was an appropriate analysis, using paired data, conducted? (Crossover trials only.)

12. Was there a treatment by period interaction? (Crossover trials only.)

Full details of the quality assessment are presented in Appendix 3. Eighteen of 48 studies reported an adequate method of random sequence generation. The majority of studies did not report, or reported suboptimal methods of allocation concealment, with five studies reporting adequate allocation concealment, defined according to Cochrane criteria. 62,77,87,95,109 As a consequence of the comparators used, only the 18 studies using sham CPAP were double-blinded; other comparators are visibly different and cannot therefore be double-blinded. Fourteen studies reported that participants were CPAP naïve; of these, 11 studies used sham CPAP as a comparator. It was unclear in the remaining studies using sham CPAP whether participants were CPAP naïve. ITT analysis was defined as all randomised patients included in the analysis within the treatment group to which they were randomised. Although a number of studies described themselves as being ITT, only four studies61,82,87,106 used ITT analysis according to this criterion. The majority of studies reported loss to follow-up; with the exception of a few studies82,85,89,97,99,104,109 this was low (< 20%), with little difference between treatment arms. Of the 26 crossover studies included in the review, 19 reported an appropriate analysis using paired data. Fifteen studies evaluated the possibility of carryover effects, with four studies89,90,92,93 reporting carryover effects in primary or secondary outcomes.

CPAP versus placebo or conservative/usual care

Data were available for the ESS from 23 trials (1334 participants). When all the studies were pooled there was a statistically significant benefit with CPAP compared with placebo/usual care for daytime sleepiness as measured by the ESS (MD –2.7, 95% CI –3.5 to –2.0). However, heterogeneity was high (I² = 71%) and this treatment effect is unlikely to be generalisable. The heterogeneity was investigated using subgroup analysis.

Clinical subgroup analyses

When studies were grouped by severity of daytime sleepiness at baseline (mild, moderate or severe, as defined by the ESS), heterogeneity was reduced. Although there was still evidence of moderate heterogeneity within the subgroups, with the exception of two studies, the direction of the effect was consistently in favour of CPAP (Figure 2). There was a statistically significant improvement in symptoms of daytime sleepiness with CPAP treatment compared with placebo or usual care for all levels of disease severity. The improvement was greatest in trials in which baseline sleepiness was severe (MD –5.0, 95% CI –6.5 to –3.5) and was consecutively smaller with moderate (MD –2.3, 95% CI –3.0 to –1.6) and mild severity (MD –1.1, 95% CI –1.8 to –0.3). The estimate of treatment effect for studies of mild sleepiness at baseline is based on only two studies, one which reported no difference between CPAP and placebo and one which reported a small but statistically significant improvement in favour of CPAP.

FIGURE 2.

Epworth Sleepiness Scale (CPAP versus placebo/usual care), stratified by severity of sleepiness at baseline (ESS).

When studies were grouped by disease severity (AHI) at baseline there was a statistically significant improvement in daytime sleepiness with CPAP compared with placebo or usual care in trials of severe and moderate disease populations but not mild disease (Figure 3). As with the subgroup analysis based on ESS, the treatment effect was largest in the severe disease population and the treatment effect was consecutively smaller with moderate and mild disease. There was moderate to high statistical heterogeneity in the subgroup analyses of trials of severe (I² = 71%) and moderate (I² = 65%) disease. Only three trials were available for the analysis of mild disease and there was low statistical heterogeneity.

FIGURE 3.

Epworth Sleepiness Scale (CPAP versus placebo/usual care), stratified by disease severity at baseline (AHI or oxygen desaturation dip rate).

CPAP versus dental devices

Data were available for the ESS from six trials (n = 337). All of these trials comprised populations with moderate daytime sleepiness (ESS) at baseline. There was no statistically significant difference in the impact on daytime sleepiness (ESS) between CPAP and dental devices (MD –0.9, 95% CI –2.1 to 0.4) (Figure 4). There was evidence of moderate statistical heterogeneity (I² = 60%) and the treatment effect ranged from MD –4.0 in favour of CPAP to a small treatment effect in favour of dental devices (MD 0.4).

FIGURE 4.

Epworth Sleepiness Scale (CPAP versus dental devices), stratified by severity of sleepiness at baseline (ESS).

CPAP versus postural therapy

Data were available for the ESS from three small crossover trials (n = 36);60,61,108 the studies were not pooled for an overall treatment effect because of differences in the comparators used. Symptom severity was moderate in all three trial populations. No statistically significant differences were found between CPAP and postural therapy (consisting of a backpack with a soft ball inside) on the ESS in patients with positional OSAHS (i.e. AHI while sleeping on the back was two or more times the AHI during sleep in the lateral position) (MD –1.5 (95% CI –2.9 to 0.8). Similarly, there was no statistically significant difference between CPAP and a shoulder–head elevation pillow (p = 0.69 for difference in change)60 or a cervicomandibular support collar61 (p = 0.22 for difference in change) on the Scottish National Sleep Survey Questionnaire. Only overall baseline ESS scores were reported for the last two studies so change scores and the corresponding mean difference could not be calculated.

CPAP versus placebo or conservative/usual care

Outcome data were available from five studies (n = 287) on the MWT. One of these studies used the Osler test. 67 There was a benefit with CPAP compared with placebo/usual care in the length of time participants could stay awake in a setting conducive to sleep (MD 3.3 minutes, 95% CI 1.3–5.3) and this was statistically significant (Figure 5). Statistical heterogeneity was low (I² = 11%) and the treatment effect was consistently in favour of CPAP being beneficial.

FIGURE 5.

Maintenance of Wakefulness Test (CPAP versus placebo), stratified by severity of sleepiness at baseline (ESS).

CPAP versus dental devices

Data were available from two crossover trials (n = 128) on the MWT (Figure 6). In both studies baseline severity of daytime sleepiness (ESS) was classified as moderate. Neither study showed a statistically significant difference between CPAP and dental devices in the length of time participants could stay awake in a setting conducive to sleep (MD 0.7 minutes, 95% CI –1.6 to 2.9). The studies reported consistent findings.

FIGURE 6.

Maintenance of Wakefulness Test (CPAP versus dental devices).

CPAP versus postural therapy

Data were available for the MWT from one small crossover trial108 (n = 13). There was no statistically significant difference between CPAP and postural therapy in the length of time participants could stay awake (MD 1.7 minutes, 95% CI –1.9 to 5.3, p = 0.32).

CPAP versus placebo or conservative/usual care

Outcome data were available from seven trials on the MSLT (n = 331). There was no statistically significant difference between CPAP and placebo/usual care in the length of time it took participants to fall asleep in surroundings conducive to sleep (MD 0.6 minutes, CI –0.7 to 1.9). There was evidence of moderate heterogeneity (I² = 46%) (Figure 7).

FIGURE 7.

Multiple Sleep Latency Test (CPAP versus placebo), stratified by severity of sleepiness at baseline (ESS).

CPAP versus dental devices

No data were available for the MSLT.

CPAP versus postural therapy

No data were available for the MSLT.

CPAP compared with control

The primary outcome of interest in the review was subjective sleepiness. Data were available on the ESS from 23 trials. Overall, CPAP reduced daytime sleepiness by a small amount compared with control; the effect probably varies among different groups of people. The average reduction on the ESS was 2.7 points, but might be anywhere between 2.0 and 3.5 points. There was considerable variation or inconsistency in the treatment effect (statistical heterogeneity); therefore some caution is needed in applying this result to all populations. Variation was reduced when studies were grouped based on baseline symptom severity and there was a trend towards a greater treatment effect with greater baseline symptom severity. It is not surprising that there would be less of a difference between CPAP and control in a population that reports only mild sleepiness at baseline.

In a severely symptomatic population the average reduction on the ESS was 5 points, but might be anywhere between 3.5 and 6.6 points; in a moderate symptom severity population the average reduction was 2.3 points, but might be anywhere between 1.6 and 3.0 points; and in mild severity the average reduction was 1.1 points, but might range anywhere between 0.3 and 1.8 points. When studies were subgrouped by disease severity at baseline, as measured by the AHI, there was a broadly similar trend. Although the definitions of disease and symptom severity used were based on current guidelines, these are arbitrary definitions and interpretation of the results for these subgroups needs to be carried out with that in mind.

The benefit with CPAP compared with control was robust across all the methodological subgroup analyses (trial design, type of data, comparator, quality) and sensitivity analyses investigating the influence of individual trials and use of a fixed-effect model.

Objective sleepiness was assessed using the MWT and MSLT. Data from the MWT were available from five trials. The length of time participants could stay awake in surroundings conducive to sleep as measured by the MWT was greater with CPAP compared with control. The average reduction in sleep latency with CPAP than with control was 3.3 minutes, but might be anywhere between 1.3 and 5.3 minutes. There was a trend towards a greater treatment effect with greater symptom severity, although this analysis was limited by the availability of only one study of a severe symptom severity population and none of a mild population. The benefit with CPAP compared with control was robust across the methodological subgroup analysis (trial design) and sensitivity analyses investigating the influence of individual trials and use of a fixed-effect model. The investigation of methodological factors was limited by the small number of trials available. There was no statistically significant difference between CPAP and control in how quickly participants could fall asleep in surroundings conducive to sleep when seven trials were pooled (MSLT).

CPAP compared with dental devices

Data were available from six trials. In a population with moderate daytime sleepiness at baseline, there was no statistically significant difference between CPAP and dental devices in the impact on daytime sleepiness. The treatment effect probably varies among different groups of people. The average effect was a reduction in sleepiness of less than one ESS point (0.9) with CPAP compared with dental devices, but might be anywhere between an increase in sleepiness of about half a point (0.4) to a decrease in sleepiness of 2.1 points. There was moderate variation in the treatment effect (statistical heterogeneity); therefore some caution needs to be taken in generalising this to all populations. The finding of no statistically significant difference between CPAP and dental devices was robust across subgroup analysis by disease severity (AHI) and trial design and sensitivity analyses investigating the influence of individual trials and use of a fixed-effect model. The investigation of methodological factors was limited by the small number of trials available.

There was no statistically significant difference between CPAP and dental devices in length of time participants could stay awake in surroundings conducive to sleep as measured by the MWT, though this is based on only two studies. No data were available for the MSLT.

CPAP versus placebo/usual care

Daytime mean arterial pressure (using ABPM)

Data were available on daytime MAP for six trials (n = 309). There was an improvement in daytime MAP with CPAP compared with placebo/usual care (MD –2.1 mmHg, 95% CI –4.3 to 0.0) and this was statistically significant (Figure 8). There was moderate statistical heterogeneity (I² = 59%).

FIGURE 8.

Daytime mean arterial pressure using ambulatory blood pressure monitoring (CPAP versus placebo/usual care), stratified by severity of sleepiness at baseline (ESS).

Clinical subgroup analyses

There was some evidence of a variation in treatment effect with severity of sleepiness at baseline (ESS), but this analysis was limited by the small number of trials (see Figure 8); only one trial each of severe and moderate baseline sleepiness was available. The single trial of severely symptomatic patients showed the largest treatment effect in favour of CPAP (MD –4.2 mmHg, 95% CI –6.4 to –2.0) and the difference between CPAP and control was statistically significant (see Figure 8). The difference between CPAP and control was not statistically significant for the moderate subgroup (MD –3.4 mmHg, 95% CI –7.9 to 1.2); the one trial of mild disease severity also reported no statistically significant difference between CPAP and control (MD 1.1 mmHg, 95% CI –2.9 to 5.1) (see Figure 8). Therefore, the overall treatment effect appears to be dominated by the one trial of severely symptomatic patients. When studies were grouped by disease severity at baseline (AHI) the treatment effect was largest with severe disease and there was a statistically significant difference in favour of CPAP; however, only one trial was available of moderate disease and none of mild disease (see Appendix 4, Table 49).

Other subgroup analyses

Studies were subgrouped based on whether they were crossover or parallel and whether end point or change data were used. This analysis was limited by four of the six trials being parallel trials using change data (see Appendix 4, Table 49). For the subgroup of parallel trials using change from baseline data there was a statistically significant improvement with CPAP compared with control. For the other two subgroups consisting of single studies there was no statistically significant difference between groups. The MAP treatment effect ranged from 1.1 mmHg to –3.5 mmHg (see Appendix 4, Table 49).

Sensitivity analyses

When studies were individually removed from the analysis the treatment effect ranged from –1.4 mmHg to –2.7 mmHg and remained statistically significant in only one instance (see Appendix 4, Table 50). There was no substantial change in the MAP results using a fixed-effect model (see Appendix 4, Table 49).

Daytime systolic and diastolic blood pressure (using ABPM)

Data were available on daytime SBP and DBP from seven trials (n = 220). There was no statistically significant difference between CPAP and control for SBP although there was a small decrease in SBP in favour of CPAP (MD –1.1 mmHg, 95% CI –3.4 to 1.2) (Figure 9). There was no evidence of statistical heterogeneity. Similarly, there was no statistically significant difference between CPAP and control for DBP although there was a small decrease in favour of CPAP (MD –1.2 mmHg, 95% CI –2.9 to 0.5); heterogeneity was low (I² = 29%) (Figure 10).

FIGURE 9.

Daytime systolic blood pressure using ambulatory blood pressure monitoring (CPAP versus placebo/usual care).

FIGURE 10.

Daytime diastolic blood pressure using ambulatory blood pressure monitoring (CPAP versus placebo/usual care).

Clinical subgroup analyses

The mean baseline daytime sleepiness (ESS) was not reported for three trials56,63,91 and in the remaining trials the populations were classified as having symptoms of moderate severity at baseline. Therefore, it was not possible to explore the difference in treatment effect with different symptom severity at baseline for SBP or DBP. With the exception of one trial,89 in which the population was classified as having disease of mild severity (AHI), the study populations were all classified as having severe disease. When the single mild disease severity (AHI) study was removed from the analyses for SBP and DBP, the difference between CPAP and control remained not statistically significant (see Appendix 4, Tables 51 and 52).

Other subgroup analyses

The treatment effects in the crossover and parallel subgroups and the end point data and change data subgroups were consistent with each other, i.e. the 95% confidence intervals overlapped. The SBP treatment effect ranged from 1.2 for parallel trials using end point data to –5.2 for parallel trials using change from baseline data although the difference between CPAP and control was not statistically significant in any of the subgroups (see Appendix 4, Table 51). The DBP treatment effect ranged from 0.5 for parallel trials using end point data to –5.7 for parallel trials using change from baseline data although the difference between CPAP and control was not statistically significant in any of the subgroups (see Appendix 4, Table 52). These analyses are limited by the small number of studies in each of the subgroups.

Sensitivity analyses

The finding of no statistically significant difference between CPAP and control for SBP and DBP did not alter when a fixed-effect model was used. For DBP the treatment effect was smaller using a fixed-effect model and there was no substantial change for the SBP results using a fixed-effect model (see Appendix 4, Tables 51 and 52).

The SE for the MD in systolic and diastolic blood pressure for Arias et al. 200556 was imputed based on an estimated within-person correlation of 0.5. The meta-analyses were rerun using an SE based on an assumed within-person correlation of 0.1 and 0.9. For SBP this altered the treatment effect slightly but the finding of no statistically significant difference between CPAP and control did not change: assuming a within-patient correlation of 0.1 gave an SE of 2.75 for the study and the overall pooled treatment effect was –1.2 mmHg (95% CI –3.7 to 1.2); assuming a correlation of 0.9 gave an SE of 1.18 for the study and the overall pooled estimate was –0.6 mmHg (95% CI –2.4 to 1.1). Similarly, for DBP the treatment effect was slightly altered but the finding of no statistically significant difference between CPAP and control did not change: assuming a within-patient correlation of 0.1 gave an SE of 1.48 and the overall pooled treatment effect was –1.3 mmHg (95% CI –3.1 to 0.5); assuming a correlation of 0.9 gave an SE of 0.53 and the overall treatment effect was –1.1 mmHg (95% CI –2.7 to 0.6).

Conventional clinic blood pressure

Three studies (n = 226) used conventional or clinic daytime blood pressure: one study reported waking blood pressure recorded as the mean of three measurements taken at 1-minute intervals between 8 a.m and 11 a.m;62 one study used a similar method to record morning (8–9 a.m.) and evening (8–9 p.m.) blood pressure70 and one provided very little information. 100 The populations in all three studies were classified as having moderate daytime sleepiness at baseline. There was an improvement in daytime SBP (MD –6.62mmHg, 95% CI –9.48 to –3.76) and DBP (MD –3.47mmHg, 95% CI –6.27 to –0.68) with CPAP compared with placebo/usual care and these were both statistically significant (Figures 11 and 12). Statistical heterogeneity was low (I² = 0% and 33% for SBP and DBP respectively).

FIGURE 11.

Daytime conventional systolic blood pressure (CPAP versus placebo/usual care).

FIGURE 12.

Daytime conventional diastolic blood pressure (CPAP versus placebo/usual care).

CPAP versus dental devices

No studies were found that reported daytime ABPM. One study reported morning and evening blood pressure in a population with moderate sleepiness and moderate disease severity at baseline (see Table 2). 70 This study (a parallel trial, n = 68) found no statistically significant difference between CPAP and dental devices in terms of morning SBP (MD –2.9 mmHg, 95% CI –11.0 to 5.2), evening SBP (MD –4.9 mmHg, 95% CI –14.8 to 5.0), morning DBP (MD –1.6 mmHg, 95% CI –7.4 to 4.2) and evening DBP (MD –1.9 mmHg, 95% CI –7.6 to 3.8).

CPAP versus placebo or conservative/usual care

Data were available on night-time MAP from six trials (n = 309). 64,65,68,87,91,109 Overall, there was an improvement in night-time MAP with CPAP compared with placebo/usual care (MD –3.0 mmHg, 95% CI –4.7 to –1.4) and this was statistically significant. There was no statistical heterogeneity. The pooled treatment effects estimated by crossover and parallel trials and end point and change data separately were similar (see Appendix 4, Figure 23).

Data on SBP and DBP were available from seven trials (n = 220). 56,63,64,83,89,109,138 There was no statistically significant difference between CPAP and control for night-time SBP (MD –2.9 mmHg, 95% CI –5.8 to 0.1) or DBP (MD –1.3 mmHg, 95% CI –3.2 to 0.7) (see Appendix 4, Figures 24 and 25).

Data on 24-hour blood pressure were available in one study that did not report daytime and night-time pressure separately (a crossover trial, n = 68 participants). 94 There was a statistically significant benefit with CPAP compared with oral placebo on 24-hour DBP (MD –1.5 mmHg, 95% CI –2.9 to –0.1); there was no statistically significant difference in 24-hour SBP (MD –1.3 mmHg, 95% CI –3.3 to 0.7); or 24-hour MAP (MD –1.0 mmHg, 95% CI –2.6 to 0.6).

CPAP versus dental devices

One study reported 24-hour blood pressure (crossover trial, n = 80) in a population with moderate symptom severity and moderate disease severity at baseline. 82 There was no statistically significant difference between CPAP and dental devices for 24-hour SBP (MD 0.6 mmHg, 95% CI –2.5 to 3.7) or DBP (MD 0.4 mmHg, 95% CI –1.7 to 2.5).

CPAP versus control

Six trials reported MAP using ABPM. There was a statistically significant reduction in MAP with CPAP compared with control; the size of the effect probably varies among different groups of people. The average reduction in MAP was 2.1 mmHg, but might be anywhere between no reduction and 4.3 mmHg. There was moderate inconsistency in the treatment effect (statistical heterogeneity), but due to the small number of studies it was not possible to adequately investigate sources of this variation. Only one study was available of severely symptomatic patients and the overall treatment effect did seem to be dominated by this trial. There was no substantial change in the MAP results when a fixed-effect model was used as a sensitivity analysis. However, when individual studies were removed from the analysis the treatment effect remained statistically significant in only one instance, indicating a possible lack of statistical power due to the small number of participants.

There was no statistically significant difference between CPAP and control for SBP or DBP (measured using ABPM). The treatment effect probably varies among different groups of people. The average effect for SBP was a decrease of 1.1 mmHg with CPAP, but might be anywhere between a decrease of 3.4 mmHg and an increase of 1.2 mmHg compared with control. The average effect for DBP was a decrease of 1.2 mmHg with CPAP, but might be anywhere between a decrease of 2.9 mmHg and an increase of 0.5 mmHg compared with control. There was no inconsistency in the treatment effect (statistical heterogeneity). It was not possible to investigate whether the treatment effect varied with disease or symptom severity at baseline due to limitations in the data available. When a fixed-effect model was used, the findings were not substantially altered, except that the treatment effect for DBP was smaller. The pooling of three studies reporting conventional clinic blood pressure showed a large and statistically significant improvement in SBP and DBP with CPAP compared with control.

The results for night-time blood pressure were similar to those for daytime blood pressure. There was a statistically significant improvement in night-time MAP (using ABPM) but not SBP and DBP. The magnitude of the effects was broadly similar.

CPAP versus dental devices

Only one study was available that reported daytime blood pressure (morning and evening blood pressure using a conventional clinic method). This trial of a moderate symptom severity and moderate disease severity population found no statistically significant difference between CPAP and dental devices. Another trial that did not report daytime and night-time blood pressure separately reported no statistically significant difference in 24-hour SBP and DBP between the two interventions.

CPAP versus placebo or usual care

CPAP versus dental devices

Data were available from a small number of studies comparing CPAP with dental devices. Where reported, the studies were all of moderate symptom severity populations at baseline.

CPAP versus postural therapy

Data were available from two small crossover trials (n = 23). 60,61 The studies were not pooled for an overall treatment effect due to differences in the comparators used, i.e. a shoulder–head elevation pillow60 and a cervicomandibular support collar. 61

CPAP versus control

Six studies reported the SF-36 subscales. There was no statistically significant difference between CPAP and control on any of the SF-36 subscales. There was moderate to high variation or inconsistency in the treatment effect (statistical heterogeneity) for most of the subscales, therefore some caution needs to be taken in generalising these findings to all populations receiving CPAP. Although the treatment effects from the crossover and parallel trial subgroup analysis were consistent with each other, in that their 95% confidence intervals overlapped, for bodily pain, general health and physical function there was a statistically significant benefit with CPAP compared with control for the parallel trials but not for the crossover trials. This may have been driven by two of the parallel trials being of severe symptom populations. In contrast, on the other generic scale, the NHP Part 2, there was a statistically significant benefit with CPAP compared with control based on a pooling of four studies. The treatment effect probably varies among different groups of people. The average effect was a reduction of 1.7 points with CPAP compared with control, but might fall anywhere between 0.5 and 2.9 points. There was no variation or inconsistency (statistical heterogeneity) in the treatment effect.

The findings from the disease-specific measures were also somewhat contradictory, although only a small number of studies were available. On the FOSQ (four trials), a disease-specific measure, there was a statistically significant benefit with CPAP compared with control for the activity level and social outcome subscales but not for general productivity, intimacy and sexual activity, vigilance or total score. Only two studies reported the SAQLI total score; one reported a significant benefit with CPAP compared with control and for one there was no statistically significant difference.

CPAP versus dental devices

For the majority of the quality of outcome measures only single studies were available. There was no statistically significant difference between CPAP and dental devices when two studies reporting the FOSQ and two studies reporting the SAQLI were pooled. Three studies reported the SF-36 but all used different scores and the findings were not consistent.

CPAP versus placebo/usual care

CPAP versus dental devices

CPAP versus control

Data were available for three psychological outcome measures from two or more studies. There was no statistically significant difference between CPAP and control on the GHQ-28 or the HADS. There was a statistically significant benefit in favour of CPAP compared with control on the energetic arousal score from the UMACL. There was no inconsistency (statistical heterogeneity) in the treatment effect.

CPAP versus dental devices

There was no statistically significant difference between CPAP and dental devices in one trial reporting the HADS.

Cognitive outcomes

Eighteen trials used formal testing to measure the effects of CPAP on cognitive function in adults with obstructive sleep apnoea. Ten of the studies used a crossover design,72,78,79,82,89,90,92,93,103,108 while eight used a parallel group design. 58,73,77,83,85,100,102,113 Six trials compared CPAP with sham CPAP,58,73,77,79,83,85 six trials with oral placebo,78,82,89,90,92,93 four trials with dental devices,72,82,102,103 two trials with conservative treatment100,113 and one trial with postural therapy. 108 Most of the studies included small sample sizes (range 14–125), with three studies reporting on a subgroup of the original randomised population. 77,113,114 Based on mean ESS score at baseline (where reported), the majority of trials were of reported moderate symptom severity populations, two trials were of severely symptomatic populations and one trial studied mild symptom severity. Based on mean baseline AHI, seven study populations were classified as having severe disease,58,73,83,85,93,103,108 seven as having moderate disease77,79,82,90,100,102,113 and three as having mild disease. 78,89,92

A total of 28 different cognitive tests were used, examining several areas of cognition (administered as verbal, pen-and-paper or computer-based tasks), making comparisons across trials difficult. The areas of functioning assessed were attention or vigilance, psychomotor function, construction, verbal fluency, IQ decrement, memory and learning (see Table 12). Seventeen tests were used by two trials or fewer; even when tests were used by multiple trials the scales used were not always uniform.

Testing protocols may have a confounding effect on performance; therefore assessment procedures were examined. Some variation between testing protocols existed (see Appendix 4, Table 54). Testing protocol issues include order of test presentation, which is a particular issue with test batteries containing many different types of test,115 and time of day when fluctuations in performance and levels of alertness can occur in response to circadian rhythms. In addition, performance may be improved by prior exposure to testing stimuli and procedures, which can have a significant beneficial impact on test performance when tests are administered on more than one occasion. Use of stimulants, such as nicotine and caffeine, can also modify cognitive performance, as can mood and depression. Therefore, ideally, testing protocols should employ measures to minimise risks of possible confounding, or account for potential biases in the analysis.

Nine trials administered a familiarisation session prior to baseline assessment,78,82,85,89,90,92,108,114,116–118 and four trials used alternative test forms in subsequent sessions in an attempt to minimise learning effects. 85,90,92,116 Thirteen trials reported the time of day that assessments were conducted (five in the afternoon, two in the morning, and six across the course of the day). 78,79,85,89,90,92,100,108,113,114,118 Eight trials reported administering tests in a standardised order in an attempt to control the impact of each test in relation to each other across the test session. 78,79,90,92,93,108,114,118 Four trials assessed for, or attempted to minimise, the effects of stimulants such as caffeine or nicotine, or the effects of alcohol consumption or drug intake. 83,90,108,114 No study specifically looked at the effect of mood in relation to cognitive function, although one trial82 stated that significant depression was present in 40% of the included participants. Level of baseline function, compared with normative standards, was reported in only one trial. 89 However, a number of papers79,83,100,113,117 indicated that many participants demonstrated normal values at baseline, highlighting the possibility of a ceiling effect.

All of these issues could affect the findings of the studies and should be considered when interpreting the results reported below. Owing to time limitations and the quantity of cognitive data from crossover trials it was not feasible to impute data for a paired analysis, where these were not reported, for all the cognitive outcomes. Where three or more studies were available for potential pooling the SE was estimated where appropriate data were available. A narrative synthesis was used where pooling was not feasible. Details of the individual study results are reported in Appendix 4, Table 53. Where end point data were not reported, change scores were used.

CPAP versus placebo or conservative/usual care

Simulated driving task

Seven studies used a simulated driving task. 78,83,90,92,93,100,118 Daytime sleepiness, based on ESS scores at baseline, varied between study populations; in one study daytime sleepiness was classified as severe,118 in one it was classified as mild83 and in four it was classified as moderate. 78,92,93,100

Six of the seven studies used the SteerClear simulated driving test. 78,83,90,92,93,100 SteerClear is a computerised program that attempts to mimic different components of attention involved in driving a car; the program simulates a long and monotonous highway drive that presents a number of obstacles over a period of 30 minutes.

Two studies reported performance in terms of the percentage of obstacles hit83,100 and four reported the number of obstacles hit. 78,90,92,93 These were treated separately.

There was no statistically significant difference between CPAP and oral placebo in terms of the percentage of obstacles hit (Figure 13), or in the number of obstacles hit (MD –5.74, 95% CI –14.75 to 3.27) (Figure 14). There was no statistical heterogeneity (I² = 0%) for trials reporting the number of obstacles hit.

FIGURE 13.

SteerClear (CPAP versus placebo/usual care), percentage of obstacles hit.

FIGURE 14.

SteerClear (CPAP versus placebo/usual care), number of obstacles hit.

One parallel group trial118 used a different simulated driving test, based on the work of Land and Horwood. 119 This computerised program presents a white on black image (as in night driving) of the moving edges of the road with an image of the vehicle bonnet at the bottom of the screen. The primary object is to steer the centre of the vehicle as accurately as possible down the middle of the road for 30 minutes. In addition, single digits are displayed at each corner of the screen (digits change randomly at an interval of 8–10 seconds) and the participant is required to identify target digits by pressing a button on either side of the steering wheel. Baseline ESS was classified as moderate in both groups. An improvement in terms of steering performance was found with CPAP compared with sham CPAP, although not all differences were statistically significant: SD of position on road (median difference –0.1, p = 0.08), SD of deterioration (median difference –0.2, p = 0.007), and length of drive (minutes) (median difference –0.3, p = 0.08). This was based on end point data as per the protocol; when difference in change was considered, a significant difference in favour of CPAP was found for SD position on road (p = 0.03) and length of drive (p = 0.02).

Trail Making Task (TMT)

The TMT is a task of complex attention given in two parts, A and B. Individuals are asked to draw lines to connect consecutively numbered circles on one work sheet (Part A), and then connect the same number of consecutively numbered and lettered circles on another work sheet, alternating between the two sequences (Part B); time taken to complete the task (seconds) and errors made are typically recorded. The test is sensitive to a range of mental processes including speed of processing and mental flexibility.

Eight studies (five parallel, three crossover, n = 260) reported on TMT Part A78,83,85,89,90,100,117,341 and 12 (six crossover, six parallel, n = 406) reported on TMT Part B. 78,82,83,85,89,90,92,93,100,113,117,341

TMT Part A

The severity of daytime sleepiness was classified as moderate in four studies,78,89,100,341 and in one study each was classified as mild83 and severe85; two studies did not report symptom severity. 58,90 None of the studies showed a significant difference between CPAP and control in the length of time it took to complete TMT Part A (Table 6). None of the studies reported adequate allocation concealment, and it was unclear whether groups were similar at baseline in four of the included studies. 78,89,90,100

TABLE 6 - Summary of TMT Part A data reported in included studies

NR, not reported.

Study	CPAP [mean (SD)]	Control [mean (SD)]	MD	p-Value
Crossover
Barnes et al., 200289	28.1 (NR)	27.6 (NR)	0.5	Not significant (precise p-value not reported)
Engleman et al., 199490	NR	NR	NR	Not significant (precise p-value not reported)
Engleman et al., 199978	26 (11)	29 (11)	–3.0	0.06
Parallel
Barbé et al., 200183	47 (NR)	47 (NR)	0	> 0.20
Dimsdale et al., 200058,117	27.4 (NR)	27.4 (NR)	0	NR
Henke et al., 200185	Only available in graph			Not significant (precise p-value not reported)
Monasterio et al., 2001100	49 (19)	49 (20)	0	0.76
Norman et al., 200673,341	26.5 (NR)	21.7 (NR)	4.9	0.49 (relates to time × treatment interaction for three treatment groups)

Five trials (three crossover, two parallel) did not report sufficient data to calculate a variance. 78,85,89,90,341 Therefore, only three of the eight trials reporting TMT Part A were used to generate a pooled estimate of treatment effect and are displayed on the forest plot below (Figure 15). When data from these three parallel group trials (n = 215) were pooled, no statistically significant between-group difference was found (MD 0.0, 95% CI –2.5 to 2.5). There was no statistical heterogeneity (I² = 0%). Given that only a proportion of the available studies could be pooled, caution needs to be taken in interpreting the pooled effect.

FIGURE 15.

Trail Making Task Part A (CPAP versus placebo/usual care), stratified by type of data.

TMT Part B

Where reported, the populations in the majority of trials were classified as having moderate baseline daytime sleepiness,78,82,89,92,93,100,341 and in one study each as having mild83 and severe85 daytime sleepiness. One trial reported a significant difference in favour of CPAP compared with oral placebo in the length of time taken to complete the task; no statistically significant between-group differences were found in the other trials (Table 7). Half of the studies did not report adequate allocation concealment, and it is unclear if the trials had sufficient power to detect a treatment effect.

TABLE 7 - Summary of TMT Part B data reported in included studies

NR, not reported.

Study	CPAP [mean (SD)]	Control [mean (SD)]	MD	p-Value
Crossover
Barnes et al., 200289	60.1 (NR)	65.2 (NR)	–5.1	Not significant (p-value not reported)
Barnes et al., 200482	73.3 (29.5)	74.2 (32.2)	–0.9	Not significant (p-value not reported)
Engleman et al., 199490	66 (28.3)	76 (28.3)	–10	0.02
Engleman et al., 199792	64.1 (22)	77.7 (36.8)	–13.6	0.02
Engleman et al., 199893	69 (32)	68 (32)	1.0	Not significant (p-value not reported)
Engleman et al., 199978	63 (33)	65 (27)	–2	Not significant (p-value not reported)
Parallel
Barbé et al., 200183	96 (32.3)	110 (50)	–14.0	0.10
Dimsdale et al., 200058,117	71.2 (31.8)	87 (34.8)	–15.8	Not significant (p-value not reported)
Henke et al., 200185	Only available in graph			Not significant (p-value not reported)
Lojander et al., 1999113	130, median	75, median		NR
Monasterio et al., 2001100	106 (42)	100 (39)	6.0	0.15 (difference in change based on median values)

Seven trials did not report sufficient data or used different scales; therefore, only data from two crossover90,93 and three parallel83,100,117 trials were pooled (n = 328) for TMT Part B (Figure 16). There was a statistically significant benefit with CPAP compared with control for time (seconds) taken to complete TMT Part B (MD –9.1, 95% CI –14.9 to –3.1). There was low statistical heterogeneity (I² = 34%). However, as only a proportion of the studies were pooled, the treatment effect may not be generalisable.

FIGURE 16.

Trail Making Task Part B (CPAP versus placebo/usual care), stratified by type of data.

Wechsler Adult Intelligence Scale (WAIS) Digit Symbol Substitution Test (DSST)

Ten studies used the DSST, a test of complex attention (five parallel and five crossover, n = 488). 73,78,82,83,85,89,90,93,100,117 In this test of attention and processing, respondents are given a code table displaying the correspondence between pairs of digits (from 1 to 9) and symbols, and then asked to fill in blank squares with the symbol that is paired to the digit displayed above the square. Six out of the eight studies reporting daytime sleepiness coud be classified as moderate78,82,89,93,100,341 and one trial each classified the study population as having mild83 and severe85 daytime sleepiness. Two trials found a significant benefit of CPAP compared with control in the number of correct responses,78,90 and one trial found a significant difference in change from baseline in favour of placebo;89 no significant between-group differences were found in the other trials (Table 8). None of the trials reported adequate allocation concealment.

TABLE 8 - Summary of Wechsler Adult Intelligence Digit Symbol Substitution Test data reported in included studies

NR, not reported.

Study	CPAP [mean (SD)]	Control [mean (SD)]	MD	p-Value
Crossover
Barnes et al., 200289	47.3 (NR)	48 (NR)	–0.7	0.07 (difference in change)
Barnes et al., 200482	47.3 (3.6)	46.8 (3.6)	0.5	Not significant (p-value not reported)
Engleman et al., 199490	52 (11.3)	51 (11.3)	1	0.05
Engleman et al., 199893	52 (13)	52 (14)	0	Not significant (p-value not reported)
Engleman et al., 199978	59 (12)	57 (14)	2	0.0004
Parallel
Barbé et al., 200183	43 (16.2)	47 (20)	–4	> 0.20 (difference in change)
Dimsdale et al., 200058,117	53.2 (11.2)	53.5 (12)	–0.3	Not significant (p-value not reported)
Henke et al., 200185	Only available in graph			A binary variable of improved or not improved was assessed; not significant (p-value not reported)
Monasterio et al., 2001100	9 (3) scaled score	9 (2)	0	0.97 (difference in change, based on median values)
Norman et al., 200673,341	73.8 (NR)	68.7 (NR)	5.1	0.26 (based on time × treatment interaction for three-armed trial)

Four studies did not provide sufficient data to calculate a variance and one trial used a different scale; therefore, only five trials (three crossover and two parallel, n = 170) were pooled (Figure 17);78,83,90,93,117 no statistically significant benefit with CPAP was found compared with control in terms of the number of correct responses (MD 0.2, 95% CI –0.6 to 1.0). There was no statistical heterogeneity (I² = 0%).

FIGURE 17.

Wechsler Adult Intelligence Digit Symbol Substitution Test (CPAP versus placebo/usual care), stratified by type of data.

Paced Auditory Serial Addition Task (PASAT)

Seven studies used the PASAT test of vigilance. 78,82,83,90,92,93,100 In this computerised task, a series of digits are presented at a set rate and the respondent is asked to add the numbers in pairs, such that each number is added to the one that immediately precedes it. Presentation rates range from 1 to 4 seconds. Different formats have been developed, for example the PASAT 1.2 and the PASAT 2.4, which are thought to be more difficult than the standard PASAT 1 and PASAT 2. One study reported outcomes for the PASAT 1.2,82 six studies for the PASAT 2,78,83,90,92,93,100 one study for the PASAT 2.4,82 two studies for the PASAT 3,83,100 and two studies for the PASAT 4. 83,90

Daytime sleepiness was reported for six trial populations; five populations were classified as moderate78,90,92,93,100 and one trial as mild. 83 One crossover trial78 found a significant benefit in favour of CPAP in the number of correct responses made; no statistically significant between-group differences were found in any of the other trials (Table 9). None of the studies reported adequate allocation concealment, and one study reported a significant treatment by period interaction for the PASAT 290 indicating a potential carryover effect of treatment.

TABLE 9 - Summary of PASAT data reported in included studies

PASAT, Paced Auditory Serial Addition Task.

Study	CPAP [mean (SD)]	Control [mean (SD)]	MD	p-Value
Crossover
Barnes et al., 200482	PASAT 1.2: 2.9 (0.9)	3.4 (0.9)	–0.5	Not significant (precise p-value not reported)
	PASAT 2.4: 3.8 (1.8)	3.7 (0.9)	0.1	Not significant (precise p-value not reported)
Engleman et al., 199490	NR	NR	NR	Not significant (precise p-value not reported)
Engleman et al., 199792	PASAT 2: 37.8 (13.2)	35.3 (11.2)	2.5	Not significant (precise p-value not reported)
Engleman et al., 199893	PASAT 2: 37 (11)	35 (11)	2	Not significant (precise p-value not reported)
Engleman et al., 199978	PASAT 2: 40 (11)	36 (14)	4	0.02
Parallel
Barbé et al., 200183	PASAT 1: 15 (5.4)	15 (5)	0	> 0.20 (difference in change)
	PASAT 2: 16 (5.4)	15 (5)	1	0.04
	PASAT 3: 12 (5.4)	12 (5)	0	0.09
	PASAT 4: 5 (5.4)	5 (5)	0	> 0.20
Monasterio et al., 2001100	PASAT 1: 5 (4)	5 (3)	0	0.32 (based on data for median values)
	PASAT 2: 12(4)	12 (4)	0	0.12
	PASAT 3: 15 (4)	15 (4)	0	0.20
	PASAT 4: 14 (4)	16 (4)	–2	0.20

Data from three or more trials were available for PASAT 1 and PASAT 2. Of the three studies reporting PASAT 1, two studies82,83 did not provide sufficient data to calculate a variance for pooling, and, of the six studies reporting the PASAT 2, two studies did not provide sufficient data to calculate a variance. 90,92 Two crossover and two parallel trials were therefore pooled (n = 234) for PASAT 2 (Figure 18). No statistically significant benefit was found with CPAP compared with control for number of correct responses made (MD 2.30, 95% CI 0.24–4.37); statistical heterogeneity was low (I² = 25%).

FIGURE 18.

Paced Auditory Serial Addition Task (PASAT) 2 (second presentation rate) (CPAP versus placebo/usual care), stratified by type of data.

CPAP versus dental devices

Cognitive outcomes were reported in four studies (two crossover and two parallel group trials, n = 160). 72,82,103,114 Where reported, symptom severity was classified as moderate, and in all but one trial103 disease severity was also reported as moderate. There were no statistically significant differences between treatment groups on any of the cognitive tests assessed (see Appendix 4, Table 53). None of these trials reported appropriate allocation concealment, and only one trial82 reported adequate randomisation methods and used intention-to-treat analysis.

CPAP versus postural therapy

Data were available for one small crossover study (n = 14). 108 No statistically significant between-group differences were found on any of the cognitive tests administered (see Appendix 4, Table 53). It is unclear whether performance differences at baseline existed, or whether the study was appropriately powered to detect an effect.

CPAP versus placebo/usual care

Nine studies reported the AHI at follow-up. 66,70,73,82,85,87,97,100,109 There was high statistical heterogeneity (I² = 97%) and any pooled effect is likely to be meaningless. All the studies reported a statistically significant reduction in the AHI with CPAP compared with placebo/usual care and the effect size ranged from –9.2 (95% CI –18.3 to –0.1) to –60.0 (95% CI –72.1 to –47.5) (see Appendix 4, Figure 44).

CPAP versus dental devices

Nine studies reported the AHI at follow-up. 70,80–82,103,105–107,119 There was a statistically significant reduction in AHI in favour of CPAP compared with dental devices (MD –8.4, 95% CI –10.5 to –6.3) (see Appendix 4, Figure 45). Statistical heterogeneity was low to moderate (I² = 40%).

CPAP versus postural therapy

Data were available for the AHI from three small crossover trials (n = 36). 60,61,108 There was a statistically significant benefit with CPAP on the AHI compared with postural therapy (shoulder–head elevation pillow, MD 15.5, p = 0.008; cervicomandibular support collar, MD in change 16.8, p = 0.001; backpack with soft ball inside, MD 6.1, 95% CI 2.0–10.2, p = 0.007).

CPAP versus postural therapy

Data were available from two trials (n = 23). 60,61 No statistically significant difference in the overall number of self-reported adverse events was found when CPAP was compared with a shoulder–head elevation pillow (MD –0.8, p = 0.16). However, there were significantly fewer self-reported adverse events with a cervicomandibular support collar than with CPAP (MD in change 4.2, p = 0.01). Type of adverse event and indication of perceived severity were not reported in either study.

ResMed model120

Overview

ResMed performed a cost–utility analysis comparing CPAP using fixed pressure and CPAP using autotitrated pressure with a ‘do nothing’ alternative for the treatment of patients with severe OSAHS. 120 The hypothetical patient population depicted by the model consisted of 55-year-old patients with severe OSAHS as defined by an AHI > 30 and daytime sleepiness represented by a score of 12 on the ESS. The analysis was undertaken from the NHS and PSS perspective.

ResMed produced a cohort-based Markov model with a 14-year time horizon and each Markov cycle lasted a year. 120 Patients enter the model following an initial outpatient visit or a diagnostic sleep study test. Treatment begins 8.4 months after whichever visit takes place first. For each year in the model patients can remain event free in the severe OSAHS state, or can have a non-fatal or fatal stroke, a cardiovascular event (CVE), e.g. myocardial infarction or a road traffic accident (RTA), as illustrated in Figure 19. In each subsequent year patients who have had a non-fatal CVE or an RTA can have a stroke, CVE or RTA. However, patients who have a stroke can no longer drive and, therefore, are not at subsequent risk of an RTA.

FIGURE 19.

Structure of ResMed Model (adaptation of Figure 6.1, p. 9, v in ResMed submission).

The primary measure of cost-effectiveness was incremental cost per quality-adjusted life-year (QALY) gained, and the secondary measure was cost per life-year gained. The QALY estimate incorporated the impact on health-related quality of life (HRQoL) of stroke, CVE and RTA. Effectiveness estimates and utilities were drawn from the HRQoL published literature and government statistics, and based on the authors’ assumptions. Data on patient management and resource use were obtained from 19 clinicians throughout the UK who had relevant clinical experience. Unit cost data on CPAP treatment, and resource use associated with CVEs and RTAs, were obtained from list prices, the published literature and government statistics. The authors undertook several univariate sensitivity analyses and probabilistic sensitivity analysis to test the robustness of findings.

Summary of effectiveness data

For the base-case analysis, utility values for treated and untreated OSAHS were obtained from Mar et al. 123 In this study, a survey of 51 OSAHS patients who attended a sleep clinic in Spain was undertaken before the initiation of CPAP and 3 months after initiation of CPAP in order to generate ‘do nothing’ and nasal CPAP (nCPAP) utility values, respectively. The EQ-5D instrument (EuroQoL Group127) was used to describe patient health states and completed data for 46 patients were obtained. These were then elicited using the time trade-off technique and valued based on UK societal preferences (Dolan et al. 128). No information was obtained on the HRQoL of OSAHS patients with stroke and coronary heart disease (CHD). To estimate these utilities, the authors assumed quality adjustment factors of 0.8 and 0.9 in relation to standard OSAS patient utilities (Table 14, based on Torrance et al. 129) To estimate utility associated with a non-fatal RTA, ResMed took the average utility for OSAHS and a non-fatal CVE in treated and untreated patients.

The annual incidence rates of fatal and non-fatal cardiovascular and cerebrovascular events in patients with severe OSAHS (AHI > 30) were calculated for CPAP-treated and untreated patients using the results of a long-term observational study by Marin et al. 130 The untreated patients comprised those who had refused CPAP treatment on initial referral to the sleep clinic. The baseline characteristics of these patients may differ compared with the treated patients for reasons other than chance, thus undermining the internal validity of the study results. Results were extrapolated from years 12–14. No method of extrapolation was reported. The authors justified the use of a 14-year time horizon because 14 is divisible by seven: the estimated NHS shelf-life of CPAP according to the authors. ResMed used the Mar et al. 123 study to estimate the ratio of CHD and stroke in patients with untreated severe OSAHS as 1.185 and 1.353 respectively compared with treated OSAHS. Therefore, they estimated the ratio of developing CHD to stroke as 1 : 1.13. Based on the same data, ResMed estimated the ratio of CHD to stroke in treated patients as 1 : 1. Thus, treatment with CPAP was assumed to reduce the incidence of CHD and stroke, and to reduce the proportion of total CHD and stroke events. Using these estimates, ResMed calculated the annual risk of CVE and stroke.

To estimate the risk of an RTA, ResMed took the average risk increase of an RTA in patients with OSAHS based on two studies. One study assessed RTAs in patients with OSAHS before and after treatment with CPAP (George et al. ). 131 Patients were followed-up for at least 3 years. The other study (Mazza et al. 132) measured driving ability in OSAHS patients before and after CPAP treatment using a ‘road safety platform’ (i.e. a stretch of road to test driving ability). The risk of an RTA was estimated as 2.6 times greater than the risk among controls, whereas the risk among treated patients was assumed to be equivalent to that among controls. Using data from the Department of Transport133 and assuming that all OSAHS patients were drivers of a licensed motor vehicle, ResMed estimated that the risk of an RTA among the control group and treated OSAHS patients was 0.009 per year and 0.023 per year for untreated OSAHS patients.

ResMed reviewed the published literature to obtain data on compliance among OSAHS patients with fixed CPAP. Compliance was defined as the percentage of patients with OSAHS of all severity levels who have not discontinued using their CPAP device. ResMed estimated that 79% of patients would continue to use CPAP after the first year of treatment, based on the results of four studies that followed patient compliance for at least 1 year. 39,40,134,135 It took the average compliance across the studies, the follow-up time for which varied between 2 and 7 years. For patients who continued their use of the device for at least 1 year, it was assumed that there would be no further loss to compliance, based on expert clinical opinion. For autotitrating CPAP it was estimated that compliance would be 84%. The increase in compliance with autotitrating CPAP compared with fixed CPAP was based on expert opinion.

Summary of resource utilisation and cost data

The opinion of 19 clinical experts was sought in order to estimate the health-care resource use associated with the management of OSAHS in the UK. Resource use and unit costs were reported separately and these are detailed in Tables 15 and 16 respectively. Unit costs were reported in 2005 prices and were based on list prices, ResMed estimates, the published literature and government statistics. Costs were calculated by multiplying the resource use by the relevant unit costs (Table 17). Confidence intervals were calculated based on the resource use estimates provided by clinical experts.

Examination of the electronic model submitted by ResMed revealed a number of modelling errors that may have affected the ability of the model to provide an accurate estimate of mean costs and QALYs. The beta distributions used to characterise the uncertainty around transition probabilities and utilities were mis-specified. The alpha and beta parameters were correctly estimated from the mean and SD, but the scale parameter was set equal to the mean, effectively truncating the distributions at the mean. This meant that for the probabilistic analysis the mean and SD of every transition probability was lower than indicated by the source data. In addition, a number of other distributions were truncated; for example, the relative risk of an RTA was modelled as a normal distribution with mean = 2.6, SD = 0.26, and truncated at a lower limit of 2.4 and an upper limit of 2.9. The reason for the truncation is unclear, and it effectively reduces the uncertainty around the model input parameters. The uncertainties around resource use and cost data were characterised using normal distributions, which allow negative values to be drawn for simulations in the probabilistic analysis.

The numbers of CHD and stroke events were calculated as a proportion of the number of patients alive in the first year of the model for every cycle rather than as a proportion of those patients at risk at a given time point. This led to an overestimate of the number of events, as patients who die as the model progresses are not removed from the pool of those at risk.

The proportion of males in the hypothetical patient population was modelled with uncertainty. Risks that differed according to gender, such as mortality risks, were averaged according to the proportion of males and females at the start of the model. Averaging over heterogeneous subgroups in this manner can produce misleading results. For example, as the risk of death is greater among men, the proportion of men in the hypothetical patient population would be expected to fall over time. By not reflecting this in the model the number of deaths will be overestimated.

A number of other minor errors were also found. The lack of internal validity indicates that the model results should be interpreted with caution.

Summary of cost-effectiveness

The expected outcomes associated with severe OSAHS at 14 years from commencement of treatment with CPAP compared with no treatment are provided in Table 18. Based on the Markov model, differences in health gain between patients receiving CPAP and patients who are untreated becomes apparent after 2–3 years of treatment.

The primary cost driver in patients with severe OSAHS was managing stroke [i.e. 68% of the total cost for the no treatment group, 48% for the CPAP (fixed) and 45% for CPAP (auto) groups]. The secondary cost driver in untreated patients was the cost associated with managing RTAs (i.e. 23% of the total cost for the no treatment group), whereas in CPAP-treated patients it was the cost associated with the device itself [i.e. 22% for the CPAP (fixed) and 26% for CPAP (auto) groups].

Key results of the cost-effectiveness model are shown in Table 19. The incremental cost-effectiveness ratio (ICER) was based on deterministic analysis and over 14 years it was estimated that CPAP dominates no treatment (i.e. CPAP was associated with lower costs and higher effects than no treatment). However, at 1 year, the cost per QALY for CPAP is expected to exceed £20,000. After 2 years the expected cost per QALY gained is £10,000 or less, and after 11 years CPAP is the dominant treatment. CPAP (fixed) was compared with no treatment and CPAP (auto) was compared with no treatment. Based on this indirect analysis, the authors found that CPAP (auto) dominated CPAP (fixed).

Several univariate sensitivity analyses were undertaken and demonstrated that the cost-effectiveness of CPAP was robust to changes in some input values. However, the model was sensitive to the following parameters: time to start of treatment, compliance rate with CPAP, risk of having a cardiovascular event or a cerebrovascular event, risk of having an RTA, utility for treated and untreated OSAHS, cost of managing a non-fatal RTA and cost of managing stroke rehabilitation.

The results of the univariate sensitivity analyses are reported in Table 20. No rationale was provided for the ranges over which input values were varied.

Cost-effectiveness acceptability curves (CEACs) were presented for CPAP (auto) and CPAP (fixed). Each was assessed in a pair-wise comparison against a ‘do nothing’ alternative. The CEAC showed that CPAP (auto) has a marginally higher probability of being cost-effective compared with a ‘do nothing’ option than CPAP (fixed) compared with a ‘do nothing’ option at a willingness to pay threshold of less than £5000 per QALY in all simulations.

Comments on methodology

Overview

Ayas et al. 122 performed a cost–utility analysis comparing CPAP with a ‘do nothing’ alternative for the treatment of patients with moderate to severe OSAHS. The base-case analysis was patients aged between 25 and 54 years who were newly diagnosed with moderate to severe OSAHS, classified as having an AHI ≥ 15 events per hour. The analysis was undertaken from the US third-party payer perspective and the societal perspective.

The authors developed a Markov model with a 5-year time horizon. Each Markov cycle lasted 1 year. The primary outcome measure used in the analysis was incremental cost per QALY. The QALY estimate for CPAP incorporated the expected gains in HRQoL due to a reduction in RTAs. Effectiveness, utility and resource use estimates were obtained from the published literature and administrative databases. A number of univariate sensitivity analyses were undertaken. In addition, the authors undertook a probabilistic sensitivity analysis.

Summary of effectiveness data

For the base-case analysis, QALY estimates were obtained using utilities valued based on the standard gamble, in patients with OSAHS, pre and post CPAP therapy (Chakravorty et al. 97) Therefore, patient preferences rather than societal preferences were used for the valuations. Overall, a weighted average of the findings was obtained for patients by age group and sex. These were adjusted for the impact of CPAP on RTAs and the impact of death due to natural causes, as explained below.

Estimates of effect were calculated based on the proportion of patients in one of six patient groups (i.e. ages 25–34, 35–44, 45–54 by sex), reflecting the distribution found in a sample of 99 patients with OSAHS: a distribution comparable to that in the United States. The probabilities of RTAs were stratified by the relevant patient groups. An RTA could either result in a fatality or property damage only or be injury related. The injury-related RTAs were stratified into one of five levels on the Maximum Abbreviated Injury Scale (MAIS), with scores ranging from one (minimal injury) to five (most severely injured). Of the patients having an RTA, the severity of injury was estimated as 85.6%, 10.5%, 3.3%, 0.4% and 0.2% for severity levels 1–5. RTA survivors in level 5 MAIS state were assumed to be unable to drive again and therefore were confined to a tunnel state in the model.

The effect of CPAP on RTAs was based on a random-effects meta-analysis of eight observational studies in which actual RTAs were observed in patients pre and post initiation of CPAP (Table 21). 131,136–142 It excluded driving simulator studies. A pooled reduction in RTA risk and an improvement in HRQoL due to CPAP were calculated by a random-effects meta-analysis using the inverse variance of the logarithm of the odds ratio. It was assumed that the RTA crash rate in OSAHS patients who received CPAP equalled that in the general population. The RTA rates in OSAHS patients who were not receiving CPAP were obtained taking the general population RTA rates and dividing these by the proportionate reduction in RTAs associated with CPAP therapy.

Based on one study (Chakravorty et al. 97) it was assumed that the average utility in patients receiving CPAP was 0.55, an increase in utility of 0.23 compared with baseline (no CPAP). The utilities were valued using the standard gamble. Quality weights for the five MAIS injury levels were obtained using the Functional Capacity Index (FCI), which used rating scale preferences from patients who had functional limitations exceeding 1 year. The FCI weights were applied using similar methods to those of Graham et al. 143

The transition probability of death was estimated by applying the yearly, sex-specific probability of an RTA death to the rate of death due to natural causes, based on US life tables.

Summary of resource utilisation and cost data

The base-case analysis included third-party payer, direct medical costs only. For the first year, the total cost of CPAP was calculated using the US Medicare fee schedule. For year 2 and year 5, ongoing annual CPAP cost components were included. The costs included those of the device and the time of medical and emergency specialists. For the first 15 months, rental fees were applied based on Medicare guidelines. Following this, patients incurred a rental fee. The CPAP machine was assumed to have a device life of 5 years.

For the analysis undertaken from a societal perspective, productivity losses associated with RTAs were also included, e.g. losses in household and market productivity, and associated workplace costs were calculated using the human capital approach. Non-medical costs including legal costs and insurance administration costs were also included. Societal costs of RTAs were stratified by level of severity using the MAIS scale and were based on national data. Lifetime costs of RTA outcomes and costs were based on public sources. In addition, it was assumed that all RTA costs were uniformly distributed over a future of 40 years for all patient groups. Unit costs were reported separately from resource use. Costs were reported for the year 2003. A discount rate of 3% was applied to the costs and effects for the base-case analysis.

Patient compliance with CPAP was included in the analysis. A compliance rate of 70% was assumed, based on findings in one article (McArdle et al. 39) It was assumed that non-compliant patients incurred rental costs for the device and the cost of a single visit to their doctor over a 3-month period but did not benefit from the device over the period.

Summary of cost-effectiveness

From a third-party payer perspective, CPAP was more costly and more effective than no CPAP and the ICER was $3354 per QALY (95% CI $1062 per QALY to $9715 per QALY). From the societal perspective CPAP was also found to be more costly and more effective and had an ICER of $314 (95% CI cost saving to $6114). Based on the probabilistic sensitivity analysis, if the value of society’s willingness to pay for a QALY is $50,000, 100% of the 1000 Monte Carlo simulations favoured the cost-effectiveness of CPAP from the third-party payer and the societal perspective. From the societal perspective, 42% of the ICERs from the simulations indicated that CPAP was dominant, i.e. it was associated with lower costs and greater effects than was no treatment.

Based on the univariate sensitivity analyses, ICER estimates were shown to be robust to many assumptions associated with the parameter estimates chosen. The analytical perspective had a substantial influence on the ICER, resulting in a more than 10-fold higher ICER when comparing the third-party payer perspective with the societal perspective. The type of utility estimate used also had considerable influence on the ICER. When EQ-5D utility estimates were used in place of standard gamble estimates, the ICER estimate increased more than fivefold.

Comments on Ayas et al.122

A single study (Chakravorty et al. 97) was used to measure the treatment effect of CPAP on sleepiness and hence HRQoL. This treatment effect was obtained from one arm of an RCT, with the pre-CPAP utility estimate used for the no treatment arm and the 3-month post-CPAP utility estimate used for the CPAP arm. As revealed by the systematic review (see Chapter 3), a considerable number of other data are available on the treatment effect of CPAP. Since only one arm of an RCT was used, in effect the data were subject to the same limitations associated with the before and after study design as mentioned in the discussion of the ResMed submission. It is worth noting that the mean change in ESS in the study arm used to inform the utility estimates was –8, which is considerably greater than the reduction with CPAP indicated by the weight of evidence incorporated in the systematic review in Chapter 3.

The impact of CPAP on RTAs was based on eight before and after studies, pre and post CPAP. Another influence that might undermine the results derived from this study design is referral bias. Patients may have been referred for suspected sleep-disordered breathing because they were involved in an RTA, thereby falsely inflating rates prior to using CPAP. The authors undertook a sensitivity analysis, using the odds ratio of the lower end of the confidence interval, which gave an ICER of around $3530 per QALY. The utility values were adjusted for various MAIS injury levels using the FCI. FCI weights represent rating scale preferences, whereas the NICE guidance focuses on choice-based measures of valuation.

For the base-case analysis, patient preferences were used to estimate utilities, based on the standard gamble technique. NICE methods guidance focuses on the use of societal preferences to inform health-care decision-making. However, it is worth noting that the authors applied EQ-5D estimates based on societal preferences in a sensitivity analysis. 123,144 The resultant CPAP ICER was within ranges typically considered to be cost-effective.

CPAP is a chronic condition; therefore, it would have been appropriate to extrapolate results over the life course.

Overview

Mar et al. 123 performed a cost–utility analysis comparing nCPAP with a ‘do nothing’ alternative for the treatment of patients with OSAHS. The base case was defined as a 50-year-old male patient with moderate to severe OSAHS, classified as having an AHI ≥ 30 per hour and an ESS > 10. The analysis was undertaken from the Spanish health-care system perspective.

The authors developed a semi-Markov (time-varying) model with a time horizon of 5 years and over the lifetime. Each Markov cycle lasted 1 year. The primary outcome measure used in the analysis was incremental cost per QALY. The QALY estimate for nCPAP incorporated the expected gains in HRQoL due to a reduction in stroke and CHD as well as the impact of these events on mortality and RTAs. At the end of each cycle patients could be in an OSAS state, have a non-fatal stroke or non-fatal CHD or die. During the cycle, patients could experience a temporary event: a stroke, CHD or an RTA. Cost estimates were also adjusted for the reduced risk of these three events. Effectiveness, utility and resource use estimates were obtained from the published literature and administrative databases. A number of univariate sensitivity analyses were undertaken. In addition, the authors undertook a probabilistic sensitivity analysis.

Summary of effectiveness data

The effect of CPAP on stroke, CHD, RTAs and HRQoL was incorporated in a series of steps. To model the reduced risk of stroke and CHD due to nCPAP compared with no CPAP, it was assumed that use of nCPAP returned blood pressure to its pre-OSAHS levels, as a consequence of the reduction in AHI by nCPAP. Based on an assessment of CVE risk, it was assumed that OSAHS patients with an AHI of greater than 30 had an increase in diastolic arterial pressure of 3.6 mmHg. It was estimated that increases in blood pressure were linearly correlated with the natural logarithm of the risk of stroke and CHD, based on MacMahon et al. 145 Therefore, applying this to the 3.6 mmHg increase in diastolic pressure, the natural logarithms of the risk could be obtained, thus estimating the relative risks of stroke and CHD. It was assumed that relative risk of these events remained constant at a given blood pressure and that absolute risk was best estimated by applying this relative risk to the baseline absolute risk for patient and age and sex using rates tables.

Mortality rates for stroke, CHD, RTAs and all causes by age and sex were based on an administrative database from the Basque country, Spain. The transition probabilities from OSAHS to death were based on the mortality rates of all causes, excluding stroke, CHD and RTAs. Age- and sex-specific mortality rates for the general population were multiplied by the corresponding relative risks to calculate the corrected rate for nCPAP and no CPAP groups (Table 22).

Utility values were obtained as described in Summary of effectiveness review for the ResMed submission (see pp. 54–5).

Summary of resource utilisation and cost data

The cost analysis was undertaken from the health-care perspective. The following direct costs were considered: costs of investigation, diagnosis and treatment of OSAHS and costs attributable to CVE morbidity, costs associated with in-home technical maintenance and medical follow-up. The nCPAP was assumed to have a device life of 5 years. Data relating to the last 5000 patients who had suspected OSAHS and who attended a sleep clinic (in the Basque country) were used to estimate the cost of diagnosis and the proportion of patients abandoning treatment during the first year. It was assumed that no benefits of treatment accrued to these patients.

Costs were reported for the year 2000 and were converted from Spanish pesetas to euros. A discount rate of 3% was applied to the costs and effects for the base-case analysis.

Summary of cost-effectiveness

The base-case analysis found that CPAP was more costly and more effective than no CPAP. The ICER for CPAP was €7861 per QALY over a 5-year time horizon and €4938 per QALY for the lifetime horizon.

Based on the univariate and multivariate sensitivity analyses, ICER estimates were robust to many assumptions associated with the parameter estimates chosen and remained in the region of €5000–€10,000. The only case where the ICER was found to exceed €20,000 per QALY was for a worst-case scenario analysis in which the authors used the lower limit of the CI obtained from the patient preference utility survey and a 5- year time horizon. As anticipated, the authors found that the cost-effectiveness ratio increases as the cost of nCPAP increases. For example, the authors assessed the impact of an increase in diagnostic costs on cost-effectiveness and the impact of two different types of diagnostic protocol on the ICER. As well as this, they presented disaggregated incremental effectiveness data (for CVE risk, RTAs and utility effect) and disaggregated incremental cost information for a 50-year-old male using nCPAP for both a 5-year and a lifespan time horizon. They found that the results were very sensitive to the time horizon specified. Over the lifespan of the patient, improvements in quality of life accounted for 84% of the incremental effectiveness of nCPAP. The purchase and maintenance costs of nCPAP accounted for 86% of the overall incremental costs. When the time horizon was reduced to 5 years, these costs amounted to 98% and 61% of the overall incremental costs respectively.

Overall, therefore, the authors suggested that nCPAP is cost-effective in patients with an AHI ≥ 30 and who also exhibit symptoms of daytime sleepiness. The authors suggested that the improvement in HRQoL associated with nCPAP is the main force behind its clinical effectiveness, as measured in QALYs, being seven times greater than that of reduced CVE mortality, which in turn is seven times greater than that of decreased numbers of RTAs. The authors suggested that the remaining uncertainties about the impact of nCPAP on long-term mortality have relatively little impact on the clinical and economic efficiency of treatment.

Comments on Mar et al.123

Data from a single survey, conducted as part of the study, were used to measure the treatment effect (in the form of change in sleepiness in terms of utilities) of CPAP versus no CPAP. As mentioned in the comments on the Ayas et al. 122 study, a large number of other data were available but were not used to inform this estimate, and the data were based on a before and after design, which represents a weak source of evidence of effectiveness.

No account in the model was taken of the impact of an RTA on morbidity/utility or costs. It is not clear that the mortality rates for RTAs (or stroke or CHD) were related to OSAHS. Probabilistic sensitivity analysis was not undertaken.

Overview

The Trent Report, written by Chilcott et al. ,44 provided the foundation for the work undertaken by Mar et al. 123 A cost–utility analysis was undertaken which compared nCPAP with a ‘do nothing’ alternative for the treatment of patients with OSAS. Summary results were reported comparing nCPAP and dental devices. The authors undertook the analysis based on a review of the literature and using clinical opinion. The analysis was undertaken from the UK health-care system perspective. The primary outcome measure used in the analysis was incremental cost per QALY. Cost estimates were also adjusted for the reduced risk of these three events. Effectiveness, utility, resource use estimates and costs were obtained from the published literature, administrative databases and clinical expert opinion. Several univariate sensitivity analyses were undertaken.

Summary of effectiveness data

No relevant utility data were available; therefore, an indirect approach was undertaken to estimate utilities. No data were reported on any change in ESS associated with treatment. The authors used SF-36 data generated by the Sleep Disorders Unit at the Royal Hallamshire Hospital in Sheffield. This involved a cohort study in which patients who were referred to the Unit for suspected OSAHS completed the SF-36 questionnaire before and after initiation of treatment, as reported in a conference abstract (Waterhouse et al. 146,147). Given that the data were not randomised, the authors attempted to validate them by comparing the results with data presented in Jenkinson et al. 77 They found that the before and after results were broadly similar for all SF-36 dimensions in the two studies. However, the population from the Waterhouse et al. 147 study appeared to have a lower initial baseline health status. The 1998 Brazier algorithm (Brazier et al. 148) (and another algorithm: no further details) was applied to the Waterhouse results to derive a preference-based single index measure of health. The QALY results at 1 year are reported in Table 23.

Summary of resource utilisation and cost data

The cost analysis was undertaken from the health-care perspective. As for the Mar et al. 123 analysis, the Trent Report considered the costs of investigation, diagnosis and treatment of OSAHS and maintenance and medical follow-up costs. In contrast to Mar et al. ,123 any costs attributable to CVE morbidity were not included in the analysis. The report did not mention the financial year of the cost data. The device life of nCPAP was estimated to be 5 years. A discount rate of 6% was applied to the costs and a discount rate of 1.5% was applied to effects for the base-case analysis.

Summary of cost-effectiveness

The results of the base-case analysis are reported in Table 24. The results were extrapolated on a time horizon of up to 5 years, assuming that the benefits accrued in the trial period were maintained.

Several univariate sensitivity analyses were undertaken, comprising impact of the analytical time horizon; costs of investigation for nCPAP; long-term costs of maintenance, follow-up and other health-care resource usage; long-term impact of gross annual health-care costs; potential impact of improved mortality from use of nCPAP treatment; impact of uncertainty in morbidity benefits from nCPAP therapy; and discount rate. All estimates of cost-effectiveness over 1 year were < £16,000 per QALY gained.

The authors suggested that the cost-effectiveness of dental devices compared with no treatment was likely to be similar or worse than the cost-effectiveness of nCPAP therapy compared with no treatment. The costs of nCPAP and dental devices were similar. Based on two studies,80,149 they found small differences in clinical effectiveness and costs when comparing nCPAP with dental devices. The implied differences in costs and outcomes, and the considerable level of uncertainty associated with both, suggested to the authors that the incremental cost-effectiveness of nCPAP over dental advancement devices was likely to be highly uncertain.

Comments on the Trent Report

Data used to estimate effectiveness were short term and based on observational data (Waterhouse et al. , 2 weeks’ duration;146,147 Jenkinson et al. , 4 weeks’ duration77). No account was taken of the potential impact of nCPAP on the risk of CVE or RTAs in terms of costs or effects. The analysis was deterministic.

Overview

Tousignant et al. 124 performed a cost–utility analysis retrospectively comparing the impact on HRQoL of pre-treatment with treatment using nCPAP. In this way nCPAP was compared with a ‘do nothing’ option in 19 patients with moderate to severe OSAHS. The study took place at a sleep clinic in Montreal, Canada.

Summary of effectiveness data

Patients attending a hospital sleep clinic (mean age 57 years, SD 10) and who had been receiving nCPAP treatment for an average of 9 months completed a standard gamble exercise. The health states valued were receiving treatment with nCPAP, pre-treatment, full health and immediate death. To assess the reliability of the results, patients completed the exercise on two occasions 2–3 weeks apart. The mean utility score for the pre-treatment health state was 0.63 (± 0.29) and the mean utility score for the nCPAP treatment health state was 0.87 (± 0.17). The intraclass correlation coefficients for the retest data were above 0.7 for both the treatment health state and pre-treatment health states. Patient life expectancy was estimated using Canadian life tables. The difference in utility pre and post treatment was multiplied by the life-years to calculate QALYs.

Summary of resource utilisation and cost data

The perspective of the cost analysis was not stated but appears to be the health-care system perspective. Costs included the cost of supplies and the rental and maintenance costs of the nCPAP device and associated devices (e.g. tubing and masks). It was estimated that the yearly cost of treating a patient in Quebec was CAN$2348 (price year not given). Costs also included the cost of one overnight sleep study at the outset of treatment at CAN$500, which included physician fees, technician salaries, supplies and amortisation of capital costs over 7 years. Alternatively, an ongoing cost of treatment per patient per year was estimated at CAN$800.

Summary of cost-effectiveness

Based on the use of the different cost estimates (above) a high estimate of CAN$9792 per QALY gained by nCPAP was calculated as well as a low estimate of CAN$3397 per QALY gained. Three patients had particularly large treatment effects. The authors explored the impact of excluding the three patients on the cost–utility ratio. Without the three patients, based on the high cost estimate, the cost–utility ratio increased to CAN$18,637 per QALY.

Comments on Tousignant et al.124

As all the patients were currently receiving nCPAP therapy, their valuation of the pre-treatment health state was done retrospectively. Given this, it is difficult to ascertain the extent to which the difference in pre-treatment and treatment utility scores is a real difference reflecting the impact of nCPAP treatment and the extent to which it reflects some sort of measurement error due to bias in recall. In addition, the results may be unreliable due to the weaknesses associated with observational data. It appears that costs (and effects) were not discounted to present values. For the most part, resource use estimates were reported separately from costs. Only the impact of treatment effects over the short term was considered. The analysis was deterministic.

Base-case analysis

• The base-case analysis compared the costs and QALYs of CPAP versus dental devices versus conservative management in a male aged 50 years.

• Subgroup analyses were undertaken by gender, OSAHS severity (as measured by ESS) and other relevant baseline patient characteristics.

Secondary analysis

• Scenario analyses undertaken to explore the impact on cost-effectiveness of:

  • [dash]excluding the impact of treatment on CVE

  • excluding the impact of treatment on RTAs

  • excluding the impact of treatment on both CVE and RTAs

  • change in ESS linked to SF-6D utility score rather than EQ-5D

  • relative risk reduction for CVE based on DBP and MacMahon et al. 145

  • autotitrating positive airway pressure (APAP) machine with 5-year life span and humidifier

  • treatment effects from bivariate random-effects meta-analysis (BRMA) (APBM only)

  • treatment effects from BRMA (APBM and office measurements).

• Subgroup analyses

  1. – for a cohort aged (35 and 65).

• – Other relevant modelling assumptions.

• – Uncertainty and value of information analysis.

The York economic model is fully probabilistic and the results from the model are presented probabilistically to reflect the implications of parameter uncertainty on decision uncertainty.

To inform research priorities, the expected value of perfect information (EVPI) was calculated for the decision problem. 150 This represents the value of obtaining perfect information on all the model parameters to eliminate the decision uncertainty (given acceptance of the model structure and evidence base). The EVPI can be compared with the potential costs of additional research to indicate whether there is value in further research to reduce the decision uncertainty.

Utility estimation for inclusion in the York economic model

As reviewed in Chapter 3, the evidence base on the effectiveness of CPAP in OSAHS consists of a number of randomised trials of varying designs (crossover and parallel), which display heterogeneity in their inclusion criteria and which report a considerable range of outcome measures covering sleepiness, HRQoL and blood pressure. Analysis was required to link the short-term outcome measures of clinical effectiveness to a preference-based measure of HRQoL in terms of utilities. The randomised trial data provided evidence on what can be viewed as intermediary outcomes in terms of sleepiness and blood pressure, but did not measure either treatment effects in terms of a reduction in the number of CVEs or a reduction in the risk of RTAs. Therefore, a model was required to link the available clinical data to long-term outcomes, HRQoL and costs in order to estimate the long-term cost-effectiveness of treatment with CPAP.

Valuing clinical effectiveness in terms of utility

The NICE Reference Case indicates that the measure of health outcome used in the cost-effectiveness analysis should be QALYs calculated with utility values derived from a validated generic, preference-based measure of HRQoL. The systematic review reported in Chapter 3 highlighted the measures used to describe the efficacy of treatments for OSAHS in RCTs, among which the ESS was the most frequently reported (n = 27 trials). Utility values and quality-adjusted survival were infrequently reported (n = 1 trial comparing CPAP to placebo) (Chakravorty et al. 97). Therefore, an additional literature search was undertaken, and this identified four other papers that contained potentially relevant HRQoL/utility data for inclusion in the model.

Table 26 reports key information on the four studies (see Appendix 7 for more details). Jenkinson et al. 152 and Chakrovarty et al. 97 also reported pre- and post-ESS scores. The former reported a pre-treatment ESS of 14 (SD 5) and a post-treatment score of 8.2 (SD 4.8). In a CPAP treatment arm, Chakrovarty et al. 97 reported a pre-treatment ESS of 16 (SD 6) and a post-treatment score of 8 (SD 6). In the lifestyle arm, Chakrovarty et al. 97 reported a pre-treatment ESS of 14 (SD 4) and a post-treatment score of 11 (SD 5). Note that none of the studies reporting utility data assessed the use of dental devices.

In order to use the trial data and to allow a comparison between CPAP and dental devices, there was a need to link the data on clinical efficacy, in the form of the disease-specific ESS, to utility. Data on mean difference in ESS were available for 23 studies comparing CPAP with placebo and six studies comparing CPAP with dental devices. To achieve the link between change in ESS and change in utility, three sets of individual patient data were obtained, two that measured ESS and SF-36 profile in the same patients87,151 and one that measured ESS, SF-36 profile and EQ-5D in the same patients. 144 The SF-36 data were used to calculate utility values based on the SF-6D, using an algorithm developed by Brazier et al. 153 based on UK public preferences.The EQ-5D data were used to calculate utility based on general UK population tariff values. 128 The three data sets were then used to develop prediction models to estimate the relationship between ESS and (1) utility values based on SF-6D and (2) utility scores based on EQ-5D.

A simple linear regression model was fitted to predict absolute utility scores from absolute ESS, controlling for baseline utility and baseline ESS. A larger number of observations were available with the SF-6D in comparison with the EQ-5D and, where multiple observations were available on the same patient, analyses were adjusted to reflect the dependence between repeated observations on the same individual. All variables were treated as continuous data, and baseline scores were centred prior to estimation. The use of ordinary least squares (OLS) regression relies on the assumption that the error terms are normally distributed. Assessment of the residuals estimated in both regressions revealed that this assumption appeared reasonable for utility scores based on the SF-6D (Figure 21). However, typically, EQ-5D scores do not follow a normal distribution, and as expected the EQ-5D scores in the data set formed a distribution skewed to the left, with a large number of observations clustered at a score of 1. Figure 21 shows that the residuals from the regression of EQ-5D scores on ESS deviate somewhat from a normal distribution. Although OLS regression methods have been found to perform well when predicting EQ-5D scores,154 a generalised linear model was also fitted to the data to ascertain whether an alternative error distribution such as a gamma might produce a better fit. However, this model did not improve the fit on the basis of the Akaike Information Criterion and so the results from the OLS model were used for both the EQ-5D and the SF-6D.

FIGURE 21.

Standardised normal plot of residuals for use in York economic model.

The results of the regression analyses are shown in Tables 27 and 28. The models indicate that an increase of 1 point in ESS is associated with a fall in utility of 0.01 and this is true for both the SF-6D and EQ-5D instruments, the results of which were remarkably similar. A test was performed to see if there was evidence for a change in relationship between different levels of baseline ESS (i.e. a change in the slope of the regression line for particular cut-off values of ESS) but there was no evidence to support such a subgroup effect.

The Cholesky decomposition of the covariance matrix from the regressions was employed to characterise the uncertainty around the estimated coefficients and to reflect the correlation between coefficients in the probabilistic sensitivity analysis. 150 The baseline utility for the hypothetical patient population was predicted from the specified baseline ESS score. Changes in ESS associated with treatment were converted to changes in utility (utility increments) using the predicted relationship between ESS and utility.

The utility decrements associated with stroke, CHD and age were based on the regression analysis reported by Sullivan et al. 155 and are reported in Table 28. Utility decrements and increments can be applied to the baseline utility of the hypothetical cohort to reflect the utility associated with being in any health state in the model. The EQ-5D scores used in the analysis by Sullivan et al. were calculated using US community preferences. However, equivalent decrements were not available based on UK community preferences. The uncertainty around the utility decrements was characterised using a normal distribution, as the utility decrements are described by the coefficients from a regression analysis. The standard errors are small enough that there is no risk of unsuitable values being selected in the probabilistic analysis.

The utility associated with experiencing an RTA was based on EQ-5D measures from the Health Outcomes Data Repository (HODaR). 156 HODaR recorded EQ-5D data for individuals 6 weeks after their inpatient episode (at Cardiff Hospital, UK) for injuries sustained from an RTA. Data were extracted for all patients who had a traffic accident as a motorcycle rider, an occupant of a three-wheeled motor vehicle, a car occupant or an occupant of a pick-up truck or a van [V20 to V59, International Classification of Diseases-10 (ICD-10) codes]. Results were found for 56 patients. A gamma distribution was used to characterise the uncertainty around the utility decrements associated with clinical events.

For the base-case analysis the effect of treatment on ESS was derived by pooling all of the available trial data to obtain an overall effect. However, in Chapter 3 it was noted that there was a high level of heterogeneity in this overall analysis that was reduced when trials were grouped by baseline severity of OSAHS. The treatment effects estimated by pooling trials grouped according to average baseline ESS (mild, moderate or severe) were applied in three separate analyses (see Chapter 3, Figure 2). These analyses cannot be interpreted as subgroup analyses reflecting differential treatment effects according to OSAHS severity as they are based on a study-level covariate in the form of average baseline ESS. In order to conduct a true subgroup analysis, trial data would have to be available that estimated the relationship at the patient level between baseline ESS and change in ESS with treatment.

Linking reduction in blood pressure to cardiovascular events

As noted above, the randomised trials provided information on the effect of CPAP on blood pressure but, for the economic model, the implications of this treatment effect for clinical events need to be estimated. The Framingham risk equations provide a link between risk factors such as blood pressure and the incidence of fatal and non-fatal CVEs. Published risk equations predict the risk of CHD and stroke157 over a range of 4–12 years as a function of either SBP or DBP. Anderson et al. 157 state that, of the two alternative measures, SBP was the best predictor of stroke, and therefore this was selected as the measure of blood pressure to be included in the economic model. We did not identify corresponding risk equations that incorporated MAP. The risk equations were estimated separately for men and women using the baseline characteristics of the hypothetical patient population shown in Table 29, which were determined from the RCT data where possible and based on plausible assumptions otherwise. It was also possible to incorporate the increased baseline risk associated with high BMI using the relative risk published by Mora et al. 158 However, as none of the relevant comparators demonstrated efficacy in terms of weight loss and reducing BMI, this was not included in the analysis.

It was assumed that the only risk factor affected by use of CPAP was blood pressure. The Framingham risk equations are based on Weibull models, and so the predicted risk is non-linear with respect to each risk factor. To determine whether the use of the mean change in blood pressure would bias the results, a check was performed on a set of individual patient data that reported change in blood pressure. The risk of CHD and stroke was predicted for each patient individually, and the mean of the individual predicted risks was compared with the risk based on the mean change in blood pressure for the whole group. The risks were found to be identical to two decimal places, and so it was felt that, although the equations are non-linear, the use of the mean change would not bias the model results.

In order to estimate the probability of CHD and stroke events per model cycle, a piece-wise exponential was assumed. The equations were used to predict the 4-year probability of an event every 4 years given the current age of the hypothetical patient cohort and starting from year 0. For the intervening years it was assumed that survival followed an exponential distribution, and so each 4-year probability was converted into a constant yearly probability to be applied over the relevant 4-year interval. When multiple equations from the Framingham set are used they should be applied in random order to take into account competing risks. However, in a cohort model structure the risk equations must be applied in the same order across the entire cohort for any given model cycle. This is relevant to patients in the initial state of the model who may experience either CHD or a stroke event. Rather than specify an order in which to apply the equations, the probability of any event was calculated by summing the hazards and then the proportion of events that were CHD or stroke events was calculated.

The relative risk reduction for CVE implied by the difference in SBP with CPAP compared with usual care is estimated to be relatively low using the Framingham risk equations (RR ≈ 0.98 for mean reduction in SBP of 1.06 mmHg). It has been posited that the Framingham risk equations may be subject to regression dilution bias when describing the relationship between a change in blood pressure and the change in risk of cardiovascular disease (CVD) events. Random fluctuations in blood pressure may cause the relationship between blood pressure and incidence of CVEs to be underestimated if the analysis is conducted on the basis of single baseline assessment of blood pressure. The Framingham risk equations specified in Anderson et al. 157 are based on the average of two office measurements of blood pressure (systolic or diastolic). MacMahon et al. 145 conducted an analysis to estimate the change in risk of stroke and CHD as a function of DBP in which they correct for regression dilution bias by incorporating data on usual DBP (average DBP over several years) as well as baseline DBP. Their results indicate that the percentage reduction in risk of stroke or CHD is approximately linearly related to DBP. They estimate that a 7.5 mmHg fall in DBP is associated with a 46% (SD 2%) reduction in risk of stroke and a 29% (SD 1%) reduction in risk of CHD. MacMahon et al. did not estimate the absolute risk of stroke or CHD events associated with particular levels of DBP. Therefore, a scenario analysis was conducted in which the baseline risks of stroke and CHD events were determined by the Framingham risk equations, but the change in risk associated with treatment was modelled using the relationship provided by MacMahon et al. The relative risk reduction for CVE implied by the difference in DBP with CPAP compared with usual care is estimated to be higher based on the MacMahon et al. analysis in comparison with the Framingham risk equations (RR = 0.96 for CHD and RR = 0.94 for stroke given a mean reduction in DBP of 1.20 mmHg).

Estimating the treatment effect of interventions on RTAs

To estimate the impact of CPAP on RTAs, the meta-analysis of before and after studies undertaken by Ayas et al. 122 was updated (see Table 21). Only one additional study was found (Barbé et al. 162). Because this study reported a relative risk rather than an odds ratio (as in the Ayas et al. paper122), the data reported on events and non-events were used to (re)calculate an odds ratio. The log odds ratios were then pooled by means of inverse variance weighting. The separate and pooled odds ratios are reported in Table 31. Note that, although the relative risk reduction of experiencing an RTA with CPAP treatment is high, the absolute baseline risk is very low.

The literature search did not identify any studies that assessed the impact of treatment with dental devices on RTAs. For the base-case analysis an adjusted odds ratio for dental devices compared with conservative management was estimated by applying the ratio of the treatment effects on ESS for CPAP and dental devices versus conservative management to the odds ratio for RTAs for CPAP versus conservative management.

It was assumed that patients left disabled following a first stroke event would no longer be at risk of an RTA. For the base-case analysis the proportion of first strokes that were disabling was estimated to be 30.9% based on the Second European Stroke Prevention Study (ESPS-2). 163 Note that the base-case analysis applies to patients who hold a driving licence. For OSAHS sufferers who do not drive the appropriate analysis is one in which the risk of RTA is excluded.

In summary, Table 32 shows the treatment effects used to populate the York economic model.

Compliance

The long-term compliance with CPAP will have implications for the estimated effectiveness in the target population. The majority of the trial data were based on less than 12 weeks’ follow-up. Given this, long-term compliance with CPAP was estimated on the basis of observational data provided by McArdle et al. 39 This study reported compliance over 6 years in a cohort of Scottish patients with a median age of 50 and an average ESS score at baseline of 12. The results indicated that 84% of patients continued to use CPAP 1 year after initiation of treatment, and that compliance was steady after a period of 4 years, with 68% of patients continuing treatment. The percentage of patients compliant at 2 and 3 years after treatment initiation was read from the survival curve (74% and 73% respectively) and these data were then used to model the rate of discontinuation from years 1 to 4 in the model. Patients discontinuing treatment were assumed to return immediately to the levels of ESS, SBP and utility associated with no treatment. In the base-case analysis it was assumed that 90% of patients who discontinued treatment with CPAP would return their machine. Equivalent data were not available for dental devices, and so in the base-case analysis it was assumed that compliance with dental devices was equivalent to that for CPAP.

Mortality rates

Table 33 reports the parameters associated with CHD, stroke and RTAs.

The mortality rate for individuals who have not experienced CHD or stroke (by age and sex) was taken from the UK life tables of the Government Actuary Department (www.gad.gov.uk). For each age band, the all-cause hazard was reduced by the proportion of people dying of cardiovascular disease (CVD) or ischaemic heart disease (IHD) causes to get the hazard of death for non-CVD or non-IHD causes using methods developed by Chiang. 168 For patients who experienced CHD or stroke, an elevated mortality rate was used based on relative risks from the literature. For patients who experienced CHD and stroke, relative risks of death of 3.2165 and 2.3,164 respectively, were employed. These relative risks were applied to the non-cardiovasular/ischaemic heart disease mortality rates in the UK population (by age and sex).