Clinical and cost-effectiveness results from the TOMADO randomised controlled Trial of Oral Mandibular Advancement Devices for Obstructive sleep apnoea-hypopnoea and long-term economic analysis of oral devices and continuous positive airway pressure

Definition of obstructive sleep apnoea–hypopnoea

Obstructive sleep apnoea–hypopnoea (OSAH) involves repeated interruption of airflow during sleep as a consequence of episodic collapse of the pharyngeal airway. Oxygen desaturations typically result and events terminate with respiratory effort-induced microarousals from sleep. When there is significant sleep disruption then daytime symptoms can result. Obstructive sleep apnoea–hypopnoea syndrome (OSAHS) incorporates excessive daytime sleepiness (EDS). 1 For the purposes of consistency, the term OSAH will be used in this review instead of OSAHS.

Consequences of obstructive sleep apnoea–hypopnoea

Obstructive sleep apnoea–hypopnoea is causally linked with hypertension. 2 There is a 2.5-fold associated increase in cardiovascular risk,3 with a reported 6% increase in stroke risk per unit increase in apnoea–hypopnoea index (AHI). 4 This association is supported by biologically plausible mechanisms and beneficial cardiovascular effects of OSAH treatment have been described. 5 However, difficult-to-exclude confounders mean that causation and the impact of OSAH treatment on cardiovascular disease (CVD) are still being explored. There are several other consequences of OSAH. Impaired vigilance is responsible for a two- to threefold increase in road traffic accident (RTA) risk,6 and health-related quality of life (HRQoL) is also impaired. 7,8 Health-care usage is almost doubled in OSAH, with one of the main determinants of increased cost being CVD. 9

Diagnosis and classification of obstructive sleep apnoea–hypopnoea

Obstructive sleep apnoea–hypopnoea is diagnosed and its severity determined on the basis of symptoms and respiratory monitoring during sleep. The simplest monitoring involves nocturnal oximetry, providing an hourly oxygen desaturation index (ODI) as a surrogate marker of respiratory events. Oximetry is relatively insensitive and so will miss some cases of OSAH. 10 The AHI is more sensitive and specific, combining oximetry with oronasal temperature and pressure monitors to measure airflow, and thoracic and abdominal movement gauges to distinguish obstructive from central events. To be scored, respiratory events must arbitrarily last at least 10 seconds. Apnoeas involve complete cessation of airflow. Hypopnoeas are variously defined by different classification systems. 11,12 Common is a degree of airflow amplitude reduction, which must be accompanied by either significant oxygen desaturation or, if measured, electroencephalography (EEG)-based evidence of microarousal from sleep. The multiple scoring systems introduce heterogeneity into what is otherwise a useful objective measure. 13 Nonetheless, the AHI provides an objective means of defining disease severity, monitoring disease course and measuring treatment response. The American Academy of Sleep Medicine (AASM), which is responsible for three hypopnoea definitions, arbitrarily defines mild OSAH as an AHI of ≥ 5– ≤ 15 events per hour; moderate OSAH as an AHI of 15–30 events per hour; and severe OSAH as an AHI of > 30 events per hour. 11 These are widely applied criteria and are used in this report.

The extent of daytime sleepiness ranges greatly in OSAH and there is only moderate correlation with respiratory event frequency. 14 With the persistent uncertainty regarding the impact of treatment on cardiovascular end points, the main treatment indication remains EDS. Therefore, OSAH is alternatively classified according to severity of EDS and impacts of treatments are measured by symptom effects.

Daytime sleep propensity is most commonly assessed using the subjective, but extensively validated, Epworth Sleepiness Scale (ESS). 15 The likelihood of falling asleep in eight different situations is rated on a scale of 0 to 3 by the patient. The total score ranges from 0 to 24, with higher scores indicating greater sleepiness. A score of ≤ 10 is considered normal in the general population. 16 The ESS score has a roughly Normal distribution in OSAH. 17 As a subjective measure, it is susceptible to significant placebo effects. 18 A recent meta-analysis estimated that 29% of ESS score improvement was attributable to placebo in high continuous positive airway pressure (CPAP) users. 19 However, the ESS has high internal consistency and has been found to be the best among a range of validated outcome measures in predicting real response to OSAH treatment. 17,18

Daytime sleepiness can be objectively measured using validated EEG-based nap tests. The multiple sleep latency test (MSLT) involves measuring sleep onset latency in four or five 20-minute tests, at 2-hourly intervals over 1 day. Subjects are put back to bed after overnight polysomnography (PSG), which serves to validate results by confirming sufficient preceding nocturnal sleep. They are instructed to try to sleep in a darkened and sound-attenuated room. The maintenance of wakefulness test (MWT) is the related EEG vigilance assessment, involving four or five 40-minute tests. 20 The Osler test is similar to the MWT, but uses repeated behavioural tasks to monitor alertness. 21 Although all are useful objective tests, and the MSLT is particularly important for diagnosing narcolepsy, results show considerable overlap between healthy subjects and those with sleep disorders. 20 They are affected by multiple factors other than disease20 and are time-consuming and resource expensive. As such, they have a limited role in the day-to-day diagnosis and management of OSAH and figure in only a few treatment trials. 22–25

Epidemiology and risk factors

Obstructive sleep apnoea–hypopnoea syndrome affects 2–7% of the adult population26 with the risk of developing the condition approximately twofold higher in men than in women. 27,28 It is common in middle age,27 but prevalence may be higher in the elderly. One community-based study found 62% of older adults (≥ 65 years) to have an AHI of ≥ 10. 29 Obesity is a major risk factor for OSAH, particularly when adiposity is distributed around the neck and upper body. 30,31 Another community study reported a quadrupling of disease prevalence associated with a one standard deviation (SD) increase in body mass index (BMI). 27 Causality is supported by longitudinal evidence of fluctuating disease severity in association with weight change. 28,32 Other lifestyle risk factors include smoking and alcohol use, while medical conditions with a possible causal association include hypothyroidism, polycystic ovary syndrome and acromegaly. 26,30

Continuous positive airway pressure therapy

Continuous positive airway pressure therapy is the cornerstone of OSAH treatment. It works by providing a pneumatic splint to the upper airway, preventing pharyngeal collapse during sleep. Treatment is applied with a nasal mask or face mask, connected via a tube to a small electric air pump generating the pressure.

Continuous positive airway pressure greatly reduces obstructive respiratory events and daytime sleepiness and improves cognitive function and quality of life (QoL). There is evidence of beneficial effects on blood pressure (BP),8,33,34 from which improvement in cardiovascular end points can be inferred, although direct evidence for this continues to be sought. CPAP improves impaired driving simulator performance and observational data have shown a reduction in excess RTA risk. 35,36 However, there is a lack of direct evidence for the latter. 8 CPAP has been shown to be clinically effective and cost-effective for moderate to severe OSAH at a willingness-to-pay (WTP) threshold of £20,000 per QALY and clinical guidelines recommend it as first-line treatment in these patients. 8,37,38

The intrusive nature of CPAP means that intolerance can undermine its effectiveness. Not all patients accept treatment and reported usage ranges from 29% to 85%. 39–42 Efforts continue to explore ways of increasing CPAP acceptance and adherence. Pressure modification (e.g. with bilevel devices, autotitrating CPAP and expiratory pressure relief) and humidification are used, although current evidence suggests that compliance with positive airway pressure is similar regardless of the mode of delivery. 43 A variety of educational, supportive and behavioural interventions continue to be tried, but techniques are diverse and the overall quality of evidence is currently too low to guide patient selection or choice of intervention. 44

The role of CPAP in the management of mild OSAH is less clear. There is a paucity of randomised trial evidence and cost-effectiveness of CPAP appears more marginal. McDaid et al. 8 calculated its cost to slightly exceed the National Institute for Health and Care Excellence (NICE) threshold for mild patients, with an incremental cost-effectiveness ratio (ICER) of £20,585 compared with conservative management (CM). Compliance with CPAP may also be worse in milder disease. 45 Clinical guidelines reflect the uncertainty, recommending that CPAP be tried when significant symptoms fail to respond to lifestyle measures and any other relevant treatment options. 37,38

Non-continuous positive airway pressure therapy treatments

The existence of modifiable OSAH risk factors with plausible causative mechanisms encourages an approach to OSAH treatment which includes lifestyle measures. Clinical guidelines recommend that interventions such as weight loss, smoking cessation, reduction of alcohol intake and positional management (supine sleep avoidance) should be considered in the treatment of individual patients. 37,38 However, conclusive randomised trial evidence of effectiveness of lifestyle modification is still lacking. 46

Various surgical techniques have been developed to try to treat OSAH. Their aim is to prevent pharyngeal occlusion by increasing upper airway dimensions, reducing collapsibility and/or bypassing obstruction. Several short-term studies have been reported. However, diverse techniques, inconsistent effects and a lack of longer-term data mean that conclusive evidence of effectiveness is lacking. 47

Pharmacotherapy continues to be explored in OSAH. Several drugs have been investigated, attempting to exploit various hypothetical pharmacological mechanisms to reduce respiratory events and/or improve symptoms. Despite some positive results from individual trials, effectiveness has not been proven. 48

Primary objectives of Trial of Oral Mandibular Advancement Devices for Obstructive sleep apnoea–hypopnoea

The primary objective was to determine whether or not MADs are more effective than no treatment and whether or not the level of MAD sophistication (bespoke, semi-bespoke and over the counter) influences outcomes for patients with mild to moderate OSAH.

Secondary objectives of the Trial of Oral Mandibular Advancement Devices for Obstructive sleep apnoea–hypopnoea

The secondary objectives were to produce a trial-based cost-effectiveness analysis to determine, from a NHS perspective, whether or not MADs are cost-effective compared with no treatment in mild to moderate OSAH, and whether or not the degree of MAD sophistication influences cost-effectiveness. It was also intended that the results would contribute to a comprehensive long-term cost–utility analysis (see Chapter 4).

Study design

The study was an open-label, four-treatment, four-period, randomised crossover trial comparing the clinical effectiveness and cost-effectiveness of three types of MAD {bespoke MAD (bMAD; NHS Oral-Maxillofacial Laboratory, Addenbrooke’s Hospital, Cambridge, UK), semi-bespoke [SleepPro 2™ (SP2); Meditas Ltd, Winchester, UK] and over the counter [SleepPro 1™ (SP1); Meditas Ltd, Winchester, UK]} and a no-treatment control for patients with mild to moderate OSAH (AHI of 5 events/hour to < 30 events/hour). Each 6-week period (4 weeks for no-treatment arm) comprised a 2-week acclimatisation phase, followed by a 4-week treatment phase. A 1-week washout period followed active treatments.

The study was reviewed and approved by the National Research Ethics Service Research Ethics Committee East of England – Cambridge Central (reference 10/H0308/4) and local (research consortia and primary care trust), ethical and research governance committees, and was registered as an International Standard Randomised Controlled Trial, number (ISRCTN) 02309506. The trial protocol can be accessed at www.thelancet.com/protocol-reviews/10PRT-4998.

Public and patient involvement

There was involvement from a patient in the study design and conduct, with input into production of patient information and other trial documentation, and membership of both the trial management group and the trial steering group. Although patient involvement in the Data Monitoring and Ethics Committee (DMEC) was arranged, the patient representative was not able to contribute to the meetings.

Participants

All newly referred or existing patients attending the Respiratory Support and Sleep Centre (RSSC), a tertiary care, specialist sleep centre, at Papworth Hospital (Cambridge, UK), were invited to be screened for eligibility in the trial if they were ≥ 18 years of age and had, or were suspected of having, mild to moderate OSAH (AHI 5 events/hour to < 30 events/hour), confirmed by either respiratory PSG (rPSG) (Embletta™; Embla Systems, Kanata, ON, Canada) or complete PSG, and who had symptomatic daytime sleepiness defined by an ESS score of ≥ 9. Potential patients did not require CPAP, as defined in NICE Technology Appraisal number 139,37 or they had refused CPAP or chose inclusion in TOMADO instead. Patients were excluded if they were pregnant or had any of the following:

• central sleep apnoea as the predominant form of sleep-disordered breathing

• coexistent sleep disorder, poor sleep hygiene or drug treatment considered likely to have a significant impact on symptoms (especially sleepiness) or assessment of MAD effectiveness

• severe and/or unstable CVD judged by clinician to warrant immediate CPAP

• other medical or psychiatric disorders judged likely to adversely interact with MADs or confound interpretation of its effectiveness

• significant periodontal disease or tooth decay; partial or complete edentulism; presence of fixed orthodontic devices

• temporomandibular joint pain or disease

• clinical history suggestive of severe bruxism

• restriction in mouth opening or advancement of mandible

• respiratory failure

• inability to give informed consent or comply with the protocol

• previous exposure to MAD treatment

• disabling sleepiness leading to significant patient-specific safety concerns.

Interventions

Three different non-adjustable MADs representing currently available devices along a spectrum of complexity and cost were studied:

1. SleepPro 1™: a thermoplastic ‘boil and bite’ device fitted by the patient following the manufacturer’s printed instructions. The patient softened the device in hot water, placed it into his or her mouth and, having bitten down on it, advanced the mandible to an individually determined ‘comfortable’ position. The device was then manually moulded against the teeth and set by subsequent immersion in cold water. Rewarming allowed remoulding.

2. SleepPro 2™: a semi-bespoke device, formed from a dental impression mould used by the patient. At the screening/baseline visit patients were given an impression kit to mould at home and then send to the manufacturer in order for the SP2 to be made. The impression kit consisted of a SP1 with holes to allow the injection of dental putty. The patient was instructed to mould the SP1 (as for the SP1 device), then wear it for two nights to ensure optimum position and fit, remoulding if necessary. The patient then made up the putty and injected it into the SP1, before sending the resulting impression back to the manufacturer. The SP2 was produced from this mould. It was designed to grip the entire dentition. Thinner walls than the SP1 were intended to result in a more comfortable fit. Involvement of the patient’s dentist in taking the impression was suggested, but not considered to be essential or key to achieving the best fit by the manufacturer.

3. Bespoke device: a custom-made MAD professionally fitted by a specialist NHS oral–maxillofacial laboratory at Addenbrooke’s Hospital, Cambridge, UK. A positional ‘wax bite’ was taken from the patient and the degree of mandibular advancement (50–70% of the maximal protrusive distance from centric occlusion, i.e. the ‘normal’ bite where the teeth all interdigitate maximally) was determined. Upper and lower full dental impressions were taken in alginate by a suitably qualified dental professional and cast in dental stone. The casts were trimmed and articulated using the positional wax bite. A blow-down splint in soft acrylic was created on each cast and then fused with a further acrylic blow-down to ensure the upper and lower dentition were positioned in the predetermined optimal position to hold the mandible forward. The patient returned roughly 2 weeks later for the fitting. The fitting allowed for optimal balance between advancing the mandible sufficiently to bring the tongue base off the posterior pharyngeal wall and patient comfort.

Degree of protrusion

As this was a pragmatic trial, the SP1 and SP2 devices were both advanced by the patient, according to manufacturer’s instructions. The bMAD was fitted by qualified dental experts, who determined the degree of protrusion with the patient, aiming for maximal comfortable advancement. The aim was to advance the mandible by a minimum of 50% of maximal protrusion. The degree of protrusion of each device was measured by the trial team, where possible, at the end of the patient’s involvement in the study.

Patients started the first treatment arm following the manufacture of all of the MADs. The first 2 weeks of each treatment period were an acclimatisation phase to allow patients to adjust to each device and not considered part of active treatment. After 2 weeks, patients were telephoned to assess initial tolerability and adherence, and to record any contact with the research team, maxillofacial laboratory or other clinical staff in the previous 2 weeks. All patients received 4 weeks of treatment with each MAD and the no-treatment control, with outcome assessment at the end of each treatment period.

A 1-week no-treatment washout period followed each active treatment to avoid carryover effects. All MADs were kept at Papworth outside the treatment period and patients were asked to return each device at the end of the treatment assessment, and before starting the next treatment.

Outcome measures

Safety monitoring

Adverse events (AEs) and adverse reactions (ARs) were monitored throughout the trial and recorded at each end of treatment visit. An AE was defined as any untoward occurrence in a clinical investigation subject who was receiving a trial intervention which did not necessarily have a causal relationship with the intervention. An AR was defined as an AE for which a causal relationship with the intervention was at least a reasonable possibility, i.e. the relationship could not be ruled out.

The main expected ARs of MAD therapy were temporomandibular joint/jaw discomfort, mouth discomfort, dry mouth, excessive salivation, gum discomfort, tooth discomfort, loose teeth, malocclusion and mouth ulcers. It was left to the investigator’s clinical judgement whether or not an AE was of sufficient severity to necessitate the patient’s removal from the trial treatment. A patient could voluntarily withdraw from treatment at any time if he or she found an AE to be intolerable.

The severity of AEs was graded as mild, moderate or severe. The relationship between the trial treatment and the AE (the causality) was graded as either unrelated, possibly related, probably related or definitely related by an independent respiratory and sleep medicine consultant physician who sat on the Trial Steering Committee.

All AEs were followed up until resolution or to the end of the AE reporting period.

Serious adverse events (SAEs) were reported to the sponsor within 24 hours of a member of the trial team becoming aware of the event. All SAEs were followed up until resolution or the event was considered stable.

Patient withdrawal

Patients could withdraw from the trial at any time without giving a reason. All patients who withdrew from the study continued to receive normal clinical care if necessary from their GP or consultant in the RSSC.

Sample size and power calculation

Based on the pre-trial systematic review of published studies, the minimum clinically important effect size was considered to be of the order of one-third. An effect size of one-third would be detected with 80% power in a sample size of 72 patients (two-sided significance of 5%). Allowing for 20% loss to follow-up, we aimed to recruit a sample of 90 patients.

Randomisation

Randomisation took place once eligibility was confirmed following measurement for the bMAD and once impression suitability for the SP2 device had been confirmed by the manufacturer. A computer-generated random number sequence produced by the trial statistician determined treatment order. Randomisation was based on two related Williams’ Latin squares designs, with patients randomised in permuted blocks of eight with sequences shown in Table 1. Although randomisation in blocks of eight meant that for every eighth patient the sequence was predictable, this was considered to be less important in a crossover trial. Randomisation sequences were held in the research and development (R&D) unit and restricted to research administration staff. The trial team were informed of the randomisation sequence to be given to a patient via telephone contact with the R&D research administrators.

Blinding

Treatment blinding was not possible in this trial. However, the primary outcome, AHI, was ascertained from anonymised PSG traces, which were analysed in batches of 10 by an independent NHS polysomnographer who was not aware of treatment allocations.

Statistical analysis

All patients were followed up irrespective of their level of compliance with the MADs, and all periods for which there was a measurement were included in the analysis using ‘intention to treat’.

Given the nature of the treatments (external devices designed to control symptoms) and the inclusion of a 1-week washout between MAD periods, carryover effects in this crossover trial were considered negligible. In exploratory analysis no treatment by period interactions were identified, which supports this view. Period effects were included in the analysis to account for the long trial period (7–8 months) and in case compliance was related to time in the study.

Initially, the distribution of the outcome measures was assessed by comparing histograms against standard parametric distributions starting with the Gaussian distribution and, if necessary, exploring other plausible families. This was completed for all observations and by treatment group and period. Based on these analyses the primary outcome, AHI, was found to be distributed as a Poisson random variable, which is consistent with a measurement of an event rate per hour. The 4% ODI was also well modelled by a Poisson distribution. All other continuous outcomes were well modelled by Normal distributions. Treatment effects were also plotted over time to further explore period effects.

Given that there were repeated measurements for each patient the main inferential analysis employed a range of mixed models. Initially a full model was fitted that included the main effects of treatment and time period, the interaction between these two and random-effects terms for patient. However, likelihood ratio tests comparing models with and without time by treatment showed that these interaction terms were negligible, and so they were not included in subsequent models. Both treatment and time period in all models were included for consistency and because there was evidence of changes over time in some of the outcome measures based on the likelihood ratio test comparing models with and without the time period effects. The main inferential models were formulated as follows.

For patient i (i =1, . . ., 90) with response y_ijk for treatment j, j = 1,2,3,4 in time period k, k = 1,2,3,4 we fitted the generalised mixed model,

and

where β_o is the intercept fixed at the control treatment in period 1, β_j, j = 2,3,4 is the vector of length 3 representing treatment fixed effect, τ_k, k = 2,3,4 is the vector of length 3 representing the time period fixed effects and μ_i is the random-effect term for patient i nested in period 1.

For AHI, a Poisson mixed-effects regression was used, with a h( ) log-link function and the random-effects exp(u_i) having Gamma(1,α) distribution. A similar Poisson-Gamma model was fitted to the 4% ODI. For both of these models an additional term was included in the regression equation for the times each person was asleep during the test in which the response was recorded. Response to treatment was classified as complete if the AHI was < 5 events/hour, and partial if the AHI was reduced by 50% but was > 5 events/hour; otherwise, patients were classed as non-responders. Mixed-effects logistic regression, using the logit link function, was used to assess treatment effect on complete/partial response, with patient random effects, u_i, having a Normal (0,σ²) distribution on the logit scale. All other outcomes were analysed using normal mixed-effects models, with h( ) the identity link function and the u_i having a Normal (0,σ²) distribution.

In all analyses estimation of treatment effects was of primary interest, but hypothesis testing was also performed. Nested models were compared using likelihood ratio tests. Model fit was assessed informally by examination of standardised residuals. The approach to multiple testing was as follows. For each of the general(ised) linear mixed models, treatment effects were described as ‘statistically significant’ if the likelihood ratio test comparing the models with and without treatment effects was < 0.05. The TOMADO protocol states that comparison of each MAD against no treatment was important so that, for models that were ‘significant’ overall, we present the significance level is presented based on the Wald test [(β_j/se(β_j))∼N(0,1)] without adjustment for multiple comparisons. For comparisons between MADs, the (conservative) Bonferroni correction should be applied, that is, standard p-values for these comparisons should be multiplied by 3. Corrections have not been routinely applied, so that readers may make their preferred corrections and where the results are uncorrected has been indicated.

The initial analysis included all patients who completed any treatment period and supplied an outcome measurement. A second analysis included patients who had completed all four periods and provided measurements (complete cases analysis). Both these analyses assumed missing at random for incomplete data and gave almost identical results, so that complete cases results for the AHI and ESS score are omitted from this report. All other results in this report relate to patients who provided any follow-up information. The majority of the missing data arose from patients who did not complete any treatment periods or from sporadic technical failures of the PSG study. These considerations, coupled with (i) the consistency of complete cases and any follow-up analyses, (ii) the consistency of inferences regarding each MAD’s effectiveness across all outcomes and (iii) the clear nature of the results, meant that further sensitivity analysis to account for missing data was considered unnecessary.

Regression analyses were conducted to assess the effects on subsequent AHI and ESS scores of baseline AHI, ESS score, degree of protrusion of the device, age, sex, BMI and compliance, and contemporaneous BMI. These analyses also explored interactions between these variables and treatment effects, although there was limited power. Before the trial, one subgroup analysis of patients who declined CPAP compared with those with mild to moderate OSAH for whom CPAP was not considered necessary. As there were only four patients in the former group, no subgroup analyses were undertaken.

All analyses were performed using Stata (StataCorp LP, College Station, TX, USA) version 12.0 and version 13.0 for Microsoft Windows (64-bit).

Adherence to treatment protocols, treatment preferences, partner scoring assessment, RTAs and AEs were summarised and compared informally. Treatment preference results were available for patients who had completed all four treatment periods and are summarised.

Trial-based economic analysis

The economic evaluation of the crossover trial provided descriptive data on the resource use, unit costs and health state utilities observed during the 4-week periods from the perspective of the NHS.

Patient recruitment

Between December 2010 and July 2012, 440 patients were screened for the trial. Two hundred and eighty-one patients were excluded at screening, 51 of whom were excluded for dental ineligibility by a sleep physician. A total of 159 patients gave written informed consent. Sixty-nine patients either refused or were ineligible following the baseline sleep study or the hospital visit for the bMAD fitting. Only two patients who were considered dentally suitable for the trial by a sleep physician were subsequently excluded by the hospital maxillofacial team for poor oral hygiene and tooth decay. The remaining 90 patients were recruited to the trial and received a randomised treatment allocation sequence (Figure 1).

FIGURE 1.

Patient flow through the trial.

Baseline characteristics

Baseline measurements are recorded in Table 2. Mean (SD) age was 50.9 (11.6) years and ranged from 26 years to 79 years. Eighty per cent were men (72/90). Mean (SD) AHI at baseline was 13.8 (6.2) events per hour, with three patients who were accepted on the basis of desaturation index (DI) having a baseline AHI of < 5 events per hour, rendering them ineligible for the trial on confirmatory PSG. These patients were retained in the trial according to ‘intention to treat’. Mean (SD) ESS score was 11.9 (3.5) and, although 12 patients had a baseline ESS score below the acceptance threshold of 9, they were eligible based on an ESS score of ≥ 9 at screening. Again these patients remained in the trial.

Risk factors for heart disease were common in this group. Median [interquartile range (IQR)] BMI was 30.6 kg/m² (27.9–35.1 kg/m²) and mean BP pressure was normal at 130/80 mmHg, but varied widely from 98/57 to 177/116 mmHg. Diabetes was present in eight (9%) patients, 23 (26%) were being treated for hypertension and 21 (23%) for hypercholesterolaemia. Five (6%) patients had been diagnosed with ischaemic heart disease and three (3%) had previous cardiovascular events (CVEs).

Of the 90 patients entered into the trial, 86 were new patients (who had not refused CPAP) and four were patients who had tried CPAP but could not tolerate it.

Withdrawals

Figure 1 shows patient progress through the trial. During the trial, 16 (18%) patients withdrew and the reasons for withdrawal are described in the Table 3.

Of the 16 patients who withdrew from the trial, seven (8%) did not complete any treatment periods, three were using the bMAD, two were using the SP1, one was using the SP2 and one patient was in the no-treatment arm. A further two (2%) patients who withdrew between the first and second treatments provided no primary outcome data as a result of technical failure of the sleep study after the first treatment period. These nine patients (7 + 2) provided no information after baseline and are excluded from all analyses. The main reasons for withdrawal in patients who did not complete any treatment period were intolerance of a device or were related to an AE. It is likely that these patients would not tolerate any of the devices and all wanted to try alternative treatments (CPAP or CM including weight loss).

Seven (8%) further patients withdrew during the trial: four were using the bMAD, one was using SP1 and two were using the SP2. Only one of these withdrawals (SP2) was as a result of intolerance to the device. These cases were included in the main analysis. Seven other sleep studies failed, leaving 305 studies (85% of 360) in 81 patients [of 90 (90%)] for AHI analysis. For all other outcomes, 314 (87%) measurements and 83 (92%) patients were available for analyses.

One patient who was randomised early in the trial was unable to remould another SP2 to replace their damaged SP2 and subsequently withdrew. Thereafter successful SP2 moulding was made a prerequisite for randomisation. Four patients were subsequently not randomised because they could not mould the SP2. This was in part because of intolerance that would probably have applied to all three devices, but technical difficulty was a factor in some cases.

Baseline characteristics for the patients who withdrew from the study were similar to those who completed follow-up (data available on request).

Primary outcome: apnoea–hypopnoea index

Table 4 shows the mean AHI (SD) for each treatment, alongside the results of the Poisson-gamma regression analysis. Mean AHI for each treatment is plotted in Figure 2. This shows that that the rate of apnoea/hypopnoea events per hour for each MAD, relative to no treatment, is reduced significantly, with estimated relative rates of 0.74, 0.69 and 0.64 for SP1, SP2 and bMAD, respectively. The reductions for the SP1, SP2 and bMAD represent effect sizes of approximately 0.36, 0.47 and 0.49 SDs, respectively, all of which exceed the minimum clinically important difference of one-third proposed during study planning. In post-hoc pairwise comparisons there were no significant differences in AHI between the different MADs (Table 5).

FIGURE 2.

Estimated mean AHI and 95% CI for the four treatments from the Poisson-Gamma model.

Examination of the standardised residuals for AHI showed that this model was a good fit to the data, with no systematic effects observed.

Apnoea–hypopnoea index: responders to treatment

Of the patients who had an AHI value for at least one treatment, complete or partial AHI response during MAD use was observed in 29 (38%) patients for the SP1, 38 (49%) patients for the SP2 and 33 (45%) patients for bMAD, compared with 17 (22%) patients during the no-treatment period (Table 6 and Figure 3). Patients who responded to one MAD were more likely to respond to others, but this was not completely predictable. Four of the 74 completers (5%) had a complete response to all treatments, nine (12%) had a partial or complete response to all treatments and 20 (27%) did not have a response to any treatment. The four patients who completely responded to all treatments also had low AHI during the no-treatment period (AHI at baseline 3.1, 5.4, 7.6 and 8.9).

FIGURE 3.

Complete or partial response of patients by treatment.

Predictors of apnoea–hypopnoea index response

Using mixed-effects logistic models for complete/partial response, all MADs had significantly greater response rates than during the no-treatment period (Table 7). Response was significantly associated with baseline BMI [odds ratio (OR) 0.89, 95% CI 0.81 to 0.98; p = 0.014] and with contemporaneous BMI (OR per kg/m² 0.88, 95% CI 0.80 to 0.98; p = 0.007). It was also weakly associated with protrusion (OR 1.03 per % protrusion, 95% CI 1.00 to 1.05 per % protrusion; p = 0.034). Baseline AHI, ESS score, sex and age (years), as well as measures of compliance, were not significantly associated with response.

Secondary outcomes

Epworth Sleepiness Scale

Table 8 shows summary statistics for the four treatment periods as well as the results of the mixed-effects linear regression. Figure 4 plots estimated ESS score by treatment. There was a clear, statistically significant, reduction (improvement) in ESS score for all MADs compared with no treatment, with effect sizes of approximately 0.35, 0.50 and 0.55 SDs compared with no treatment. In addition, there was a weakly significant difference between the SP1 and the bMAD in post-hoc pairwise comparisons (Table 9).

TABLE 8 - Summary of results from mixed-effects model for ESS score (n = 83)

Note

Period effects not shown.

ESS score (n = 83)	Mean (SD)	Coefficient	95% CI	p-value	Global p-value
Constant		10.65	9.64 to 11.66	< 0.001
Difference in ESS score compared with no treatment
No treatment	10.1 (4.3)	–	–	–	< 0.001
SP1	8.5 (4.0)	−1.51	−2.29 to −0.73	< 0.001
SP2	8.0 (4.1)	−2.15	−2.99 to −1.31	< 0.001
bMAD	7.7 (3.8)	−2.37	−3.22 to −1.53	< 0.001

FIGURE 4.

Estimated mean ESS score and 95% CI for the different treatments from the mixed-effects model.

TABLE 9 - Comparison of ESS score between different MADs

Note

These comparisons are not adjusted for multiple comparisons. Differences are significant at 5% level according to the Bonferroni method if the p-value is < 0.017.

Comparison	Observed contrast	95% CI	p-value
SP2 with SP1	−0.64	−1.44 to 0.15	0.112
bMAD with SP1	−0.86	−1.63 to −0.09	0.029
bMAD with SP2	−0.22	−0.97 to 0.53	0.568

Patient evaluation of treatments

On average, patients considered the SP2 and the bMAD to be as comfortable as no treatment, but the SP1 was significantly less comfortable than all other treatments (VAS for comfort, Table 13). This resulted in greater satisfaction for the SP2 and the bMAD than for no treatment or the SP1 (VAS for satisfaction, Table 13). Table 14 shows that patients reported that the SP1 was more likely to fall out during sleep than the SP2, and that the SP2 was more likely to fall out than the bMAD. In addition, patients reported that they were more likely to remove the SP1 during sleep than either the SP2 or the bMAD (Table 15).

TABLE 13 - Summaries of the visual analogue valuations of treatment comfort and satisfaction

Max., maximum; min., minimum.

Treatment	n	Median treatment comfort (IQR)	Min.	Max.
No treatment	78	50 (50–97)	1	100
SP1	81	34 (16–50)	0	91
SP2	78	52 (36–82)	0	100
bMAD	77	50 (25–76)	0	97
Treatment	n	Median treatment satisfaction (IQR)	Min.	Max.
No treatment	78	50 (25–50)	0	100
SP1	81	43 (14–65)	0	99
SP2	78	67 (41–87)	0	100
bMAD	77	71 (38–87)	0	100

TABLE 14 - Patient report of frequency that device fell out

One missing value for the SP2, as the patient did not wear the device for longer than 1 minute.

On average, how often the device fell out	Frequency (%) for SP1 (n = 81)	Frequency (%) for SP2a (n = 78)	Frequency (%) for bMAD (n = 77)
Never fell out	27 (33%)	43 (56%)	51 (66%)
Fell out occasionally, but not every night	35 (43%)	26 (34%)	22 (29%)
Fell out once or twice every night	11 (14%)	5 (6%)	4 (5%)
Fell out > 2 times every night	8 (10%)	3 (4%)	0 (0%)

TABLE 15 - Patient report of frequency that device was removed

One missing value for the SP2, as the patient did not wear the device for longer than 1 minute.

On average, how often the device was removed	Frequency (%) for SP1 (n = 81)	Frequency (%) for SP2a (n = 78)	Frequency (%) for bMAD (n = 77)
Never removed	25 (31%)	40 (52%)	34 (44%)
Removed 1–3 nights/week	33 (41%)	23 (30%)	28 (36%)
Removed 4–6 nights/week	10 (12%)	9 (12%)	8 (10%)
Removed every night	13 (16%)	5 (6%)	7 (9%)

The 74 patients who completed all treatments were asked to state their preferred treatment. Of these, 30 (41%) ranked the bMAD highest and 23 (31%) ranked it second. The SP2 was ranked highest by 22 (30%) patients and second by 34 (46%) (Figure 5). Only 10 (14%) patients ranked no treatment highest.

FIGURE 5.

Bar chart of patient preference.

Most patients (56/90, 62%) continued with their preferred device after the study ended, with five (6%) others retaining the MAD that gave the best results for them. Other treatments undertaken by patients after the trial are listed in Table 16.

TABLE 16 - Patient management after completing TOMADO

Number of patients (n = 90)	Future care
16	Start CPAP
56	Use preferred MAD
5	Continued with the MAD that provided the best results
2	Withdrew – continued with current MAD
3	No further treatment
3	Advised weight loss
2	CM only
2	Lost to follow-up – recommended to use SP2
1	Ropinirole for restless legs syndrome

Functional Outcomes of Sleep Questionnaire

Eighty-three (92%) patients had at least one FOSQ result (Table 17). Figure 6 summarises the results for the five subscales. The overall FOSQ scores showed a weak period effect (p = 0.021), suggesting that there may be some adjustment of questionnaire responses over time. After including period effects in the model, there were significant improvements for all the MADs compared with the no-treatment period. In addition, there were small but significant differences between the SP1 and SP2 and between the SP1 and bMAD but not between the SP2 and bMAD (Table 18). The plot of individual FOSQ scales (Figure 7) suggests that this improvement is because of small increases in all dimensions but particularly for activity level and general productivity.

TABLE 17 - Summary of results from mixed-effects model for the FOSQ (n = 83)

Note

Period effects not shown.

FOSQ (n = 83)	Mean (SD)	Coefficient	95% CI	p-value	Global p-value
Constant		16.21	15.65 to 16.78	< 0.001
Difference in FOSQ compared with no treatment
No treatment	16.62 (2.55)	–	–	–	< 0.001
SP1	17.13 (2.42)	0.50	0.08 to 0.92	0.018
SP2	17.70 (2.14)	1.10	0.65 to 1.55	< 0.001
bMAD	17.90 (1.92)	1.31	0.84 to 1.78	< 0.001

FIGURE 6.

Estimated mean FOSQ and 95% CI for the different treatments from the linear mixed-effects model.

TABLE 18 - Comparison of total FOSQ score between different MADs

Note

These comparisons are not adjusted for multiple comparisons. Differences are significant at 5% level according to the Bonferroni method if the p-value is < 0.017.

Comparison	Observed contrast	95% CI	p-value
SP2 to SP1	0.60	0.18 to 1.03	0.005
bMAD to SP1	0.81	0.41 to 1.20	< 0.001
bMAD to SP2	0.21	−0.19 to 0.60	0.304

FIGURE 7.

Box plots of the mean score for each domain of the FOSQ.

Short Calgary Sleep Apnoea Quality of Life Index

The summaries and model results for the overall score are shown in Table 19 and Figure 8. In common with the FOSQ overall score, there was a significant effect of all MADs compared with no treatment and a small but significant difference between the SP1 and SP2 and between the SP1 and bMAD, but not between the SP2 and bMAD (Table 20). Figure 9 shows results for each subscale of the SAQLI and, again, shows a small improvement across all dimensions, particularly daily activities and symptoms.

TABLE 19 - Summary of results from mixed-effects model for the SAQLI (n = 83)

Note

Period effects not shown.

SAQLI (n = 83)	Mean (SD)	Coefficient	95% CI	p-value	Global p-value
Constant		4.79 (0.15)	4.50 to 5.09	< 0.001
Difference in SAQLI compared with no treatment
No treatment	5.01 (1.24)	–	–	–	< 0.001
SP1	5.25 (1.20)	0.27 (0.10)	0.07 to 0.48	0.008
SP2	5.60 (1.12)	0.62 (0.12)	0.38 to 0.86	< 0.001
bMAD	5.64 (1.06)	0.65 (0.13)	0.41 to 0.90	< 0.001

FIGURE 8.

Estimated mean SAQLI score and 95% CI for the different treatments from the linear mixed-effects model.

TABLE 20 - Comparison of total SAQLI score between different MADs

Note

These comparisons are not adjusted for multiple comparisons. Differences are significant at 5% level according to the Bonferroni method if the p-value is < 0.017.

Comparison	Observed contrast	95% CI	p-value
SP2 to SP1	0.35	0.13 to 0.57	0.002
bMAD to SP1	0.38	0.17 to 0.59	< 0.001
bMAD to SP2	0.03	−0.20 to 0.26	0.785

FIGURE 9.

Box plots of the mean score for each domain of the SAQLI.

Short Form questionnaire-36 items

Summaries of results for the SF-36 standardised PCS and MCS are shown in Table 21 and Figures 10 and 11. Predictably, this general HRQoL instrument is less sensitive to differences between the treatments than the disease-specific instruments, with only the comparison between the SP1 and SP2 showing a borderline significant difference in PCS in favour of the SP2. There was a similar borderline significant increase in the MCS for the bMAD compared with the SP1.

TABLE 21 - Summary of results from mixed-effects model for the SF-36 standardised PCS and MCS (n = 83)

Note

Period effects not shown.

PCS (n = 83)	Mean (SD)	Coefficient	95% CI	p-value	Global p-value
Constant		41.61 (1.58)	38.51 to 44.71	< 0.001
Difference in PCS compared with no treatment
No treatment	43.06 (12.86)	–	–	–	0.058
SP1	42.73 (12.22)	−0.17 (0.84)	−2.27 to 1.92	0.990
SP2	45.11 (12.33)	2.42 (1.17)	0.38 to 4.45	0.145
bMAD	43.12 (13.81)	0.48 (1.22)	−1.74 to 2.70	0.386
MCS (n = 83)	Mean (SD)	Coefficient	95% CI	p-value	Global p-value
Constant		45.24 (1.27)	42.75 to 47.72	< 0.001
Difference in MCS compared with no treatment
No treatment	46.20 (10.78)	–	–	–	0.112
SP1	46.87 (9.63)	0.89 (1.02)	−1.10 to 2.88	0.380
SP2	47.34 (11.24)	1.20 (0.93)	−0.62 to 3.01	0.198
bMAD	48.81 (9.00)	2.72 (1.20)	0.36 to 5.08	0.024

FIGURE 10.

Standardised SF-36 physical health summary.

FIGURE 11.

Standardised SF-36 mental health summary.

Protrusion achieved

The protrusion achieved was measured for all three devices at Papworth Hospital (Table 22). The SP1 achieved the greatest protrusion, being 0.89 mm (95% CI 0.42 to 1.37 mm; p < 0.001) greater than the SP2 and 0.66 mm (95% CI 0.17 to 1.14 mm; p = 0.008) greater than the bMAD. In a model that contained the degree of protrusion, MAD and time period, protrusion did not influence AHI [hazard ratio (HR) 0.997, 95% CI 0.991 to 1.001; p = 0. 206]. Protrusion did have a small effect on the probability of a response to treatment (see earlier section on predictors of response to treatment).

Learning effect

The SP1 and SP2 devices were moulded and protruded by patients independently; in contrast, in the case of the bMADs, protrusion was determined by a medical professional. Variability between the mean protrusion values for each device may have been the result of a variety of factors, including patient-determined compared with clinician-determined protrusion and previous experience of wearing a device on the trial. For example, the SP1 was moulded at the start of that treatment period. Therefore, in patients with SP1 as their second or third device, jaw protrusion may have been either more or less depending on acclimatisation to jaw protrusion and any positive or negative effects experienced while using previous devices. A few patients may have been unintentionally guided by their experience of the bMAD-fitting process. Six patients commented that they had found the visit to the maxillofacial team for bMAD fitting useful in subsequently informing SP2 moulding, including protrusion. Two patients commented that the bMAD-fitting experience helped when they later moulded the SP1. Two others did not describe inadvertent dental guidance, but found the SP1 easier to fit having already made the SP2 mould.

Safety reporting

Partner-evaluated snoring scale

Fifty sleeping partners of trial patients completed the snoring VAS for all four periods (Tables 24 and 25 and Figure 12). This showed a clear improvement for all MADs compared with no treatment, and between the SP1 and the two more sophisticated devices.

FIGURE 12.

Partner-evaluated snoring scale.

Adverse events

There were four SAEs during the trial. There was one case of sick sinus syndrome with atrial flutter and one case of hypoglycaemia during periods of no treatment, both considered possibly related to OSAH, one case of complete heart block and one case of non-specific chest pain during bMAD use, both considered possibly related to OSAH and MAD use. These occurred in four separate patients and all events were resolved within 7 days.

A total of 851 minor AEs were recorded in 86 patients who enrolled in the trial (Table 26). These were mainly mouth discomfort and excess salivation. They were recorded equally frequently for all three MADs and less frequently during the no-treatment periods. Among patients who withdrew from the study, there were 63 AEs in 12 patients, mainly mouth discomfort (52, 83%). Almost all minor AEs in both completers and withdrawals were classed as probably related to MADs (528 events in 85 patients) or possibly related to both OSAH and MAD use (174 events in 54 patients) by an independent sleep physician.

Specific events included in each category are given in Appendix 9.

Trial-based economic analysis

Costs

Table 27 shows that the SP1 device cost the least (£1.62) pro rata over the 4-week trial period, followed by SP2 (£9.85), and that the bMAD is considerably more expensive (£28.64). The mean non-device costs during the no-treatment period were £78.50, while they were £73.02 for SP1, £53.58 for SP2 and £76.25 for bMAD (Table 27). Figure 13 shows box plots of total costs for each group. While costs were similarly clustered for each trial group, SP2 had the narrowest spread of cost. The bunching of outliers close to the upper quartile tend to comprise patients with more frequent primary care (e.g. to dentist or GP) or outpatient visits and these occurred in all groups. However, in both the control and bMAD groups, a few patients incurred very high costs as a result of rare events such as an atrial flutter, pacemaker implantation and hypertension with chest pain.

TABLE 27 - Trial-based comparison on costs incurred over 4 weeks

Five participants required an additional fitting visit for bMAD, cost at £92.04. Average across all participants = £5.98.

Intervention cost components	No treatment (n = 78)	SP1 (n = 81)	SP2 (n = 78)	bMAD (n = 77)
Device costs (fixed)	–	£21.00	£128.00	£350.00
Measurement for device	–	–	–	£110.37
Fitting of device	–	–	–	£92.04
Additional fitting visit if required (average across all patients)	–	–	–	£5.98a
Subtotal	–	£21.00	£128.00	£558.39
Device lifespan (months) (fixed)	–	12	12	18
Fixed cost of intervention – pro rata (4 weeks) subtotal	–	£1.62	£9.85	£28.64
	Mean (SE)	Mean (SE)	Mean (SE)	Mean (SE)
Variable resource use cost (4 weeks)	£78.50 (£19.97)	£73.02 (£10.47)	£53.58 (£8.05)	£76.25 (£24.40)
Total 4-week costs	£78.50 (£19.97)	£74.64 (£10.47)	£63.43 (£8.05)	£104.89 (£24.39)

FIGURE 13.

Box plots of total cost during each 4-week treatment period.

Combining the device and resource use costs and comparing each intervention with control over the 4-week intervention period shows that the SP1 compared with control was £4 less (SE £21), and that SP2 was £15 less on average (SE £21), but that the mean cost of bMAD was £26 greater than mean costs in the control group (SE £28) (Table 27). Differences were not statistically significant.

Quality-adjusted life-years

Figures 14 and 15 show the distribution of the QALY scores at baseline and by treatment group using the EQ-5D-3L and SF-6D. The EQ-5D-3L shows that the SP2 and bMAD have better profiles, with more people scoring around 0.078 or above and bMAD also having fewer people with scores around zero. Of those people with low outlying QALY scores, one person had consistently low scores during baseline and all four intervention periods and another at baseline and three treatment periods. The remaining differences show two participants with QALYs outside the lower IQR while on the SP2 and bMAD, and one participant during the no-treatment period. In only one case did the participants who reported lower EQ-5D-3L QALY scores also accrue higher costs, and this was during the no-treatment phase.

FIGURE 14.

Box plot of EQ-5D-3L QALY results by treatment.

FIGURE 15.

Box plot of SF-6D QALY results by treatment.

The mean QALY score based on the EQ-5D-3L data for the control period was 0.065 (SE 0.002). To give some perspective, 4 weeks in perfect health is associated with a QALY score of (1 × 4)/52 = 0.0769. The control score is, therefore, less than perfect health, equating to a QALY score of 0.065. The difference in EQ-5D-3L QALY values for each MAD compared with no treatment (see Appendices 10 and 11) was 0.0009 (SE 0.001) for SP1, 0.0009 (SE 0.001) for SP2 and 0.0018 (SE 0.001) for bMAD (see Table 28). Although the gain was greatest for bMAD, there was substantial uncertainty, shown by the large SEs. The 95% CI for the effectiveness of each device compared with control spanned zero (i.e. no statistically significant effect).

The SF-6D showed that the SP2 conferred the best health outcomes, with mean QALY score around 0.057, followed by bMAD and no treatment, around 0.053, and SP1, around 0.052. The one participant recorded as an outlier using SF-6D QALYs had fewer QALYs on no treatment, SP2 and bMAD. As with the EQ-5D-3L QALYs, those with low outlying QALY values were not necessarily those with higher costs. The mean QALY score from the mixed-effects model for the control period was 0.053 (SE 0.0008). The difference in SF-6D QALYs (see Appendices 11 and 12) compared with no treatment during the 4-week intervention period was 0.0004 (SE 0.0008) for the SP1, 0.0019 (SE 0.0007) for SP2 and 0.0009 (SE 0.0009) for the bMAD. Of all the MADs, the SP2 showed the greatest change in QALYs compared with control and was also the only intervention with a statistically significant difference (p = 0.013).

Cost-effectiveness

Table 28 shows that the ICERs were negative for the SP1 and SP2 compared with control, i.e. costs were lower and outcomes better for the two interventions than for no treatment. Note that EQ-5D-3L QALY differences between devices were small and non-significant. Of these two, the SP2 is more beneficial as costs were lower than the SP1.

TABLE 28 - Trial-based comparison of costs and QALYs from devices against control

Resource use and total costs by intervention, estimated using a mixed-effects model controlling for baseline data. All costs in 2011/12 (£).

QALY scores calculated using the area under the curve method to represent the true QALY score for the 4-week intervention period to be consistent with the costs presented. Based on EQ-5D-3L responses.

Cost-effectiveness components	No treatment (n = 78)	SP1 (n = 81)	SP2 (n = 78)	bMAD (n = 77)
	Mean (SE)	Mean (SE)	Mean (SE)	Mean (SE)
Total costs over 4 weeks
Total costa	£78.50 (£19.97)	£74.64 (£10.47)	£63.43 (£8.05)	£104.89 (£24.39)
Incremental cost (MAD – no treatment)	–	−£3.87 (£21.38)	−£15.08 (£20.62)	£26.39 (£27.94)
Total utility over 4 weeks
QALYb	0.0649 (0.0017)	0.0658 (0.0017)	0.0658 (0.0019)	0.0667 (0.0017)
Incremental QALY (MAD – no treatment)	–	0.00094 (0.00105)	0.00088 (0.00123)	0.00177 (0.00147)
Cost-effectiveness measure (UK £, 2011)
ICER	–	dominant	dominant	£14,876
INMB (WTP = £20,000) vs. no treatment	–	£23	£33	£9

Table 28 also shows that bMADs have the greatest impact on QALY gain, and at a cost of £14,900 per additional QALY gained, would be considered a cost-effective treatment compared with control. However, compared with the SP2, the bMAD costs an additional £46,000 per QALY (£105 – £64)/(0.0667 – 0.0658 QALYs). These results are mirrored by the net monetary benefit, which shows that the SP2 achieved the highest INMB, compared with no treatment, at £33 per 4 weeks assuming a WTP of £20,000 per QALY (Table 28).

The uncertainty around estimates of cost per QALY gained is represented in the cost-effectiveness planes (Figures 16–18) and CEAF (Figure 19). These indicate the results are robust. The CEAF (Figure 19) shows the SP2 to be most cost-effective up to a WTP per QALY of £39,800, at which point the bMAD supersedes it (39% likelihood of being cost-effective compared with 35% for the SP2). Below a WTP of £5000 per QALY only SP2 is more cost-effective than no treatment. Deterministic sensitivity analyses also showed that results are robust to using only complete case analysis as well as changes in a device’s price and lifespan (see Appendix 13, Figures 36–39). When the bMAD price exceeds £525 or average lifespan falls < 14 months, it no longer has a positive INMB. When the price of the bMAD falls to below £60, or its length of life extends to beyond 3 years (with no change in the SP1), it becomes more cost-effective than the SP1. However, even when assuming the same price for the bMAD of £60 or that its lifetime is at least 5 years, the bMAD remains less cost-effective than the SP2.

FIGURE 16.

Incremental cost-effectiveness plane: SP1 compared with no treatment.

FIGURE 17.

Incremental cost-effectiveness plane: SP2 compared with no treatment.

FIGURE 18.

Incremental cost-effectiveness plane: bMAD compared with no treatment.

FIGURE 19.

Cost-effectiveness acceptability frontier for each MAD compared with no treatment.

The cost-effectiveness analysis was repeated using the SF-6D data for health outcomes. Compared with no treatment, the SP1 has a QALY gain of 0.0004 (SE 0.0007) with the same cost saving described above (−£4 vs. control), meaning the SP1 was both cheaper and more effective, dominating no treatment. However, neither the difference in costs compared with no treatment nor the difference in health outcomes was statistically significant. The SP2 had a statistically significant improvement in health outcomes when compared with no treatment of 0.0019 (SE 0.0007) QALYs, with a p-value equal to 0.013. Combined with the costs saving of £15 over 4 weeks compared with no treatment, showing the SP2 to be dominant over no treatment; being both cheaper and more effective than no treatment. The bMAD provided an improvement in health outcomes compared with no treatment of 0.0009 (SE 0.0009) QALYs, although this was not statistically significant. The bMAD cost £26 more than the no-treatment control, giving an ICER of £30,743 per QALY.

Applying a WTP per QALY of £20,000 the INMB of each treatment compared with control was calculated. The INMB for the SP1 was £12, for the SP2 £52 and for the bMAD £9. Probabilistic sensitivity analysis was used to produce the CEAC and CEAF based on the SF-6D results (Appendix 13, Figure 44) representing the uncertainty in costs and QALY estimates. This analysis found the SP2 to have the highest probability of being the most cost-effective treatment at all WTP thresholds per QALY. Above a WTP of £20,000, the SP2 had a probability of being the most cost-effective in excess of 95% compared with the SP1, bMAD or no-treatment alternatives.

Primary objectives

The primary objective was to update previously conducted systematic reviews of the effect on OSAH of treatment by MADs and by CPAP compared with each other and with no active treatment. However, this review will be stratified by severity of OSAH.

To estimate the effect on AHI and ESS score of treatment by MADs and by CPAP compared with each other and with controls in three meta-analyses of RCTs. Heterogeneity as a result of OSAH severity, trial methodology and duration of follow-up will be assessed. Results from this analysis will feed directly into the economic modelling in Chapter 4.

Secondary objectives

The secondary objectives were to estimate the effect on the secondary outcomes, daytime BP and the QoL scales SAQLI and FOSQ, of treatment by MADs and by CPAP compared with each other and with controls for use in the long-term cost-effectiveness model, for mild to moderate OSAH.

Long-term effects of treatment on cardiovascular risk and RTAs will be assessed as part of the decision model development described in Chapter 4 and are not studied further here.

Search strategy

The systematic review searched for all RCTs of adult OSAH patients in which at least one arm was randomised to MAD or CPAP. Studies comparing two different MADs or two different types of CPAP delivery were excluded since the differences between treatment modalities are known to be small and numerous different devices were trialled. Animal studies and non-randomised studies were excluded. Trials published in a language other than English were excluded.

Information sources

The search strategy updated two existing systematic reviews8,51 to August 2013 (see Appendix 14 for full search strategies). The main stages of the search are described below:

1. All studies from the published McDaid et al. 8 systematic review were included. McDaid et al. 8 (York University Centres for Reviews and Dissemination and for Health Economics) searched 14 databases up to November 2006 and included all RCTs of CPAP compared with either MADs or a non-MAD control. This search was repeated in 2012 by McDaid et al. 8 and the results were shared with the TOMADO group. This search strategy was again replicated, by the same authors, to retrieve articles from March 2012 to August 2013 using MEDLINE, EMBASE and the Science Citation Index, the three most sensitive databases reported by McDaid et al.,8 in order to identify recent trials.

2. The search by McDaid et al. 8 did not include studies of MADs against non-CPAP controls. Therefore, additional papers were identified from the review by Lim et al. ,51 and an updated version of the Lim strategy rerun to cover the period June 2008 to August 2013, using MEDLINE, EMBASE and the Science Citation Index.

3. Reference lists of papers were also searched and were supplemented by the research team’s knowledge of the area to identify other trials missed in updated searches.

Inclusion criteria

All studies identified in the previously published McDaid et al. 8 and Lim et al. 51 searches were reviewed. For the subsequent searches, titles and abstracts were screened independently for relevance by two members of the TOMADO team (two of MB, MG, AC-J, RC and MP). Disagreements were resolved by consensus.

Data extraction

For previously published reviews, estimates of treatment effects recorded in the relevant papers were used, with the exception of a small number of transcribing errors identified and corrected in this meta-analysis. 8,51 For the newly identified studies, information from full papers was extracted independently by two members of the TOMADO study team (two of MB, MG, AC-J, RC and MP) and entered onto a bespoke data extraction form. Any queries were resolved by consensus. Studies that were reported only as abstracts were included provided that they included sufficient information to confirm inclusion criteria and details of results that could be used in one of the meta-analyses. For the updated review of MADs, if data in the published abstract, index paper, or a related publication were unclear, the authors were approached for further information. Owing to the timescale of the study it was not possible to pursue authors of trials involving CPAP for data that were not published in the abstract, index paper, or a related publication.

Data extracted included details of the patient population and baseline characteristics, intervention and comparator, outcome measurements, details of trial methodology, treatment duration and results.

Mean differences between the groups for continuous outcomes, and SEs of the group differences, were extracted for the meta-analysis. Outcomes at the end of the treatment period were preferred. In a small number of studies only the change in the outcome measure was reported, and this was included on the basis that the expected values of baseline measurements in randomised trials should be equivalent, although the SEs may be inaccurate.

Quality assessment

The Jadad score was calculated as a measure of quality that also facilitated consistency with previous published reviews. 8,51 For studies from previously published reviews, the scores were reassessed and any discrepancies between published and new Jadad scores were resolved by discussion. For newly identified studies and where it was missing from studies in published reviews, the Jadad score was calculated by one reviewer and checked by a second.

Publication bias

Funnel plots were examined as an informal method of assessing publication bias. These plots showed little evidence of asymmetry but, with the exception of analyses of primary outcomes for comparisons of CPAP against CM, the numbers of studies were too small to allow more formal analysis.

Data analysis

Three separate series of meta-analyses were conducted, one for each of the comparisons (i) MAD with non-CPAP controls, (ii) MAD with CPAP and (iii) CPAP with non-MAD controls. Meta-analyses used random-effects methods and were implemented using metan and related commands in Stata version 13.0. 64 In brief, this model was formulated as follows. From each study, i, we have an estimate of the treatment effect compared with the control treatment as β^i and we assume that these estimates follow a Gaussian distribution with,

where β_i is the underlying mean treatment effect and σi2 is the standard error in trial i. We assume that the trials are exchangeable a priori and that the underlying trial parameters β_i are drawn from a Gaussian distribution with mean μ = E[β_i] and variance τ² = Var[β_i].

The methods used in Stata follow DerSimonian and Laird,64 who take a classical approach to random-effects meta-analysis. The expected treatment effect μ is estimated as the weighted average,

where the weights are given by the inverse of the estimated total variance w^i=1/(σi2+τ2).

The SE of μ^ is approximated by (1/∑w^i) and an approximate 95% CI is given by,

Although the data analysis in TOMADO was consistent with an overdispersed Poisson distribution for AHI results, which is usual for an event rate, all other publications assumed that AHI followed a Normal distribution. Therefore, the TOMADO results were reanalysed assuming that AHI was Normally distributed, and this estimate is included in the meta-analysis for consistency. For studies in which the rate is of the order of 20, the Normal distribution is a valid approximation, but for smaller values the SEs will be biased, and this should be taken into account when interpreting results. 65 All other outcomes were assumed to be Normally distributed. The SE term σi2 was estimated by the within-trial SE error. Where only 95% CIs were available the SE was estimated using (upper limit – lower limit)/3.92.

The FOSQ has been calculated according to the original scoring system56 in some papers and according to the revision (manual scoring revision dated 11 July 2000) in others. Where possible, the scores have been recalculated according to the latter.

Heterogeneity between studies was represented by the I²-statistic and the chi-squared test for heterogeneity. 66 In order to investigate the sources of heterogeneity, the combined treatment effects for AHI and ESS score were re-estimated in each of the following subgroups:

1. Mean baseline AHI/DI: mild (AHI 5–14.99 events/hour, DI 5–9.99), moderate (AHI 15–29.99 events/hour, DI 10–29.99) and severe (AHI > 30 events/hour, DI > 30).

2. Mean baseline ESS score: mild (0–9), moderate (10–15) and severe (16–24).

3. Study design: parallel and crossover.

4. Treatment duration for studies involving MADs: short (2–12 weeks) and long (> 12 weeks); and for studies of CPAP against CM: short (2–4 weeks) medium (5–12 weeks) and long (> 12 weeks).

In addition, trial results for mild and moderate OSAH groups were combined and results recalculated to feed into the economic model in Chapter 4.

Quantity and quality of studies

Figure 20 summarises studies identified at each stage of the search process. The updated search conducted by York Centre for Health Economics, which was responsible for the McDaid et al. 8 review, and the two searches conducted by the TOMADO team identified 7341 references. After removing duplicates and adding in additional references from other sources, the total number screened for relevance was 4404. After screening, 83 full articles were retrieved and read in detail, of which 27 were eligible for inclusion in the study. These were combined with 44 studies identified from previous reviews that satisfied the inclusion criteria and that had not been superseded by a later publication from the same study. These 71 studies are listed in Appendix 15. Three studies included comparisons involving MADs, CPAP and CM and so each contribute to three separate comparisons, a total of 77 separate comparisons. 23,67,68 There was a greater number of studies of the effectiveness of CPAP than of MADs (Figure 20). The characteristics of the 56 excluded studies are listed in Appendix 16.

FIGURE 20.

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram.

Summary of included studies

Summaries of the baseline characteristics for the included studies are shown in Tables 29–31 for the three comparisons. There were 12 studies including 629 patients that compared MADs with CM, 13 studies with 746 patients comparing MADs with CPAP, and 52 studies with 5400 patients comparing CPAP with CM.

Primary outcome I: apnoea–hypopnoea index

Mandibular advancement devices compared with non-continuous positive airway pressure controls

Twelve studies, including TOMADO, and 629 patients provided an estimate of the effect of AHI, but one of these69 provided only a point estimate and could not be included in the meta-analysis (Figure 21). After combining the studies, the mean difference (reduction) in AHI for MADs compared with control groups was −9.29 (95% CI −12.28 to −6.30; p < 0.001). There was significant heterogeneity between studies (I² = 60%; p = 0.005). Figure 22 suggests that this partly arises from differences in baseline AHI, although the relationship is not monotonic and heterogeneity within these strata remains. Note that only two studies were in patients with mild OSAH and the treatment effect for these studies differed by more than nine events per hour. Seven studies reported baseline ESS score, of which six had moderate EDS according to ESS score; the other had severe EDS. Restricting analysis to the six studies with moderate EDS according to baseline ESS score resulted in less heterogeneity (I² = 35; p = 0.177), with mean difference in AHI as a result of MADs of −6.69 (95% CI −8.98 to −4.41) (Table 32). Six of the 11 studies had a crossover design and these studies had more heterogeneous results than parallel-group trials (Table 32). Treatment effects were greater in crossover trials than in parallel-groups designs, although the difference was not large. In addition, treatment effects in trials of short duration were larger than in longer-term trials, which may indicate reduced compliance over time or progression in the underlying mechanisms of OSAH.

FIGURE 21.

Meta-analysis of AHI results from trials of MADs compared with CM. Note that weights are from random-effects analysis. ES, effect size.

FIGURE 22.

Meta-analysis of AHI results from trials of MADs compared with CM, stratified by baseline AHI. Note that weights are from random-effects analysis. Mean baseline AHI/DI: mild (AHI 5–14.99 events/hour, DI 5–9.99); moderate (AHI 15–29.99 events/hour, DI 10–29.99); and severe (AHI > 30 events/hour, DI > 30). ES, effect size.

TABLE 32 - Subgroup analysis of AHI results (events per hour) for comparison of MADs with non-CPAP controls (negative estimates favour MAD)

Mean baseline AHI/DI: mild (AHI 5–14.99 events/hour, DI 5–9.99); moderate (AHI 15–29.99 events/hour, DI 10–29.99); and severe (AHI > 30 events/hour, DI > 30).

Mean baseline ESS score: mild (0–9); moderate (10–15); and severe (16–24).

Study design: parallel and crossover.

Subgroup	Number of studies	Difference in AHI: MAD–control (95% CI)	p-value for effect	I 2	Heterogeneity p-value
Baseline AHI
Mild	2	−7.79 (−16.38 to 0.79)	0.075	65%	0.091
Moderate	6	−10.72 (−14.59 to −6.85)	< 0.001	52%	0.064
Severe	3	−7.95 (−15.94 to −0.05)	0.051	32%	0.232
Baseline ESS score
Moderate	6	−6.69 (−8.98 to −4.41)	< 0.001	35%	0.177
Severe	1	−2.10 (−12.33 to 8.13)	0.687	–	–
Trial design
Crossover	6	−10.17 (−14.27 −6.07)	< 0.001	76%	0.001
Parallel	5	−8.57 (−12.39 to −4.75)	< 0.001	0%	0.533
Duration of treatment
2–12 weeks	8	−9.69 (−13.27 to −6.12)	< 0.001	68%	0.003
> 12 weeks	3	−6.78 (−13.24 to −0.33)	0.039	23%	0.560
Overall MAD compared with control
Overall	11	−9.29 (−12.28 to −6.30)	< 0.001	60%	0.005

Mandibular advancement devices compared with continuous positive airway pressure

Thirteen trials including 746 patients compared MADs with CPAP. The estimated overall difference in AHI was 7.03 events/hour (95% CI 5.41 to 8.66 events/hour; p < 0.001), with post-treatment AHI being lower in those treated with CPAP than in those treated with MADs (Figure 23). Again there was important heterogeneity between study results, with smaller studies78–80,82 and early studies75–80 estimating greater effects than larger and later studies. No MADs–CPAP head-to-head comparisons were reported in patients with mild baseline AHI (Table 33). Estimates of the difference in post-treatment AHI were consistent and were not related to baseline AHI, baseline ESS score, trial design or duration of treatment, with all significantly lower after CPAP (Table 33).

FIGURE 23.

Meta-analysis of AHI results from trials of MADs compared with CPAP. Note that weights are from random-effects analysis. ES, effect size.

TABLE 33 - Subgroup analysis of AHI results (events per hour) for comparison of MADs with CPAP (positive estimates favour CPAP)

Mean baseline AHI/DI: mild (AHI 5–14.99 events/hour, DI 5–9.99); moderate (AHI 15–29.99 events/hour, DI 10–29.99); and severe (AHI > 30 events/hour, DI > 30).

Mean baseline ESS score: mild (0–9); moderate (10–15); and severe (16–24).

Study design: parallel and crossover.

Treatment duration for studies involving MAD: short (2–12 weeks) and long (> 12 weeks); and for studies of CPAP against no active treatment controls: short (2–4 weeks), medium (5–12 weeks) and long (> 12 weeks).

Subgroup	Number of studies	Difference in AHI: MAD–CPAP (95% CI)	p-value for effect	I 2	Heterogeneity p-value
Baseline AHI
Moderate	8	7.48 (5.77 to 9.19)	< 0.001	28%	0.203
Severe	4	7.22 (3.20 to 11.25)	< 0.001	74%	0.010
Baseline ESS score
Moderate	9	6.70 (4.86 to 8.54)	< 0.001	57%	0.098
Trial design
Crossover	9	6.91 (5.11 to 8.71)	< 0.001	48%	0.054
Parallel	4	7.72 (3.58 to 11.87)	< 0.001	69%	0.022
Duration of treatment
2–12 weeks	9	7.19 (5.25 to 9.12)	< 0.001	59%	0.013
> 12 weeks	4	6.78 (3.25 to 10.31)	< 0.001	42%	0.157
Overall MAD compared with CPAP
Overall	13	7.03 (5.41 to 8.66)	< 0.001	52%	0.015

Continuous positive airway pressure compared with non-mandibular advancement device controls

Of the 52 trials comparing CPAP with CM, 25 trials including 1596 patients reported post-treatment AHI. The estimated effect from combining these studies was −25.37 (95% CI −30.67 to −20.07; p < 0.001) (Figure 24). There was a significant amount of heterogeneity between study results, both overall and within strata. Some of this is explained by baseline AHI, and the potential for treatment effect is naturally governed by the extent of disease in the population. Only one of these studies was in patients with mild baseline AHI128 and the estimated mean effect in this trial was small at −2.40 events/hour (95% CI −3.67 to −1.13 events/hour). Moreover, the mean difference in AHI between CPAP and control patients increased with baseline severity, from −13.67 (95% CI −16.13 to −11.20) for moderate OSAH according to AHI at baseline to −33.04 (95% CI −39.75 to −26.34) for severe (Table 34). The pattern was somewhat different for groups defined by the baseline measure of subjective daytime sleepiness, i.e. the ESS score. Only one study reported mild baseline EDS according to ESS score110 but had severe baseline OSAH according to AHI, so that the estimated effect of CPAP on AHI was large at −32.50 (95% CI −43.55 to −21.45) (Table 34). However, the studies in patients with moderate baseline EDS according to ESS score reported a smaller effect of CPAP compared with controls than those with severe baseline EDS. There was some evidence that the treatment effect was lower for crossover trials than for parallel-group trials and for trials with longer treatment duration (Table 34).

FIGURE 24.

Meta-analysis of AHI results from trials of CPAP compared with CM, stratified by baseline AHI. Note that weights are from random-effects analysis. ES, effect size.

TABLE 34 - Subgroup analysis of AHI results (events/hour) for comparison of CPAP with non-MAD controls (negative estimates favour CPAP)

Mean baseline AHI/DI: mild (AHI 5–14.99 events/hour, DI 5–9.99); moderate (AHI 15–29.99 events/hour, DI 10–29.99); and severe (AHI > 30 events/hour, DI > 30).

Mean baseline ESS score: mild (0–9); moderate (10–15); and severe (16–24).

Study design: parallel and crossover.

Treatment duration for studies of CPAP against no active treatment controls: short (2–4 weeks), medium (5–12 weeks) and long (> 12 weeks).

Subgroup	Number of studies	Difference in AHI: CPAP–control (95% CI)	p-value for effect	I 2	Heterogeneity p-value
Baseline AHI
Mild	1	−2.40 (−3.67 to −1.13)	< 0.001	–	–
Moderate	7	−13.67 (−16.13 to −11.20)	< 0.001	47%	0.081
Severe	17	−33.04 (−39.75 to −26.34)	< 0.001	90%	< 0.001
Baseline ESS score
Mild	1	−32.50 (−43.55 to −21.45)	< 0.001	–	–
Moderate	15	−17.54 (−22.51 to −12.56)	< 0.001	95%	< 0.001
Severe	3	−34.73 (−58.90 to −10.57)	0.005	95%	< 0.001
Trial design
Crossover	5	−19.71 (−27.95 to −11.48)	< 0.001	87%	< 0.001
Parallel	20	−27.08 (−33.68 to −20.48)	< 0.001	97%	< 0.001
Duration of treatment
2–4 weeks	11	−32.90 (−43.78 to −22.02)	< 0.001	93%	< 0.001
5–12 weeks	11	−22.34 (−29.84 to −14.85)	< 0.001	96%	< 0.001
> 12 weeks	3	−14.25 (−19.03 to −9.46)	< 0.001	82%	0.004
Overall CPAP compared with controls
Overall	25	−25.37 (−30.67 to −20.07)	< 0.001	96%	< 0.001

Primary outcome II: Epworth Sleepiness Scale

Mandibular advancement devices compared with non-continuous positive airway pressure controls

Of the 12 studies, 10 reported a point estimate of the effect of ESS score on MADs, but only nine reported sufficient data to allow calculation of the SE of the treatment effect. These nine studies included 485 patients and the combined treatment effect on ESS score was −1.64 (95% CI −2.46 to −0.82) (Figure 25 and Table 35). Again there was significant heterogeneity between study results, with small studies70,73 more likely to report large treatment differences. Only the TOMADO study was conducted in patients with mild OSAH according to AHI at baseline, and the effect on ESS score was between, and of a similar order to, estimates from trials in patient groups with moderate and severe baseline AHI. Patients from one trial by Blanco et al. 70 reported severe baseline EDS according to ESS score and also reported a large treatment effect. This trial was small, randomising 12 patients to either an advanced or a non-advanced mandibular device for a period of 3 months, and reporting on 20 patients who completed treatment. Excluding this trial and restricting analysis to the six trials that had a moderate baseline EDS according to ESS score resulted in a combined treatment difference of −1.36 (95% CI −2.07 to −0.64; p < 0.001). Owing to the small number of trials and the large influence of Blanco’s study it is not possible to reliably assess reasons for heterogeneity further.

FIGURE 25.

Meta-analysis of ESS score results from trials of MADs compared with CM, stratified by baseline AHI. Note that weights are from random-effects analysis. Mean baseline AHI/DI: mild (AHI 5–14.99 events/hour, DI 5–9.99); moderate (AHI 15–29.99 events/hour, DI 10–29.99); and severe (AHI > 30 events/hour, DI > 30). ES, effect size.

TABLE 35 - Subgroup analysis of ESS score results for comparison of MADs with non-CPAP (negative estimates favour MADs)

Mean baseline AHI/DI: mild (AHI 5–14.99 events/hour, DI 5–9.99); moderate (AHI 15–29.99 events/hour, DI 10–29.99); and severe (AHI > 30 events/hour, DI > 30).

Mean baseline ESS score: mild (0–9); moderate (10–15); and severe (16–24).

Study design: parallel and crossover.

Treatment duration for studies involving MAD: short (2–12 weeks) and long (> 12 weeks).

Subgroup	Number of studies	Difference in ESS score: MAD–controls (95% CI)	p-value for effect	I 2	Heterogeneity p-value
Baseline AHI
Mild	1	−2.01 (−2.70 to −1.32)	< 0.001	42%	0.142
Moderate	5	−1.38 (−2.48 to −0.27)	0.150	73%	0.025
Severe	3	−2.68 (−5.89 to 0.54)	0.103	48%	0.051
Baseline ESS score
Moderate	6	−1.36 (−2.07 to −0.64)	< 0.001	–	–
Severe	1	−8.50 (−13.64 to −3.36)	0.001	55%	0.037
Trial design
Crossover	4	−1.75 (−2.25 to −1.25)	< 0.001	2%	0.380
Parallel	5	−2.18 (−4.80 to 0.44)	0.102	68%	0.015
Duration of treatment
2–12 weeks	7	−1.75 (−2.22 to −1.28)	< 0.001	0%	0.521
> 12 weeks	2	−3.26 (−13.15 to 6.63)	0.518	90%	0.001
Overall MAD compared with controls
Overall	9	−1.64 (−2.46 to −0.82)	< 0.001	48%	0.051

Mandibular advancement devices compared with continuous positive airway pressure

Of the 12 studies directly comparing MADs and CPAP, 10 trials and 675 patients contributed to the meta-analysis of ESS score results, with a combined estimate of 0.67 (95% CI −0.11 to 1.44; p = 0.093) (Figure 26). The positive estimate indicates that the post-treatment ESS score was lower (better) in the CPAP group. There was less between-study heterogeneity in this analysis and the results of stratified analysis show that any treatment effect is small, with clinically significant differences likely only for those with severe baseline OSAH according to AHI (Table 36). However, the number and size of trials remains too small to make reliable conclusions, particularly regarding mild OSAH.

FIGURE 26.

Meta-analysis of ESS score results from trials of MADs compared with CPAP, stratified by baseline AHI. Note that weights are from random-effects analysis. ES, effect size.

TABLE 36 - Subgroup analysis of ESS score results for comparison of MADs with CPAP (positive estimates favour CPAP)

Mean baseline AHI/DI: mild (AHI 5–14.99 events/hour, DI 5–9.99); moderate (AHI 15–29.99 events/hour, DI 10–29.99); and severe (AHI > 30 events/hour, DI > 30).

Mean baseline ESS score: mild (0–9); moderate (10–15); and severe (16–24).

Study design: parallel and crossover.

Treatment duration for studies involving MAD: short (2–12 weeks) and long (> 12 weeks); and for studies of CPAP against no active treatment controls: short (2–4 weeks), medium (5–12 weeks) and long (> 12 weeks).

Subgroup	Number of studies	Difference in ESS score: MAD–CPAP (95% CI)	p-value for effect	I 2	Heterogeneity p-value
Baseline AHI
Moderate	6	0.06 (−0.61 to 0.72)	0.864	0%	0.659
Severe	4	1.42 (−0.24 to 3.08)	0.094	68%	0.024
Baseline ESS score
Moderate	9	0.81 (−0.04 to 1.65)	0.062	49%	0.049
Trial design
Crossover	6	0.54 (−0.48 to 1.57)	0.301	60%	0.030
Parallel	4	0.97 (−0.16 to 2.11)	0.093	0%	0.399
Duration of treatment
2–12 weeks	8	0.82 (−0.09 to 1.73)	0.078	55%	0.031
>12 weeks	2	−0.06 (−1.66 to 1.54)	0.944	0%	0.461
Overall MAD compared with CPAP
Overall	10	0.67 (−0.11 to 1.44)	0.093	45%	0.059

Continuous positive airway pressure compared with non-mandibular advancement device controls

Thirty-eight of the 52 trials comparing CPAP with non-MAD controls reported the estimated post-treatment effect on ESS score, with 4894 patients included in the comparisons. These trials are plotted in Figure 27 and the combined estimate of treatment effect on ESS score was −2.23 (95% CI −2.76 to −1.71; p < 0.001). Again there is significant heterogeneity, and some stratified analyses are reported in Table 37. In common with AHI, the effect of CPAP on ESS score increases with baseline AHI severity, from −1.23 (95% CI −2.19 to −0.27) for the mild group to −2.64 (95% CI −3.44 to −1.84) for the severe group. A similar but steeper effect is seen with increasing baseline ESS score, the effect increasing from −0.83 (95% CI −1.16 to −0.51) for mild baseline EDS according to ESS score to −4.99 (95% CI −6.51 to −3.47) for severe EDS according to ESS score. The trial design has less impact on outcomes but longer duration of treatment is associated with decreasing treatment effect, which again mirrors the analysis of AHI.

FIGURE 27.

Meta-analysis of ESS score results from trials of CPAP compared with CM. Note that weights are from random-effects analysis. ES, effect size.

TABLE 37 - Subgroup analysis of ESS score results for comparison of CPAP against non-MAD controls (negative estimates favour CPAP)

Mean baseline AHI/DI: mild (AHI 5–14.99 events/hour, DI 5–9.99); moderate (AHI 15–29.99 events/hour, DI 10–29.99); and severe (AHI > 30 events/hour, DI > 30).

Mean baseline ESS score: mild (0–9); moderate (10–15); and severe (16–24).

Study design: parallel and crossover.

Treatment duration for studies of CPAP against no active treatment controls: short (2–4 weeks), medium (5–12 weeks) and long (> 12 weeks).

Subgroup	Number of studies	Difference in ESS score: CPAP–control (95% CI)	p-value for effect	I 2	Heterogeneity p-value
Baseline AHI
Mild	5	−1.23 (−2.19 to −0.27)	0.012	59%	0.045
Moderate	10	−1.82 (−2.73 to −0.92)	< 0.001	60%	0.008
Severe	22	−2.64 (−3.44 to −1.84)	< 0.001	86%	< 0.001
Baseline ESS score
Mild	5	−0.83 (−1.16 to −0.51)	< 0.001	30%	0.222
Moderate	28	−2.19(−2.84 to −1.53)	< 0.001	76%	< 0.001
Severe	5	−4.99 (−6.51 to −3.47)	< 0.001	46%	0.115
Trial design
Crossover	12	−2.32 (−3.33 to −1.31)	< 0.001	79%	< 0.001
Parallel	26	−2.15 (−2.74 to −1.55)	< 0.001	82%	< 0.001
Duration of treatment
2–4 weeks	13	−2.58 (−3.66 to −1.51)	< 0.001	75%	< 0.001
5–12 weeks	17	−2.20 (−3.02 to −1.39)	< 0.001	68%	< 0.001
> 12 weeks	8	−1.87 (−2.83 to −0.90)	< 0.001	93%	< 0.001
Overall CPAP compared with controls
Overall	38	−2.23 (−2.76 to −1.71)	< 0.001	83%	< 0.001

Secondary outcome I: daytime blood pressure

Continuous positive airway pressure compared with non-mandibular advancement device controls

Fifteen studies reported daytime BP from 1772 patients (Figures 28 and 29). The combined effect of CPAP on SBP was −2.36 mmHg (95% CI −3.65 to −1.06 mmHg; p < 0.001). Again a smaller difference was estimated for DBP of −1.49 mmHg (95% CI −2.17 to −0.80 mmHg; p < 0.001). As CPAP trials are generally conducted in patients with more severe OSAH, these results have been stratified for baseline AHI level in Table 38 and show that the effect of CPAP on both SBP and DBP increased with increasing AHI.

FIGURE 28.

Meta-analysis of SBP results from trials of CPAP compared with CM. Note that weights are from random-effects analysis. ES, effect size.

FIGURE 29.

Meta-analysis of DBP results from trials of CPAP compared with CM, stratified by baseline AHI. Note that weights are from random-effects analysis. ES, effect size.

Secondary outcome II: sleep-related quality of life

Quality-of-life assessment was restricted to the two sleep-related QoL measures used in the TOMADO study (see Chapter 2), the SAQLI and the FOSQ, and to studies identified in the searches described above.

The McDaid et al. model

McDaid et al. 8 developed a state-transition Markov model to assess the long-term cost-effectiveness of CPAP therapy compared with MAD and CM as part of a NICE technology appraisal. 37 The model charted the movement of a hypothetical cohort of 50-year-old men, with characteristics pooled from a meta-analysis of clinical trials of OSAH interventions. Patients were typically overweight (mean BMI = 30 kg/m²) and had high BP (SBP = 130 mmHg). Baseline EDS, measured by mean ESS score, was 12. Various CPAP devices provided by different manufacturers were treated as one class of intervention. The large numbers of differing MADs used in trials were pooled for an overall treatment effect. CM involved a one-off consultation with a GP, with some level of lifestyle advice on how to reduce or cope with symptoms better. Outcomes were summarised as an incremental cost per QALY for each intervention. The model structure is explained briefly below.

Given the chronic nature of OSAH, the McDaid et al. 8 model adopted a lifetime horizon and incorporated the possibility of CVEs, strokes and involvement in RTAs, as well as accounting for symptomatic effects of OSAH on QoL. Patients started in an OSAH state and were able to move into a number of different health states [OSAH post coronary heart disease (CHD), OSAH post stroke and death], reflecting morbidities linked to long-term OSAH suffering. The model ran on a yearly cycle to chart a hypothetical cohort of 10,000 patients over time.

Figure 30 provides a diagrammatic representation of the model. Elliptical boxes represent health states and square boxes represent events. Arrows show the direction of transitions between health states and the occurrence of events. All members of the cohort started in the OSAH state and could stay in that state, unless a transition occurred, until death. They could move into the post-CHD state if they experienced an acute CVE and survived. This state allowed for the increased morbidity and mortality associated with having had a first CHD event. If they did not survive, they moved to the absorbing death state. If they did survive, they could remain in this post-CHD state until death, or experience a RTA (fatal or non-fatal) or suffer a stroke. If they survived a RTA, they remained in the same health state post event. If they survived a stroke, they moved to the OSAH post-stroke state, where they were again able to remain until death or experience a RTA. They were not able to move back to a CHD state once they had suffered a stroke. Patients who had a disabling stroke were assumed to no longer be able to drive and, hence, a proportion of those in the post-stroke state were not able to have a RTA event.

FIGURE 30.

Long-term model structure developed by McDaid et al. 8

Patients could suffer a stroke while in the initial OSAH state, in which case, if they survived, they would move to the post-stroke health state. Here they would be subject to the increased risk of mortality and morbidity following the first event. Provided the stroke was not disabling they could experience a RTA (fatal or non-fatal). Patients in the initial OSAH state may at some point have experienced a RTA and, provided it was not fatal, would stay in the OSAH state until another transition or death.

Movements between states were determined by a set of transition probabilities, derived from various sources. In the base case, transitions that relate to CVEs and risk of stroke were informed by the Framingham risk equation, utilising information on baseline characteristics of an OSAH population to calculate the probability of a CVE (Table 40). Differences in SBP observed under the treatment options (from a meta-analysis of RCTs) were used in the Framingham equation to differentiate the risk of CVEs and strokes under each intervention. The equation is based on Weibull models, meaning that predicted risk is non-linear with respect to each risk factor. McDaid et al. 8 tested whether or not use of mean BPs would bias the results using a set of individual patient data. From the equation, the risk of CVEs and stroke were predicted using BP for each patient, and the mean taken. This was compared with risk calculated based on the mean of group BPs. The risk calculated by the two different methods was the same to two decimal places and, so, use of aggregate-level data did not significantly bias results. The equation was used to calculate the 4-year probability of an event, with a piece-wise exponential used to convert this into a yearly probability to correspond to the cycle length.

Long-term observational studies were consulted for estimates of the increased risk of mortality following events relating to stroke and CHD once an initial event had occurred. 144,145 The underlying risk of RTAs (fatal and non-fatal) was estimated from Department of Transport146 data and was adjusted based on the OR of RTAs given treatment with CPAP compared with no treatment, taken from an updated meta-analysis by Ayas et al. 136 Given a lack of data on the likelihood of a RTA when using MADs, the ratio of ESS scores for MAD treatment compared with CM was applied to the OR for RTAs of CPAP compared with CM. Symptomatic relief provided by different interventions was accounted for using evidence from a meta-analysis of ESS scores, which were mapped to a QoL scale, in the absence of good HRQoL data. Regression techniques were used to estimate an algorithm for expressing utility changes, as measured by EQ-5D-3L and SF-6D pre-scored preference questionnaires to changes in ESS score. Utilities and costs were assigned to each of the health states and differed depending on the intervention being received. Each health event had an associated utility loss and acute cost attached to the event.

Costs of interventions were estimated in 2005 prices (£), incorporating the cost of devices and any on-going resource usage associated with maintenance and replacement, including equipment, staff time and overheads. CPAP device costs were acquired from McDaid et al. 8 Estimation of resource use during the titration process was taken from a manufacturer’s submission to NICE, which included data elicited from a group of clinicians regarding proportion titrated by different methods in clinical practice to ascertain appropriate costs. The machine was assumed to have a lifespan of 7 years (clinical opinion) and masks replaced annually. It was assumed that the MADs being used was a Thornton Adjustable Positioner^® (Airway Management Inc., Dallas, TX, USA), commonly in use at the time and this was costed according to NHS Dental contract costs, given the lack of an appropriate NHS cost of the device. The lifespan of a MAD was assumed to be 2 years (clinical opinion) compared with 12–18 months in the TOMADO study. Unit costs for NHS resource use (sleep specialist consultations, nurse appointment and GP consultations) were taken from nationally available NHS reference costs, as well as unit costs published by the Personal Social Services Research Unit (PSSRU). 58,147 Published sources were consulted for estimates of the cost of other morbidities (CHD, stroke and RTAs) associated with OSAH. Two economic evaluations which had estimated costs of an acute CHD (and on-going treatment costs of chronic conditions) and stroke events in a NHS setting were used. 148,149 RTA costs were taken from UK Department of Transport estimates. 146 Cost and effects were discounted at 3.5% per annum.

The modelling was implemented in Microsoft Excel (Microsoft Corporation, Redmond, WA, USA) and results presented as ICERs representing the long-term mean cost per QALY gained for the different interventions. Uncertainty was explored using probabilistic techniques, by attaching distributions to input parameters and randomly sampling from them, performing 10,000 iterations to produce estimates of the distributions of the outcome. This uncertainty was summarised using CEACs, showing the likelihood that any given device is cost-effective at a given WTP threshold.

Results from McDaid et al. 8 indicated a 78% probability that CPAP was cost-effective for the hypothetical cohort at a threshold of £20,000 per QALY. At this WTP, MADs and CM had a probability of being cost-effective of 21% and 1%, respectively. Sensitivity analysis suggested that CPAP had the highest probability of being cost-effective over a wide range of WTP thresholds, even for mild and moderate subgroups, though the probability of MADs being cost-effective increased for milder subgroups.

Updating model parameter values

For this cost-effectiveness analysis, the parameters used to populate the economic model were revisited, to update where necessary and possible. Treatment effects were restricted to a mild/moderate severity group of OSAH sufferers and taken from the meta-analyses presented in Chapter 3, which incorporated both TOMADO and other RCT data. Within-trial effects were used in a sensitivity analysis to investigate potential between device differences in long-term cost-effectiveness. Other data from TOMADO used in the model included costs and HRQoL. The remaining data for the economic model were produced following replication of searches first performed by McDaid et al. 8 on cardiovascular risk and sleep apnoea, HRQoL data, and RTA risk and sleep apnoea. A new review on compliance of CPAP and MADs was also conducted. The decision about whether or not new evidence was chosen in preference to that already parameterising the model was based on the following criteria:

• evidence was specific to a mild to moderate OSAH population

• estimates were UK specific or more relevant to the NHS

• data were more robust (based on characteristics such as sample size and study design)

• evidence was contemporary compared with previous estimates or

• new evidence facilitated improved modelling (for instance longer-term data or enabling structural improvements) of OSAH and its treatment.

Mapping Epworth Sleepiness Scale score to European Quality of Life-5 Dimensions three-Level version and Short Form questionnaire-6 Dimensions

TOMADO presented a large dataset of both SF-36 and EQ-5D-3L data for people with mild to moderate OSAH. Given repeated measurements, it yielded 402 data points of ESS score and SF-6D and 404 data points of ESS score and EQ-5D-3L that could be used in a regression-based mapping exercise to estimate an algorithm mapping ESS to utility scores. The algorithms for SF-6D and EQ-5D-3L were estimated using a linear mixed-effects regression model. The ESS score was an explanatory variable; a dummy variable was used to control for differences in baseline utility and participants were included as a random effect. These models rely on an assumption that the residuals are Normally distributed, though this may not always hold. 187 The models are shown in Table 43 for SF-6D and Table 44 for EQ-5D-3L.

TABLE 43 - Mixed-effects model for mapping ESS scores and utility based on SF-6D (n = 402)

Variable	Coefficient	SE	p-value	95% CI
ESS	−0.0067	0.0011	0.000	−0.0087 to −0.0046
Baseline	−0.0020	0.0079	0.799	−0.0175 to 0.0134
Constant	0.7529	0.0116	0.000	0.7302 to 0.7756

TABLE 44 - Mixed-effects model for mapping ESS scores and utility based on EQ-5D-3L (n = 404)

Variable	Coefficient	SE	p-value	95% CI
ESS	−0.0061	0.0020	0.003	−0.0101 to −0.0020
Baseline	0.0139	0.0145	0.340	−0.0146 to 0.0423
Constant	0.9094	0.0220	0.000	0.8664 to 0.9525

Figure 31 shows that the residuals appear to be reasonably close to normality for SF-6D, but less so for the EQ-5D-3L. This is consistent with our a priori knowledge of the discrete nature of the EQ-5D-3L, the ceiling effect often observed in relatively healthy groups of patients188 and the findings of the McDaid et al. 8 mapping exercise. Other studies of utility indices derived from EQ-5D-3L in OSAH sufferers confirm this phenomenon and suggest that SF-6D may display a distribution closer to normality. 189

FIGURE 31.

Residuals from linear model mapping ESS to (a) SF-6D; and (b) EQ-5D-3L utility scores.

The results of this regression analysis indicate that a 1-unit decrease in the ESS is associated with a 0.0061 (p < 0.001) rise in utility based on EQ-5D-3L and a 0.0067 (p < 0.001) rise in utility based on the SF-6D instrument. For probabilistic sensitivity analysis the estimated variance matrix from the linear mixed models was used when sampling from the parameter distributions. The baseline utility of the population in the economic model was estimated based on the mean baseline ESS score of patients in TOMADO. The coefficients in the mapping equations estimated from the TOMADO data were similar to, but slightly lower than, those estimated by McDaid et al. 8 This should be expected as the population of patients recruited to TOMADO had mild to moderate OSAH and so represented a subsection of the range of disease severity.

Treatment effects of the use of MADs and CPAP from the meta-analyses of ESS scores in Chapter 3 were converted into utility increments using the algorithm. The baseline utility was estimated based on the mean ESS score of the trial participants in the TOMADO. The utilities used in the model are shown in Table 45.

TABLE 45 - Utilities for CVEs and RTAs

Utility	Mean	SD	Source
OSAH untreated (baseline)	Baseline ESS score × −0.006 + 0.91	–	TOMADO EQ-5D-3L mapping algorithm
OSAH treated with MAD	ΔESSMAD-CM × −0.006	–	TOMADO EQ-5D-3L mapping algorithm
OSAH treated with CPAP	ΔESSCPAP-CM × −0.006	–	TOMADO EQ-5D-3L mapping algorithm
Stroke (decrement)	−0.0524	0.0002	Sullivan and Gushchyan190
CHD (decrement)	−0.0635	0.0001	Sullivan and Gushchyan190
RTA	0.6200	0.2700	Currie et al.191
Age (decrement per year)	−0.0007	0.0000	Sullivan and Gushchyan190

McDaid et al. 8 relied on data from a study conducted by Sullivan and Ghushcyan,190 which used EQ-5D-3L data from a panel of 38,678 patients to estimate decrements associated with a range of chronic diseases. The utility associated with a RTA was based on EQ-5D-3L data from a data repository 6 weeks after an inpatient episode for injuries sustained from a RTA in the UK. No additional robust sources of utility data were identified and these values were retained.

Methods of analysis

The base case includes a hypothetical cohort of 10,000 men informed by the characteristics of the TOMADO population. These characteristics are shown in Table 40. ESS treatment effects were taken from the meta-analysis stratified to include studies of OSAH that fell into the mild to moderate range according to mean baseline AHI. Costs were based on the SP2 device, with an assumed lifespan of 12 months. All models incorporated the uncertainty around model input parameters by repeatedly sampling (n = 15,000) from the parameter distributions and recalculating model outputs conditional on each sample, in order to estimate the distribution of the outputs. Distributions were chosen dependent on the nature of the parameter being sampled. Gamma distributions were used for unit costs, Normal distributions were used for input parameters that were estimated from regression coefficients (including the Cholesky decomposition of mapped utility values) and log-Normal distributions were used for relative risks. Several scenario analyses were conducted, which still incorporated the probabilistic elements of the modelling and, where relevant, adjusted distributions of input parameters accordingly.

Scenario analyses were conducted to investigate sensitivity of outputs to:

• the lifespan of the interventions

• the cost of devices incorporating SP1 and bMAD costs

• ESS treatment effects observed in TOMADO

• reduced CPAP compliance in lower severity disease using a multiplier

• the time horizon

• use of an alternative source of the relative risk of vascular events given a reduction in SBP

• use of an alternative source for the effect of effective treatment of OSA on RTA events.

All results are presented as incremental cost per QALY. For the base case, uncertainty in the estimates is presented as the likelihood of being cost-effective at WTP thresholds of £10,000, £20,000 and £30,000 per QALY, the CEAC for a range of WTP thresholds and the CEAF to identify the most cost-effective treatment option over the range of WTP thresholds. All costs are in 2011/12 prices.

Base-case analysis

The results of the base case are presented in Table 49. This shows that MADs compared with CM are more costly but also more effective in patients with mild to moderate OSAH. The additional costs are a result of much higher treatment costs, with a reduction in RTA and CVE costs mitigating this difference somewhat. The ICER of MADs compared with CM is £6639 per additional QALY gained. CPAP compared with MADs is more expensive but more effective. The ICER of CPAP compared with MADs is £14,012 per QALY gained.

At a threshold value of £20,000/QALY, CPAP has the highest mean INMB compared with CM (£3879) and the probability that CPAP is cost-effective is 0.52. At a threshold value of £30,000/QALY, this probability increases to 0.55 with a mean INMB of £6914. Oral devices have a mean INMB compared with CM of £3794 at a threshold value of £20,000/QALY and the probability that they are cost-effective is 0.47. At a threshold value of £30,000/QALY, this probability decreases to 0.45 with a mean INMB of £6643.

Figure 32 depicts the uncertainty surrounding decisions of which approach is most cost-effective in the base-case analysis, for a range of values decision-makers may be willing to pay per QALY gained. It shows that at very low WTP thresholds, CM is the most likely to be cost-effective. Over the conventional range of £20,000–£30,000, CPAP has the highest likelihood of being the most cost-effective, with the decision becoming less uncertain as WTP per QALY increases. At a WTP of approximately £20,000/QALY the probability that CM is the most cost-effective falls to zero.

FIGURE 32.

The cost-effectiveness acceptability curves (base-case analysis).

Figure 33 gives the CEAF for the base case. It shows the intervention which yields the highest mean net benefit over the range of WTP. It can be seen that, while MADs have the highest mean net benefit after a threshold of £6687, there does remain uncertainty of whether or not it is likely to be more cost-effective than CM. From £15,367, CPAP becomes cost-effective, and at this point the likelihood of MADs and CPAP being cost-effective is very similar, 0.48 and 0.49, respectively. At higher WTP, CPAP always has the highest mean net benefit and highest likelihood of being the most cost-effective, although with considerable uncertainty.

FIGURE 33.

The cost-effectiveness acceptability frontier (base-case analysis).

Sensitivity analyses

A series of one-way deterministic sensitivity analyses were undertaken to explore the additional impact of changing specific input values on the cost-effectiveness results from the base case. These are presented in Table 50, which shows that decisions are not sensitive to the use of SF-6D utilities scores. This is also true for use of an alternative source of relative risk reduction associated with decreasing SBP. However, results and decisions are sensitive to assumptions about costs. For example, replacing device costs from SP2 with those for SP1 or bMAD costs leads to a different decision about the relative value of CPAP; in the case of SP1, CPAP would no longer be considered cost-effective (ICER = £89,182) by usual NICE threshold values, as the additional benefits of CPAP become relatively more expensive. Replacing SP2 device costs with bMAD leads to CPAP dominating bMAD as the benefits of CPAP are greater and costs are lower than bMAD. This is the case even if the lifespan of bMAD is assumed to be 2 years rather than 18 months.

The assumed lifespan of devices makes a difference to the optimum decision. A conservative estimate for the lifespan of the SP2 based on manufacturer and expert clinical opinion was 1 year. However, if the lifespan is increased to 18 months, SP2 becomes the most cost-effective intervention.

Use of device-specific costs and effects as observed in TOMADO indicates that SP2 dominates CPAP, given the comparatively higher QALYs gained. A comparison of bMAD with CPAP shows that both the costs and benefits of CPAP are lower. However, at a conventional threshold of £20,000 to £30,000 per QALY, the cost savings of CPAP compared with bMAD are larger than the value to ‘compensate’ for lower benefits of CPAP.

If a shorter time horizon is considered, CPAP becomes less cost-effective. This is because much of the benefit of CPAP results from its greater effectiveness in lowering BP. The benefits of reducing this risk factor for CVD would accrue later in patients’ lives.

The Trial of Oral Mandibular Advancement Devices for Obstructive sleep apnoea–hypopnoea

In an adequately powered and efficiently designed randomised study, TOMADO showed that, for patients with mild to moderate OSAH, the rate of apnoea/hypopnoea events per hour for each of the three non-adjustable MADs studied was significantly lower than with no treatment. Although the effect on AHI compared with no treatment increased with sophistication of the MAD, the between-device differences were small and not statistically significant at commonly used thresholds. Arbitrarily defined response to treatment was achieved in just under half the patients when using the SP2 and bMAD and in approximately one-fifth when using no treatment, compared with baseline measurements. The likelihood of response was most closely related to BMI at baseline and during the study. Results for 4% ODI, which is more commonly used in clinical practice in the UK, were very similar to those for AHI.

The effects of MAD on ESS score mirrored those for apnoea/hypopnoea events per hour, with the SP2 and bMAD having a greater effect than the SP1.

Although the trial treatments were administered over a short time period, some evidence of compliance with treatment emerged. This indicated that one reason for the poorer performance of the SP1 may be lower compliance, as shown by the shorter duration of use per night and greater likelihood of discontinuation during the treatment period. The SP1 was also less likely to be chosen as the preferred device by those patients who completed the trial. Similarly, partners of the trial patients reported improvements in snoring during MAD use, with the SP1 having a weaker effect than SP2 and bMAD.

The relationship between MAD treatment and sleepiness-related functioning and QoL (FOSQ and SAQLI) showed a similar pattern to that for AHI and ESS outcomes, with significant effects for all MADs compared with no treatment, and the SP1 performing less well than SP2 and bMAD. Detailed examination of the questionnaires suggested small improvements across all dimensions, but that scales measuring the effect of sleepiness on activities (FOSQ, SAQLI), general productivity (FOSQ) and symptoms (SAQLI) were particularly affected by MAD treatment. General HRQoL measures were largely insensitive to MAD treatment, with the exception that the SP2 was associated with significantly higher SF-6D QALYs compared with control (accounting for baseline differences).

A range of secondary outcome measurements were taken and the general messages from these outcomes were consistent with those of the a priori stated primary outcome (AHI) and main secondary outcome (ESS score). Although these secondary outcomes were useful indicators of patient compliance and QoL and give a more complete picture of the effects of MADs, they should not be overinterpreted or be used in combination as an indicator of the strength of the effects of different MADs.

There were few SAEs during the study period and, out of the four SAEs reported by four patients, three were short-term, cardiac-related events. Two were considered possibly related to OSAH and one was considered possibly related to OSAH or MADs because the patient was on MAD treatment at the time. Almost all patients reported at least one minor AE, with mouth discomfort and excess salivation being the main problems.

The short treatment period meant little opportunity to observe an effect of MADs on RTAs or CVEs, which are the desired longer-term implications of control of EDS. However, patients did report improvements in sleepiness during driving and fewer interruptions to journeys during MAD use.

The trial-based cost-effectiveness analysis was also limited by the short treatment period, but the improvements in HRQoL for all MADs compared with no treatment meant that all were cost-effective at a WTP of £20,000 per QALY. The SP2 was the most cost-effective MAD up to a WTP per QALY of £39,800.

Thus, based on TOMADO, while all MADs have significant benefits and few harms compared with no treatment, the SP2 appears to achieve more benefits than the SP1 and almost all the benefits of the more sophisticated bMAD. However, it achieves these benefits at a lower cost than the bMAD and, so, it can be recommended on cost-effectiveness grounds.

Meta-analysis

Cochrane reviews from Lim et al. 51 and Giles et al.,33 and a meta-analysis from McDaid et al. ,8 were updated for the major outcomes that were included in TOMADO. The systematic review identified 12 studies including 629 patients comparing MADs with CM, 13 studies including 746 patients comparing MADs with CPAP and 52 studies including 5400 patients comparing CPAP with CM, all of which had an AHI or ESS score as one of the study end points. Study participants were predominantly middle-aged men who were overweight or obese. Trials including CPAP were generally conducted in patients with more severe OSAH according to AHI than trials of MADs with CM. CM included a range of treatments including sham devices, sham CPAP, placebo tablets, lifestyle advice and no treatment. Although we included only randomised trials, quality was variable, with many trials having fewer than 50 patients and treatment periods were generally short. Both parallel-group and crossover trials have been used.

Heterogeneity between studies, assessed by the I² statistic, was variable and often unreliable as a result of the small number of studies available. Some heterogeneity could be explained, particularly by baseline severity, but there remained unexplained heterogeneity. Although random-effects methods were used, the validity of combining trials in formal meta-analysis is questionable and cautious interpretation is required. Partly for this reason a network meta-analysis including all trials was not attempted.

Both MADs and CPAP resulted in significant improvements in AHI, with the greatest benefit evident in trials of CPAP against CM. However, the reduction in AHI was strongly related to baseline AHI, which is natural since a higher baseline allows greater scope for an absolute decrease. In head-to-head trials of MADs against CPAP, the performances of the two treatments were more similar and there were no head-to-head trials in patients with mild-range AHI.

Excessive daytime sleepiness assessed by the ESS is less variable than AHI so most trial populations were classed as having moderate baseline EDS. The differences between the effects of MADs and CPAP on subjective daytime sleepiness assessed by ESS were smaller and not significant in head-to-head comparisons. The estimated effects on EDS were strongly related to baseline severity and, to a lesser extent, baseline AHI. When trials of similar baseline characteristics were compared, there was little difference between the effects of MADs and CPAP on post-treatment ESS score when assessed against CM, and this is reinforced by the results from head-to-head trials. Treatment effects appeared to be stronger in trials with short duration of treatment, possibly reflecting a tailing-off of compliance over time. 45

The meta-analysis did not provide much insight into the effect of treatment on daytime BP above previous meta-analyses. There was a large amount of heterogeneity in the methodology used for assessing surrogates of cardiovascular outcomes. Our meta-analysis focused on daytime SBP and DBP because it was included as the primary marker of hypertension in TOMADO and because it is used in the Framingham equation that provides input into the long-term economic model. There was a small effect of both CPAP and MADs on SBP compared with CM, with little to choose between the two.

Few trials, apart from TOMADO, have contributed to the literature on HRQoL so that it was difficult to draw reliable conclusions. In common with TOMADO, there was evidence for a significant improvement in HRQoL as a result of these treatments in the meta-analysis, but the size of the effects is unlikely to be clinically important. The paucity of information did not allow more detailed analysis of published HRQoL. Given the demonstrated clinical effectiveness of both CPAP and MAD, further trial-based studies of HRQoL are unlikely to be conducted, but observational studies to supplement existing trial data would be useful.

Cost-effectiveness

In order to assess the effect of CPAP and MADs on long-term outcomes, including cardiovascular hazards and RTAs, we reviewed and updated an economic model provided by the University of York Centre for Health Economics, developed for McDaid et al. 8 The model inputs were adapted to better represent patients with mild to moderate OSAH and updated to reflect new research since the original model was developed. Systematic searches of published literature were undertaken to update model inputs related to CHD and stroke risk, RTA rates, HRQoL and costs. In addition, data from TOMADO were used for device-specific costs and to create a more precise mapping function between ESS score and utility measures (both EQ-5D-3L and SF-6D) that would also be more applicable to patients with mild to moderate OSAH.

In the base case, using the SP2 as the ‘standard’ device, MADs were found to be more costly and more effective than CM in patients with mild to moderate OSAH, with an estimated ICER of £6687 per QALY. Compared with MADs, CPAP was more costly and more effective, with an estimated ICER of £15,367. While it was clear that both of these treatments were better than CM, there was substantial uncertainty in the choice, with probabilities of being cost-effective at a WTP of £20,000 per QALY of 47% for MADs and 52% for CPAP. Corresponding figures at a WTP of £30,000 per QALY are 45% for MADs and 55% for CPAP.

The results were sensitive to a number of parameter inputs. If the average lifespan of the SP2 is increased from 12 months to 18 months, the ICER for CPAP compared with MADs becomes £44,066, which is more than traditionally accepted WTP thresholds. Additionally, choice of device has an important effect on the economic decision, with the ICER for the SP1 compared with CM being £1552, and for the bMAD compared with CM being £13,836. The ICER for CPAP compared with the SP1 is high at £89,182, but CPAP is both cheaper and more effective than the bMAD. Using device-specific inputs for treatment effects further confirms the superiority of the SP2 as the most cost-effective treatment for patients with mild to moderate OSAH, although substantial decision uncertainty remains. Differential compliance rate for CPAP also reduces its cost-effectiveness so that MADs become both less costly and more effective if compliance to CPAP is of the order of 90% of SP2.

Strengths

The TOMADO study was a relatively large and rigorously conducted RCT, with robust and precisely estimated treatment effects. To our knowledge, TOMADO is the first trial of MADs in mild to moderate OSAH with both clinical, patient-centred and cost-effectiveness outcomes. The interpretation of results is clear and consistent among different outcome measures. In contrast with most other published randomised trials, TOMADO included a detailed study of HRQoL. This showed consistency with clinical outcomes and highlighted the effects of MADs on activity, general productivity and symptoms. Although these effects might be described as modest, it is remarkable that they can be observed after a short period of treatment. TOMADO fed into updated meta-analyses that offered stronger insights into the relative effectiveness of MADs and CPAP in patients with OSAH. In addition, the effects of baseline severity have been highlighted and used to explain some of the differences in effects between MADs and CPAP. TOMADO also fed into an updated model of the long-term cost-effectiveness of MADs and CPAP devices which was adapted, for the first time, to mild to moderate OSAH.

The study was applicable to general sleep practice as it recruited participants who had been referred from primary care to the sleep clinic at Papworth Hospital. The SP1 and SP2 devices used in the study are available in many countries and are similar to other thermoplastic and semi-bespoke MADs on the market. Although the bMAD was fitted and manufactured by a hospital maxillofacial laboratory, it was done using skills, materials and facilities common to dental sleep services.

Limitations

Women, younger patients and patients with a BMI in the normal range were under-represented in the patients included in TOMADO and other trials so that results may not be generalisable to these populations.

In evaluating three non-adjustable MADs representing a range of sophistication and cost, TOMADO could not also include an adjustable MAD (aMAD). These are increasingly recommended,202,203 but are often more costly. They allow gradual titration of mandibular advancement according to tolerance and efficacy. This may give larger treatment effects by achieving ultimately greater jaw protrusion without lowering compliance, but whether or not aMADs are more effective than non-adjustable MADs remains unproven.

The aim for the bMAD was at least 50% maximal protrusion, but in practice this value was often lower; and similar to that achieved independently by patients with the other devices. This reflects the pragmatic nature of TOMADO, making its findings more applicable to the wider NHS. AHI effects have been shown to be proportional to mandibular protrusion. 204–206 Mean (SD) protrusion in this trial was between 52.5% (27.8%) of maximal advancement with the SP2 and 63.4% (22.6%) of maximal advancement with the SP1. These figures are lower than reported in other studies,69,74,76,83 many of which used an aMAD. 23,52,68,72,75,79,81,84 Nevertheless, although heterogeneity limits comparison, many of these trials did not report greater AHI effects than TOMADO. 23,74,76,83 Furthermore, TOMADO showed no association between protrusion and AHI effects, adding to existing evidence that greater protrusion may be no more effective in milder OSAH. For example, Tegelberg et al. 207 compared patients with mild to moderate OSAH who had devices at 50% and 75% maximal protrusion and found no difference in AHI effects. Quality studies comparing aMADs to non-adjustable devices are lacking. A small, non-randomised trial compared an aMAD with a thermoplastic MAD and found a modest difference in AHI favouring the adjustable device. 208 However, the sample size was small and the differing costs of the two devices (paid for by the patients) probably influenced device selection. A retrospective study of 805 patients demonstrated a small but statistically significant difference in AHI between an adjustable and non-adjustable device (7.6 vs. 10.0, respectively), but did not show a significant difference in ESS score or tolerability. 209 This study was also limited by device selection which was non-randomised. Other studies which have featured both adjustable and non-adjustable MADs have reduced the likelihood of finding real-life effect size differences by using similar or identical protrusions for both devices. Therefore, whether or not adjustability improves MAD effectiveness in OSAH remains uncertain and requires rigorous RCT evaluation.

In the meta-analysis and because of the economic decision analysis, all MADs were considered as a single treatment modality. There were too few studies to allow robust subgroup analyses and so we were unable to identify the more modest differences in effects between different MADs. It has been suggested that future meta-analyses distinguish between trials of non-adjustable MADs and those using aMAD. 209,210 From the 2006 Cochrane analysis, considering only trials comparing CPAP with aMADs moved the sleepiness (ESS score) effect size in favour of MADs, but not significantly. 33 We considered performing a similar subgroup analysis when updating the meta-analysis. However, device adjustability could not always be determined,80,82 and the potential advantage of titratable advancement was sometimes negated by the use of uniform aMAD protrusion. 83 Classifying trials as fixed MADs and aMADs, in order to perform separate meta-analyses, is not straightforward. For example, one trial with relatively weak treatment effects that has previously been excluded from aMAD reviews22 used two non-adjustable MADs, but the authors reported near-maximal (80%) jaw protrusion and performed ‘pseudotitration’ by adapting devices to optimise comfort and benefits. For these reasons we did not include a meta-analysis based on MAD adjustability.

Three separate meta-analyses were conducted comparing MADs with CM, MADs with CPAP and CPAP with CM. A more sophisticated analysis would have been to combine the studies into a network meta-analysis, thereby adding strength to all comparisons and better aligning the studies. However, these analyses rely on the associative law, which was unlikely to be true in this case, given the greater severity of OSAH in populations undergoing trials of CPAP. Furthermore, the heterogeneity observed between studies in Chapter 3 suggested that combining results within and across different treatments may not be sensible. The likely implication of doing separate meta-analyses is a loss of some precision in the results.

Conservative management encompassed a wide range of control treatments so that their influence on the trial-based treatment effects was difficult to estimate with any precision.

In our systematic review we used the Jadad score as a measure of study quality in order to be consistent with previous reviews. 8,33,51 This is a rather insensitive tool and did not provide substantial insight into the relative quality of different studies. It did, however, provide a broad structure for summarising design features reported in existing clinical trials.

The use of data in the model still reflects the lack of robust sources in some important areas. RTA risk after treatment with MADs is still inferred based on ESS score treatment effects compared with CPAP, rather than using direct data. While the link between OSAH, hypertension and cardiovascular outcomes may be increasingly understood, the treatment effects in the model are still based on short-term BP data rather than long-term CVD outcomes. Given the similarity in ESS score, treatment effect pooled from the meta-analysis, the importance of compliance and determining how prolonged the effects are, is clear but there remains a lack of good data to reflect this for MADs. Only crude sensitivity analyses were able to explore the effect this has on cost-effectiveness.

Implications for service

• CPAP remains the most clinically effective and cost-effective treatment for patients with moderate to severe OSAH based on reduction in AHI. For patients who are intolerant of CPAP, treatment with a MAD is also effective compared with no treatment.

• CPAP and MADs are equally effective treatment options for patients with mild to moderate OSAH and there is little to choose between them in terms of clinical effectiveness and cost-effectiveness, although this is contingent on similar compliance rates.

• Of the three MADs investigated, the semi-bespoke SP2 (or an equivalent MAD) is the most cost-effective treatment in the short term and should be used as the first-choice device, with the custom-made bMAD reserved for patients who have difficulties producing the SP2 mould, whose dental eligibility is more marginal or for whom other obstacles to using the SP2 may be overcome by dental intervention with a bMAD.

Implications for research priorities

• Head-to-head pragmatic clinical effectiveness and cost-effectiveness comparisons of adjustable and non-adjustable MADs, across the entire range of OSAH severity, are still required.

• Head-to-head comparisons of CPAP and MADs in milder OSAH, would reduce the uncertainty surrounding the current guideline stance that CPAP should be reserved as second-line treatment in this patient group.

• There is increasing evidence to suggest that the similar effects of CPAP and MADs on EDS may be as a result of differential adherence to treatment. However, there is limited information on this beyond short-term trials. Medium- to long-term compliance with MADs and CPAP should be monitored and reported. Such work would be strengthened by emerging tools to objectively monitor MAD compliance, which would benefit from further clinical and additional economic evaluation in their own right. Observational studies of HRQoL over time to supplement existing trial data would be useful to understand the treatment outcomes of greatest relevance to patients. In particular, it would be useful to know more about the durability of devices.

• Further data on longer-term risk of CVD and its risk factors would reduce model uncertainty and improve the precision of estimates of clinical effectiveness and cost-effectiveness.