Colistimethate sodium powder and tobramycin powder for inhalation for the treatment of chronic Pseudomonas aeruginosa lung infection in cystic fibrosis: systematic review and economic model

Brief statement of the health problem

Cystic fibrosis (CF) is an inherited disease that shortens life expectancy and greatly reduces the health-related quality of life (HRQoL) of patients. CF is characterised by abnormal ion movement across transporting epithelia. This leads to the production of thick sticky mucus in the lungs, pancreas, liver, intestine and reproductive tract, and an increase in the salt content in sweat. People with CF have problems with digestion, which can affect growth and body mass index (BMI), and are prone to lung infections by a range of pathogens, including Staphylococcus aureus, Pseudomonas aeruginosa and Burkholderia cepacia. This is thought to be because the thick mucus makes it difficult for the body to clear inhaled bacteria, and because people with CF have an increased airway inflammatory response to pathogens. 1 Although both digestive problems and lung infections contribute to morbidity and mortality, respiratory tract infections with P. aeruginosa have been shown to be a major risk factor contributing to mortality. 2

In the early stages of disease, management aims to identify and vigorously treat infection,3 and thereby limit structural changes that may predispose a patient to chronic infection with P. aeruginosa. If bacterial infection is not successfully prevented or treated, a chronic infection/colonisation can develop, whereby bacterial microenvironments, known as ‘biofilms’, form within the bronchial tree. Biofilms are difficult for immune cells and antibiotics to penetrate, and once established, are associated with clinical deterioration and ultimately increased mortality. 2 Treatment of chronic infection typically involves regular use of nebulised antibiotics, such as tobramycin [Bramitob^® (Chiesi) or TOBI^® (Novartis Pharmaceuticals)] and colistimethate sodium [Promixin^® (Profile Pharma) or Colistin^® (Forest Laboratories)], to suppress bacterial growth and prevent flare-ups (known as exacerbations), and to maintain lung function and quality of life. Treatment can be time-consuming for patients, with administration of nebulised antibiotics taking up to 1 hour per day during good health, and longer during periods of ill health. 1 Newer nebulisers, such as the eFlow^® rapid nebuliser (PARI Medical, West Byfleet, Surrey, UK) or the I-neb™ (Philips Respironics, Murrysville, PA, USA) adaptive aerosol delivery (AAD) system, may allow for more rapid administration of treatment. Pulmonary exacerbations may have a substantial negative impact on a patient’s quality of life4 and are usually treated using intravenous (i.v.) antibiotics, either in hospital or at home, or in a combination of these settings. 5

Aetiology and pathology

Cystic fibrosis is an autosomal recessive disorder, for which both copies of the gene that codes for a protein called the cystic fibrosis transmembrane conductance regulator (CFTR) contain a mutation. Over 1600 different mutations of the gene have been identified, causing different changes to the function of the protein, and hence different severities of disease in the individual. The most common mutation is the deletion of phenylalanine at codon 508. This deletion was present in an estimated 91.3% of the mutant alleles in the UK in 2010. 6

Cystic fibrosis transmembrane conductance regulator is a large (∼170-kDa) multidomain protein belonging to the adenosine triphosphate (ATP)-binding cassette family of membrane transporters. 7 It is located in the cell membrane of various cells in the body, including epithelial cells in the respiratory tract, pancreas, liver, intestine and reproductive tract, where it regulates fluid secretion. CFTR acts as an ion channel that utilises the energy released by the binding and hydrolysis of ATP to open. When open, chloride ions pass across the cell membrane by diffusion in the direction of their electrochemical gradient. 8 When functional, this promotes efflux of chloride ions from the cell into the extracellular fluid. Sodium ions and water follow by a paracellular route (between cells rather than through cells) and hence the volume of liquid on the epithelial surface is regulated. In CF, impaired CFTR function is most commonly thought to lead to a decrease in the surface liquid volume of epithelial cells (although there are theories that also consider reduced antibacterial properties of mucus and increased mucin secretion as putative mediators of the characteristics of CF). In the epithelia of the airways, these changes result in a decrease in mucociliary clearance in the respiratory tract, which is the body’s primary defence against invading pathogens. People with CF are more prone to respiratory infections as a result. In addition, people with CF have an excessive inflammatory response. The aetiology of this is unknown,9 but along with the damage caused by respiratory infections it leads to bronchiectasis and obstructive pulmonary disease, the primary causes of death among people with CF.

Expression of the CFTR gene in the body is widespread and symptoms are not confined to lung disease. Reproductive function in both males and females may be disrupted (although there is conflicting evidence in women). Exocrine tissues in the pancreas are also affected, where abnormal mucus can block and damage pancreatic ducts. This process starts in utero and causes a decrease in the secretion of digestive juices, which contain the enzymes, bicarbonate and water that are essential to digestion, which, in turn, leads to malabsorption of ingested food and malnutrition. Ultimately, damage to the pancreatic tissue can also lead to destruction of the pancreatic β cells in the islets of Langerhans. 10 These endocrine cells normally secrete insulin into the bloodstream, and their absence leads to diabetes mellitus. It is thought that this has a negative impact on lung disease, as lung function is affected by maintaining a normal body weight. This is also associated with a negative impact on survival. Insulin replacement therapy improves both lung function and body mass. 10

Children with CF are born without lung infection, but from the moment they are born they are exposed to pathogens and they become infected over time. Common infections include S. aureus, Haemophilus influenzae, P. aeruginosa and B. cepacia complex. P. aeruginosa is the most prevalent infection, with 37.5% of patients of all ages having a chronic infection in 2010. 6 Between the ages 20 and 49 years, between 55% and 65% patients have chronic P. aeruginosa infection. P. aeruginosa infection starts as an intermittent infection with non-mucoid variants of the bacterium. Studies suggest that this phenotype can be eradicated by antipseudomonal antibiotics,11,12 and current practice is to treat all incidents of infection energetically, with the aim of clearing the infection from the respiratory tract using oral or nebulised antibiotics (or both, depending on the UK centre). 3 However, over time, intermittent infections develop into colonisation. Chronic infection is associated with increased mortality and morbidity. 13 The environmental pressures imposed on the bacteria by the conditions within the CF lung are thought to drive the conversion of the non-mucoid phenotype to the mucoid phenotype, which secretes large quantities of alginate exopolysaccharide and forms biofilms. Biofilms are aggregates of cells set in an extracellular matrix composed largely of the alginate secreted by the mucoid phenotype. It is hypothesised that these slippery biofilms grow in microaerophilic or anaerobic environments created by the thick mucus that is characteristic of CF. Other factors present in CF lungs, such as actin, deoxyribonucleic acid (DNA) and decreased bacteriocidal secretions, are also thought to contribute to the formation of the biofilms. 14 The biofilms are very resistant to antibiotic treatment,15 and once a chronic mucoid infection has been established then eradication is not possible. Acquisition of this phenotype is again associated with the worsening of symptoms16 and a considerably worse prognosis. 17

Once bacterial colonisation is established, patients experience a gradual deterioration in lung function as lung tissue is damaged by the infection, which ultimately results in atelectasis (diminished lung volume), severe bronchiectasis, respiratory failure and death. 1 Patients experience increasingly frequent respiratory exacerbations, which severely affect quality of life and are usually treated with i.v. antibiotics and may require admission to hospital. Episodes of haemoptysis and pneumothorax may also occur. Historically, there have been differences in the diagnostic criteria for determining an exacerbation. These events have usually been characterised by an acute worsening of symptoms, such as increased cough, increased expectoration, decreased tolerance to physical activity, loss of weight or appetite and a deterioration in respiratory function. A marked increase in airway bacterial load [in colony-forming units (CFUs)/ml] has been cited as a criterion that may indicate an exacerbation,18 but is subject to some contention. Although forced expiratory volume in first second percentage predicted (FEV₁%) usually improves with treatment, Wagener et al. 19 demonstrated a progressive decrease in the best FEV₁% recorded in the 180 days after the exacerbation compared with the best FEV₁% recorded in the year prior to the exacerbation. The authors interpret this as being suggestive of an overall decline in FEV₁% associated with each exacerbation. In 2011 the EuroCare CF Working Group published a proposed definition for exacerbations. 20

Patients in the end stages of lung disease may be assessed for lung or lung and heart transplant, and may be added to the transplant waiting list. Owing to the systemic nature of the disease, transplants for other organs (e.g. liver, kidney) may also be necessary. In the UK, in 2010, 169 patients were evaluated and 82 accepted on to the transplant list. 6 Kidney transplants are sometimes needed as a consequence of the toxicity of the high-dose aminoglycoside antibiotics that are used to treat exacerbations. Once a lung transplant has taken place, the risk of death for people with CF is the same as the risk of death for all lung transplants. However, not all patients are fortunate enough to find an appropriate donor in time, and only 27 patients within the UK CF Registry eventually received a bilateral lung or heart and lung transplant in the UK in 2010. 6

Prognosis

The impact of CF on survival is substantial. In 2010, 103 deaths were recorded in UK patients with CF, details of whom were held within the CF Registry; the median age at death was 29 years (minimum = 0 years; maximum = 61 years). 6 Figure 1 shows the age distribution of deaths in patients with CF, based on 2010 data.

FIGURE 1.

Age distribution of deaths in patients with CF. 6

Although more people with the condition are living longer than in previous decades, only half of those patients living with CF are likely to live beyond their late thirties. Figure 2 shows recent estimates of survival for males and females with CF, based on a large UK-based cohort study. 22 Similar actuarial survival estimates are not currently available from the CF Registry.

FIGURE 2.

Survival of the UK CF population by sex, 2003. 22

It has been suggested that a number of other factors, such as genetics, medical treatment and environmental exposures, may interdependently influence prognosis, as illustrated in Figure 3.

FIGURE 3.

Multifactorial causes of variability in outcomes. 23

Epidemiology: incidence and prevalence

According to 2010 estimates from the Cystic Fibrosis Trust, over 9300 people in the UK have CF. Complete data on 7937 of these individuals are available from the CF Registry for 2010. The majority of CF cases are diagnosed by neonatal screening or during early infancy. Around 55.5% of those included in the registry are > 16 years of age and the incidence is spread evenly between males and females. For UK patients registered as having CF, approximately 82.2% are located in England, 3.9% in Wales, 4.7% in Scotland and 9.3% in Northern Ireland (Table 1).

According to the Cystic Fibrosis Trust, approximately five babies are born with CF each week. During the period 2007–10, between 235 and 301 new cases of CF were registered each year. Around 1 in 25 people are thought to be carriers of the CF gene, although this incidence of disease estimate is limited to white people living predominantly in Europe and America. Incidence in other races is lower but CF is increasingly being reported. 1 Figure 4 shows the age distribution of those patients for whom data are available within the 2010 CF Registry report.

FIGURE 4.

Age at diagnosis of CF based on 2010 estimates. 6

The prevalence of lung infection among the broader CF population is high. Around 37.5% of people living with CF are chronically infected with P. aeruginosa. Age-specific prevalence rates of Pseudomonas infection are shown in Figure 5. The prevalence of P. aeruginosa infection increases markedly with increasing age, up to around 25–30 years of age, with slightly lower rates in older age groups. These lower rates may be due to these patients having less severe mutations, which make them less likely to develop colonisation.

FIGURE 5.

Prevalence of CF lung infections by age group in 2010. 6 MRSA, methicillin-resistant Staphylococcus aureus.

Impact of health problem

Cystic fibrosis has a significant impact on the survival and quality of life of patients. The disease also impacts on carers and requires a considerable commitment of health-care resources. In 2003, an analysis of data from 196 adult patients with CF attending the Manchester Cystic Fibrosis Unit reported that 113 (57.6%) patients were attending work or study; however, 1799 days were lost as a result of sickness. 5 More recently, based on an analysis of complete data records from patients aged > 16 years, the Cystic Fibrosis Trust reported that 69.7% of patients are in work or studying. Although 18.5% were reported to be unemployed, only 5.6% of patients classed themselves as ‘disabled’. 6 Patients require monitoring and treatment by the NHS for the duration of their lives. Additionally, two young lives a week are lost to CF, which represents a significant impact on the families of CF sufferers. As the UK’s most common life-threatening inherited disease, CF continues to present a considerable cost burden for the NHS.

Measuring disease in cystic fibrosis

Cystic fibrosis can be broadly categorised into early stage, intermediate stage and end stage with complications. Patients with early-stage CF are characterised by the absence of infection with P. aeruginosa, or intermittent infection, which can usually be eradicated using antibiotics. Patients with intermediate-stage disease [forced expiratory volume in first second (FEV₁)] ∼30–70% predicted are chronically infected with P. aeruginosa or other less common organisms, whereas patients with end-stage disease (FEV₁ < 30% predicted) suffer from severe haemoptysis, pneumothorax and respiratory failure. 1 Patients have routine check-ups to monitor the status and stage of their disease. Measurements during these check-ups usually include assessment of bacterial infection and measurement of lung function, both of which contribute to treatment planning and prognosis. In some centres sputum tests are not performed routinely.

In the context of clinical trials, the Committee for Medicinal Products for Human Use (CHMP) research guidelines for the development of new medicinal products for CF18 recommend additional disease measures to gauge the efficacy of interventions. These include measuring rates of microbial resistance, the number of acute exacerbations and the patient’s HRQoL. FEV₁ is recommended as the primary end point for studies investigating CF treatments; however, a microbiological primary end point is also considered necessary for confirmatory trials. 18

Management of disease

The care of most patients in the UK is co-ordinated by a tertiary CF centre, with formal ‘shared care’ with local clinics. Further, primary care teams may also play a role in the surveillance and early treatment of infection, the provision of dietary and nutritional support, and the provision of social and psychological support for patients and their families. 1 A wide range of treatments may be required at various stages of the disease, including physiotherapy, pharmacological therapies, educational advice and surgical interventions for certain complications.

There are two main stages of P. aeruginosa lung infection, each of which requires a different approach to treatment. The first stage is characterised by intermittent growths of both mucoid and non-mucoid P. aeruginosa, and typically develops during infancy and childhood. This can be treated and sometimes eradicated with antibiotics to maintain respiratory function. Colonisation develops subsequently, and may be associated with mucoid change: it is a marker of reduced survival. Chronic infection cannot be eradicated by antibiotics as biofilm formation prevents antibiotics from working effectively. Acute exacerbations characterised by an acute decrease in respiratory function occur and become more frequent as the disease progresses. It is thought that acute exacerbations may contribute to a stepwise decrease in lung function, with FEV₁% failing to return to pre-exacerbation baseline values. However, evidence to support this theory remains limited. At this stage, for most patients, continuous antibiotic use will be required.

Current management pathways for patients with chronic Pseudomonas aeruginosa infection

Figure 6 presents a general management pathway for patients with CF with chronic P. aeruginosa lung infection. This is intended to be representative of the UK Cystic Fibrosis Trust guidelines,3 which, in turn, reflect usual clinical practice in the majority of UK CF centres. There is likely to be some variation in practice across some of the smaller centres, and specific antibiotic choices may differ by centre according local bacterial sensitivities. Generally speaking, decisions concerning the use of particular treatments tend to be more related to severity than age, therefore treatment use is broadly similar across both paediatric and adult populations. 6

FIGURE 6.

Treatment pathway for patients with chronic P. aeruginosa. PEP, positive expiratory pressure.

Current usage

Table 2 shows current registry estimates of antibiotic use among patients with chronic P. aeruginosa infection. The data suggest that approximately 78.8% of individuals with chronic P. aeruginosa infection receive at least one antibiotic post transplant. The CF Registry states that around 90% of patients with chronic P. aeruginosa should be prescribed one or more of these treatments. 6

Recent UK-relevant cost estimates relating to the treatment of CF are limited. A recent UK cost of illness study, undertaken in the east of England, estimated the mean annual cost of treating 174 patients to be £1,040,087 (£5976 per patient). 58 Multiplying this estimate up to the current number of patients in the CF registry yields a crude annual cost of around £57M for patients with CF in England and Wales. However, the true cost to the NHS may be considerably higher (Diana Bilton, Department of Respiratory Medicine, Royal Brompton Hospital, 2012, personal communication).

Variations in services and uncertainties about best practice

It has been noted elsewhere that many aspects of current practice in the management of CF have evolved without being subjected to high-quality clinical trials. 1 This may be partly a result of the rarity of the disease and associated difficulties with recruitment to clinical trials, as well as variations between patients in terms of how the disease manifests and is treated. With respect to interventions for the management of lung infection, evidence relating to the long-term clinical and mortality benefits of treatments is rarely available.

There is currently no NICE guidance relating to the detection, diagnosis or management of patients with CF. A single technology appraisal of mannitol dry powder for inhalation (DPI) for the treatment of CF was completed in 2012. This appraisal did not specifically relate to the management of patients with P. aeruginosa lung infection.

Since 1 April 2011, the Department of Health has adopted a ‘Payment by Results’ (PbR) tariff for patients with CF. This will link ‘activity’ to funding received, whereby money will follow the patient through their hospital journey, paying for treatment and care (excluding drugs) received along the way. 25

Summary of interventions and comparators

This assessment includes two interventions that are delivered as a DPI: colistimethate sodium DPI [Colobreathe^® (plus Turbospin^®), Forest Laboratories] and tobramycin DPI [TOBI^® (plus Podhaler^®), Novartis Pharmaceuticals]. The antibiotics colistimethate sodium and tobramycin also represent the relevant comparators for the assessment, albeit in nebulised form.

Colistimethate sodium (Colobreathe/Colomycin/Colistin) belongs to the polymyxin group and is a cyclic polypeptide antibiotic derived from Bacillus polymyxa, var. colistinus. Colistimethate sodium works by disrupting the structure of the bacterial cell membrane in a detergent-like way by changing its permeability, leading to bacterial death. It is also thought to act intracellularly to precipitate ribosomes and other cytoplasmic components. Colistimethate sodium is active against aerobic Gram-negative organisms, including P. aeruginosa, Acinetobacter baumanii and Klebsiella pneumoniae. Forest Laboratories currently markets colistin sulphate (Colomycin) as a tablet or syrup, and colistimethate sodium as a powder for injection or nebulisation. Profile Pharma currently markets colistimethate sodium (Promixin) as a powder for i.v. injection or for inhalation specifically using an I-neb device. Colobreathe (which is also colistimethate sodium) is available as 125-mg hard capsules and is administered specifically using the Turbospin inhaler device. It is anticipated that both the treatment and the Turbospin device will be marketed and packaged together.

The summary of product characteristics (SmPC) (www.ema.europa.eu/) lists the following AEs for colistimethate sodium DPI: unpleasant taste (dysgeusia), cough, throat irritation, dyspnoea, dysphonia, coughing, bronchospasm, balance disorder, headache, tinnitus, haemoptysis, asthma, wheezing, chest discomfort, lower respiratory tract infection, productive cough, crackles – lung, vomiting, nausea, arthralgia, pyrexia, asthenia, fatigue, decreased forced expiratory volume, drug hypersensitivity, weight fluctuation, decreased appetite, ear congestion, chest pain, exacerbated dysphonia, pharyngolaryngeal pain, epistaxis, sputum purulent, abnormal chest sound, increased upper airway secretion, diarrhoea, toothache, salivary hypersecretion, flatulence, proteinuria and thirst. Sore throat or mouth (probably due to Candida albicans infection or hypersensitivity) has been reported for nebulised colistimethate sodium and the SmPC states that this may occur with Colobreathe also. AEs listed in the electronic Medicines Compendium (eMC) (www.medicines.org.uk/) for the nebulised form also include bronchospasm, cough, hypersensitivity reactions and skin rash/rashes.

Tobramycin belongs to the aminoglycoside group of antibiotics and is obtained from cultures of Streptomyces tenebrarius. It enters susceptible bacterial cells via a complex active transport mechanism and acts by binding irreversibly to the 30S ribosomal subunit. It is thought that this interferes with essential steps in protein synthesis and consequently affects the permeability of the cell membrane, although there is some suggestion that it may also act directly on the cell membrane. 59 Once the cell envelope becomes compromised, cell death follows. It also acts to induce misreading of the genetic code of the messenger ribonucleic acid (mRNA) template, resulting in incorporation of incorrect amino acids, which can result in cellular malfunction. Two tobramycin nebuliser solution (TNS) products are available: Novartis Pharmaceuticals currently markets TOBI^® nebuliser solution, and Chiesi market Bramitob^® nebuliser solution. TOBI DPI is available as 28-mg capsules and is administered specifically using the Podhaler device. Both the treatment and device are marketed and packaged together.

Adverse events listed by eMC (www.medicines.org.uk/) for tobramycin DPI include hearing loss, tinnitus, haemoptysis, epistaxis, dyspnoea, dysphonia, productive cough, cough, wheezing, rales, chest discomfort, nasal congestion, bronchospasm, oropharyngeal pain, vomiting, diarrhoea, throat irritation, nausea, dysgeusia, rash, musculoskeletal chest pain and pyrexia. Cough was the most frequent adverse reaction. With respect to nebulised tobramycin, AEs reported in controlled clinical trials include dysphonia and tinnitus. AEs reported in the post-marketing phase include laryngitis, oral candidiasis, fungal infection, lymphadenopathy, hypersensitivity, anorexia, headache, dizziness, aphonia, somnolence, tinnitus, hearing loss, ear disorder, ear pain, dysphonia, dyspnoea, cough, pharyngitis, bronchospasm, chest discomfort, lung disorder, productive cough, haemoptysis, epistaxis, rhinitis, asthma, hyperventilation, hypoxia, sinusitis, dysgeusia, nausea, mouth ulceration, vomiting, diarrhoea, abdominal pain, rash, urticaria, pruritus, back pain, asthenia, pyrexia, chest pain, pain, malaise and pulmonary function test decreased.

Place in the treatment pathway

Both interventions are to be used for the ongoing treatment of chronic P. aeruginosa, as described above (see Current service provision). One of the principal anticipated benefits of the interventions is that they are quicker to use and are portable, which means that they can be self-administered by the patient as indicated, thereby avoiding time required for inhalation using a nebuliser. The DPIs may also result in savings in terms of the time associated with cleaning traditional nebulisers. It is hypothesised that these benefits may lead to improvements in compliance with treatment.

Identification of important subgroups

Specific subgroups have not been identified a priori within this appraisal. Consideration was given within this assessment to evidence relating to those groups of individuals for whom these therapies may be particularly clinically effective or cost-effective.

Current usage in the NHS

The use of TOBI in conjunction with the Podhaler device was granted full marketing authorisation by the EMA in 2011. TOBI Podhaler is indicated for the suppressive therapy of chronic pulmonary infection due to P. aeruginosa in adults and children with CF, aged ≥ 6 years. The Podhaler inhaler device bears an initial date of Conformité Europeénne (CE) marking of 28 July 2005. The Novartis Pharmaceuticals submission states that this date is noted on the European Commission (EC) Declaration of Conformity to the European Union (EU) Medical Device Directive 93/42/EEC as amended for this Class I device. 60

Colobreathe used in conjunction with the Turbospin device was granted full marketing authorisation by the EMA in February 2012. Colobreathe is indicated for the management of chronic pulmonary infections due to P. aeruginosa in patients with CF aged ≥ 6 years.

Anticipated costs associated with the intervention

Table 3 summarises the acquisition costs associated with the interventions and comparators, based on list prices from the British National Formulary (BNF). 61

Interventions

Two interventions are included in this assessment:

1. colistimethate sodium DPI, used in conjunction with the Turbospin device

2. tobramycin DPI, used in conjunction with the TOBI Podhaler device.

Populations and subgroups

The population for the assessment includes people aged ≥ 6 years with CF and chronic P. aeruginosa pulmonary colonisation. Subgroups are considered according to the available evidence.

Relevant comparators

The interventions are compared against each other. Other relevant comparators include antibiotics used for nebulised inhalation, including colistimethate sodium for nebulised inhalation and tobramycin for nebulised inhalation. The availability of evidence of the effectiveness of other less commonly used nebulised antibiotics (e.g. aztreonam) with antipseudomonal activity is also considered within the assessment.

Outcomes

The following outcomes are considered within this assessment:

• rate and extent of microbial response (e.g. sputum density of P. aeruginosa)

• lung function measured in terms of FEV₁%

• respiratory symptoms

• frequency and severity of acute exacerbations

• HRQoL

• AEs of treatment (including rate of resistance to antibiotic treatment)

• cost-effectiveness measured in terms of the incremental cost per quality-adjusted life-year (QALY) gained.

Identification of studies

A comprehensive search was undertaken to systematically identify literature relating to the clinical effectiveness of colistimethate sodium DPI and tobramycin DPI for the treatment of P. aeruginosa in CF. The search strategy comprised the following main elements:

• searching of electronic databases

• contact with experts in the field

• hand-searching of bibliographies of retrieved papers.

The following electronic databases were searched from inception for published trials and systematic reviews:

• MEDLINE: Ovid. 1950 to present

• MEDLINE in-Process & Other Non-Indexed Citations: Ovid. 1950 to present

• EMBASE: Ovid. 1980 to present

• The Cochrane Library: Wiley Online Library

  • Cochrane Database of Systematic Reviews (CDSR), 1996 to present

  • Database of Abstracts of Reviews of Effects (DARE),1995 to present

  • Cochrane Central Register of Controlled Trials (CCRT), 1995 to present

  • Cochrane Methodology Register, 1904 to present

  • Health Technology Assessment (HTA) database , 1995 to present

  • NHS Economic Evaluation Database (NHS EED), 1995 to present

• Cumulative Index to Nursing and Allied Health Literature (CINAHL): EBSCOhost, 1982 to present

• Web of Science Citation Index: Web of Knowledge, 1899 to present

• Conference Proceedings Citation Index (CPCI): Web of Knowledge, 1990 to present

• Bioscience Information Service (BIOSIS) Previews: Web of Knowledge, 1969 to present.

Additional searches were carried out for unpublished studies (e.g. ongoing, completed):

• Agency for Healthcare Research and Quality (AHRQ)

• Bandolier

• Centre for Health Economics (CHE); University of York

• ClinicalTrials.gov

• Current Controlled Trials

• The National Research Register Archive: NIHR, 2000–7

• The metaRegister of Controlled Trials: Springer Science + Business Media, 2000 to present.

Manufacturers’ submissions received by NICE, as well as any relevant systematic reviews were also hand-searched in order to identify any further clinical trials.

The MEDLINE search strategy is presented in Appendix 2. The search strategy combined free-text and medical subject heading (MeSH) or thesaurus terms relating to CF with free-text and MeSH or thesaurus terms relating to P. aeruginosa, relevant antibiotics and classes of antibiotics, and the devices and comparator devices of interest. The search strategy was translated across all databases. No date or language restrictions were applied. Literature searches were conducted during February and March 2011. References were collected in a bibliographic management database and duplicates removed.

Inclusion and exclusion criteria

Inclusion and exclusion criteria were based on the scope provided by NICE. 62 These are set out below.

Data extraction and critical appraisal strategy

Data were extracted without blinding either to authors or journal. Data were extracted by one reviewer using a standardised form and checked by a second reviewer. Where multiple publications of the same study were identified, quality assessment and data extraction were based on all relevant publications, and listed as a single study. The quality of included studies was assessed according to three sets of criteria. The purpose of quality assessment was to provide a narrative account of trial quality for the reader, and to inform subgroup analyses (where data allow). In order to assess the risk of bias, items listed in the NHS Centre for Reviews and Dissemination (CRD) report63 were used and were scored as ‘yes’, ‘no’ or ‘unclear’. To assess the clinical relevance and quality of the studies, items were generated from the EMA research recommendations. 18 Two trials were non-inferiority trials; a separate quality assessment form64 specific to this type of study was also used.

Data synthesis methods

The prespecified outcomes were tabulated and discussed within a descriptive synthesis. Where populations, interventions, outcome measures and available data were comparable and statistical synthesis was considered appropriate, classical meta-analysis or NMA was planned using Bayesian techniques, or Review Manager (RevMan) software version 5 (The Cochrane Collaboration, The Nordic Cochrane Centre, Copenhagen, Denmark; http://ims.cochrane.org/revman). If sufficient trials were available, sensitivity analysis was planned to examine whether the removal of poor-quality trials (in terms of risk of bias or compliance with research guidelines) influenced the results of the meta-analysis. Consideration was also given to subgroup analyses based on study characteristics.

Quantity and quality of research available

The search retrieved 743 potentially relevant citations (734 from searches of electronic databases, nine from secondary searches of relevant reviews, articles and sponsors submissions). Of these, 723 were excluded at the title and abstract stage, leaving 20 potentially includable citations.

The full texts of the 20 articles were obtained for scrutiny. Fifteen did not meet the inclusion criteria and were excluded (see Appendix 3). Three studies60,65,66 comparing colistimethate sodium DPI or tobramycin DPI with a nebulised antibiotic were included in the review. One study was of tobramycin DPI in combination with the TOBI Podhaler,60,65 and two studies were of colistimethate sodium DPI in combination with the Turbospin device. 66 Information about the three trials included in the systematic review was available from five sources,60,65–68 as indicated in Figure 7. These comprise one published journal article,65 two conference abstracts67,68 and the two manufacturers’ submissions to NICE,60,66 with subsequent clarifications. It should be noted that data for the pivotal colistimethate sodium DPI trial, the COLO/DPI/02/06 trial,66 were available from the manufacturer’s submission,66 the clinical study report (CSR),69 the trial protocol70 and personal communication/clarifications only. None of this information was available in the public domain. The search process is summarised using a PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) diagram in Figure 7.

To assess the viability of a NMA, key study characteristic data were extracted from an additional 13 studies, from 16 citations. 65–67,71–83 Owing to clinical heterogeneity between the studies and incompleteness of the evidence network, a NMA was not performed (see Appendix 4).

Study characteristics

The included trials and the treatments assessed are summarised in Table 4. All studies were open-label, multicentre studies, two of which were multinational studies. 66 The EAGER (Establish A new Gold standard Efficacy and safety with tobramycin in cystic fibrosis) trial65 was a large trial (n = 533) that compared tobramycin DPI with nebulised tobramycin. The COLO/DPI/02/06 trial66 was slightly smaller (n = 380) and compared colistimethate sodium DPI with nebulised tobramycin. Both of these trials were powered to detect clinically relevant changes in FEV₁%. The COLO/DPI/02/05 trial66 was much smaller (n = 16) and compared colistimethate sodium DPI with nebulised colistimethate sodium. The EAGER65 and COLO/DPI/02/0666 trials were both of 24 weeks’ duration, whereas COLO/DPI/02/0566 was a crossover trial, which reported outcome data at 4 weeks (before crossover) and 8 weeks (after crossover) only.

FIGURE 7.

Study inclusion (adapted from PRISMA).

Interventions and comparators

Intervention and comparator dosing complied with current UK licensing (www.medicines.org.uk/EMC/medicine/). In the two trials of colistimethate sodium DPI (COLO/DPI/02/0666 and COLO/DPI/02/0566), patients took colistimethate sodium DPI treatment every day throughout the study. In both the EAGER trial65 and the COLO/DPI/02/06 trial,66 the dosing pattern for nebulised tobramycin was cycles of 28 days on treatment, followed by 28 days off treatment, for three cycles. Within the EAGER trial,65 the same treatment approach was used for tobramycin DPI. This administration cycle is standard practice for tobramycin,61 with the aim of preventing antibiotic resistance.

Inclusion and exclusion criteria

Inclusion and exclusion criteria are summarised and compared in Table 5. Criteria seem largely compatible between the two major trials (EAGER65 and COLO/DPI/02/06),66 although criteria for COLO/DPI/02/0666 were complicated. Inclusion and exclusion criteria are not reported in full here.

The EAGER65 trial and COLO/DPI/02/06 trial66 selected patients with ‘confirmed’ or ‘documented’ CF, who were clinically stable, and aged ≥ 6 years, whereas the COLO/DPI/02/05 trial66 selected patients who were ≥ 8 years. Patients in the EAGER65 trial and COLO/DPI/02/06 trial66 had to have an FEV₁% value of ≥ 25%, up to 75%, whereas in the COLO/DPI/02/05 trial66,67 no upper limit for FEV₁% was set. Patients in all trials continued with usual CF treatments (except other routine antipseudomonal treatments). Patients in all three trials had a chronic P. aeruginosa infection. The criteria used to define a chronic infection did not meet with EMA recommendations18 in any trial, as all called for only two positive cultures in the last 6 months, rather than three. In the case of the COLO/DPI/02/06 trial,66 three positive cultures were required in the last 6 months, but patients could also qualify with only two in the last 2 months. As such, it is unclear whether or not the trials have truly selected chronically infected patients, and how comparable the degree of infection is between the two trials.

The COLO/DPI/02/06 trial66 had a run-in period whereby participants were required to have received 16 weeks (two cycles) of nebulised tobramycin prior to beginning the trial. Tobramycin has been documented to peak rapidly in efficacy in the first cycle of treatment with the effect not being sustained over time. 76 Therefore, the run-in phase was intended to eliminate this short-term change in FEV₁% predicted. In addition, this run-in phase was intended to exclude any patients who could not tolerate tobramycin. In comparison, the EAGER trial65 had a washout period of any systemic or inhaled antipseudomonal antibiotics for 28 days prior to randomisation, which ensured that patients already on tobramycin complied with the standard dosing schedule of 28 days on treatment followed by 28 days off treatment. The difference between these two criteria may result in slightly different populations.

Patient characteristics

The baseline characteristics of patients in the three trials are presented in Table 6. The patients in the COLO/DPI/02/06 trial66 had a lower mean age than those in the EAGER trial. 65 Mean age was not reported for trial COLO/DPI/02/05. 66 As age and FEV₁% status are thought to have an inverse correlation, it might be expected that the patients in the COLO/DPI/02/06 trial66 were earlier in their stage of chronic Pseudomonas infection than those in the EAGER trial. 65 However, the baseline FEV₁% predicted values are similar between these two trials, with the FEV₁% predicted in COLO/DPI/02/0666 being slightly lower. This may be due to inclusion criteria for chronic infection not being defined according to EMA recommendations. 18 For all trials, this may result in the recruitment of patients with intermittent infections, who may respond differently to treatment than chronically infected patients. It is also probable that some patients recruited to the COLO/DPI/02/06 trial66 may be slightly less well than the EAGER trial65 participants, as criteria were more stringent in this population. In both trials, the lack of consistency and conformity with the EMA guidelines18 may affect generalisability, with the trial populations not being entirely made up of the chronically infected patient population as defined by the EMA and European and French consensus conference. 27

In line with the potentially slightly poorer health (based on FEV₁% values) of the COLO/DPI/02/0666 trial patients, the BMI was also, on average, lower than in the EAGER trial. 65 However, it should be noted that these differences have not been subjected to statistical scrutiny and may not be significant. The clinical relevance of differences of this size are also uncertain.

Concomitant medication use could not be compared between trials as few data were provided (after a request for clarification from the Assessment Group) for the EAGER trial. 65 Many allowed medications (e.g. macrolides and bronchodilators) that could affect FEV₁% measurements, and their impact on the trial results are unknown, and may be different between studies.

In terms of prior antipseudomonal use, patients in the COLO/DPI/02/06 trial66 had all used nebulised tobramycin immediately before the trial, whereas only around 25% of patients in the EAGER trial65 had used nebulised tobramycin immediately before the trial [with an additional 55% (approximately) having used it within 3 months prior to the trial]. As such, patients in the COLO/DPI/02/0666 trial may have been more tolerant of tobramycin in terms of AEs, and will have experienced the initial peak in tobramycin FEV₁% results within the first 4 weeks of the run-in period, rather than during the trial itself. Conversely, the EAGER trial65 had a proportion of patients who were not tobramycin tolerant having never used tobramycin, and a proportion who had not used tobramycin immediately prior to the 28-day washout period. Some or all of these patients may have experienced an initial peak in efficacy (see Table 6) during the trial, and may be more likely to experience AEs associated with tobramycin than patients in the COLO/DPI/02/0666 trial.

Given that age, BMI, concomitant medications, prior exposure to antipseudomonal antibiotics and FEV₁% all have prognostic value in CF, it is difficult to determine whether or not these cohorts are comparable in terms of overall health and propensity to benefit from antipseudomonal treatments.

Study withdrawals

Table 7 shows the number of participants in each arm of each trial and the numbers of participants who withdrew throughout the study. Both trials saw a relatively high dropout rate, and this was higher in the intervention arm of both major trials. 65,66 In both key trials, there appear to be data missing and unaccounted for in some analyses, with more patients missing than are listed as withdrawals in the intention-to-treat (ITT) analyses for EAGER trial65 [commercial-in-confidence (CiC) information has been removed] (see Table 7). The implications of these missing data points are unknown.

Table 8 describes the reasons for withdrawals. In both trials, more patients withdrew owing to AEs than for any other single reason, with withdrawal of consent/patient request the second most common reason. In the EAGER trial,65 AEs accounted for proportionately more withdrawals in the tobramycin DPI arm than in the nebulised tobramycin arm. Similarly, more patients withdrew consent for the trial in the DPI arm. In the COLO/DPI/02/06 trial,66 the same pattern was seen, with more patients withdrawing from the colistimethate sodium DPI arm than from the nebulised tobramycin arm owing to AEs, although withdrawals owing to patient request were lower in the DPI arm. In this trial, the difference between arms appears larger than in the EAGER trial,65 although the absolute number of withdrawals is smaller in COLO/DPI/02/06. 66 Differences between the two trials in dropout numbers may be attributable to differences between patients’ tolerance to nebulised tobramycin at baseline; patients who tolerated nebulised tobramycin poorly were likely to have been excluded before randomisation in COLO/DPI/02/06. 66 The Forest Laboratories submission to NICE66 reports 16 screening failures but it is unclear if these patients failed during the run-in period because of lack of tolerance for tobramycin. However, if this was the case, it could account for at least some of the difference in withdrawals between arms, and between the two main studies.

Study end points and outcomes

The outcomes reported across the three studies are documented in Table 9,65,66 alongside the outcomes listed in the NICE scope, and the outcomes recommended by the EMA research guidelines. 18

All outcomes requested by NICE were reported in the two major trials; however, where a study reports that an outcome is measured, this does not necessarily indicate that the study was sufficiently powered to detect a clinically meaningful effect or that the outcome was assessed and reported according to EMA guidelines. 18 The COLO/DPI/02/05 trial66 was a Phase II safety trial and therefore the outcomes are more limited in this 8-week trial than the other two larger trials.

The primary outcome for efficacy trials recommended by the EMA is change in FEV₁%. 18 This outcome is reported by all three trials, and although it was not always the primary outcome of the trial, both major trials60,65,66 were powered to detect clinically relevant changes in FEV₁%. The COLO/DPI/02/0666 trial followed the American Thoracic Society (ATS) guidelines. The methods by which FEV₁% measurement were made (Table 10) were not clear within either the EAGER trial65 or trial COLO/DPI/02/05,66 which may allow a margin for imprecision and/or inaccuracy in the data, and is a potential source of bias in an open-label trial.

The EMA recommends that data on exacerbations, i.v. treatment and hospitalisations (as listed in Table 9) should be reported alongside FEV₁% to establish clinical benefit to the patients in terms of harder, more clinically relevant outcomes. The EAGER trial65 did not define an acute exacerbation, and only provided data on a poorly defined AE termed ‘lung disorder’; the European Public Assessment Report (EPAR)84 for DPI states ‘Exacerbations and hospitalizations related to respiratory events were collected to support the data for the relative change from baseline in per cent predicted’, although these data were not provided by the manufacturer on request by the Evidence Review Group (ERG). The data stated only how many patients had at least one event, rather than the overall incidence of events (patients could have multiple events within the timescale of the trial). Incidence data were not provided on request from the Assessment Group. The COLO/DPI/02/0666 trial fully defined an exacerbation, and provided data on the time to the first event, and data on incidence on request from the Assessment Group. Additional antibiotic treatments for exacerbation did not have to be i.v. treatments in either trial reporting this outcome. 65,66

The EMA recommends a microbiological secondary outcome for all trials of bronchopulmonary infection with a clinical primary outcome (e.g. FEV₁%),18 and these should include both measures of colony density (e.g. sputum density) and measures of resistance (e.g. MIC values). The two major trials both report measures of resistance but only the EAGER trial65 reports sputum density. Both trials report MIC₅₀ for tobramycin (although it is assumed that the MIC values are MIC₅₀ and not MIC₉₀ in the EAGER trial,65 based on the quoted breakpoints matching MIC₅₀ breakpoints), and COLO/DPI/02/0666 reports MIC₅₀ for colistimethate sodium as well. Both trials provided these data at the old BSAC breakpoint of 8 mg/l for resistance but only COLO/DPI/02/0666 reported this outcome at the new breakpoint issued by BSAC of 4 mg/l. 85 As noted in Chapter 1 (see Measuring disease in cystic fibrosis), the relevance of MIC susceptibility breakpoints to inhaled antibiotics is debated owing to higher than usual concentrations reaching the lungs, and because colonisation usually comprises multiple phenotypes of P. aeruginosa, which have different sensitivities to antibiotics and have different sensitivity when grown in culture. However, the monitoring of breakpoints has relevance in indicating whether or not isolates are becoming more resistant as a population, rather than to indicate whether or not the treatment is likely to be effective at the concentrations delivered. Although the clinical and long-term relevance of increases in MIC₅₀ remain unclear, and as the EMA guidelines recommend their use, the MIC₅₀ concentrations are presented in this report for consideration.

The EMA recommend that BMI is recorded only in studies at least 6 months in duration. A low emphasis is placed on this outcome in both major trials, and this outcome is not listed in the NICE scope. Quality of life was not recorded in the EAGER trial65 and was recorded using a non-preference-based instrument in COLO/DPI/02/06. 66 All trials aimed to measure AEs but it was not clear how this was achieved in either of the major trials (see Table 10).

Quality assessment

The quality assessment of the three included studies is presented in Table 11. None of the studies performed consistently well for all quality assessment items. Internal validity items (as scored according to the CRD criteria)63 were addressed reasonably well, with the exception of blinding (owing to the open-label trial design). External validity items (scored according to the EMA research guidelines)18 were less well addressed, with omissions in a number of key recommendations. The two non-inferiority trials (EAGER65 and COLO/DPI/02/0666) did not perform well when analysed using the CONSORT (Consolidated Standards of Reporting Trials: www.consort-statement.org) checklist for non-inferiority and equivalence trials as a guide.

Lung function

The most commonly reported measure of lung function across the included studies was FEV₁%. Data were sought at 4, 20 and 24 weeks where available. Tobramycin was administered 28 days on and 28 days off, which results in a peak and trough in FEV₁% values. 65 This has the potential to bias results, and it would seem appropriate to consider results at both the peak and trough of the efficacy cycle. As such, both 20- and 24-week data are presented within this review.

The presentation of data varied across the studies (see data extraction tables in Appendix 4).

• The COLO/DPI/02/0666 reported several analyses for FEV₁% data. These included:

  • ITT population with last observation carried forward (LOCF) imputation

  • ITT population with no imputation (completers)

  • PP population with LOCF imputation, and

  • PP population with no imputation (completers).

Tests specified a priori showed that the data were non-normal in distribution; an additional non-parametric analysis and an analysis using logarithmic transformed data were also performed by the manufacturer to correct for this. 66 As such, 12 analyses were presented for these data. Analysis of covariance (ANCOVA) comparative data were reported, with adjustment for baseline FEV₁% and pooled centre.

The EAGER trial65 data were not transformed, nor was a non-parametric test performed, although no test of normality was apparently planned or performed either. No imputation was performed on the Novartis Pharmaceuticals data, and only limited data were presented at 24 weeks. Some adjusted comparative data were presented, with adjustments for main effects treatment, baseline FEV₁% predicted and pooled centre.

Where data were not available in the manufacturers’ submissions to NICE or within journal publications, the Assessment Group requested or calculated missing values. However, some values remained missing or unclear. Given the available evidence, a NMA was not possible (see Appendix 4) and, as such, a narrative synthesis is presented for the results.

Although all trials reported ITT analyses, the EAGER trial65 did not perform any imputation. In the COLO/DPI/02/06 trial,66 an analysis of completers is presented (where only those for whom there are data at both baseline and the time point of analysis are analysed) and an analysis with LOCF is presented (which should include all patients at every time point but appears to vary from time point to time point – see Table 12). The differences in exclusion of data in the ‘no-imputation’, ‘LOCF’ and ‘completers’ analyses are likely to affect results but it is unclear in which direction. The most usual direction of effect of attrition is to overestimate efficacy. 86–88 Attrition is most problematic in the EAGER trial65 ‘no imputation’ analysis, as demonstrated by the n numbers in Table 12. As such, results from the trials are not directly comparable. However, to allow some form of simple comparison to be made, data from the ‘no-imputation’ (EAGER trial)60,65 and ‘completers’ (COLO/DPI/06)66 analyses have been collated and synthesised in parts of this section. Note that the data used for COLO/DPI/0666 are from the original analysis, not the transformed or non-parametric analysis. PP analyses are discussed where appropriate. Further results are presented in the data extraction tables in Appendix 5.

Microbiological outcomes (colony density and resistance)

The EMA recommends that microbiological data should support FEV₁% efficacy data. 18 The COLO/DPI/02/0566 trial did not report any microbiological data, whereas COLO/DPI/02/0666 reported resistance data (Table 14) and EAGER65 reported both resistance data and sputum density data (Tables 14 and 15).

As noted in Chapter 1 (see Measuring disease in cystic fibrosis), the BSAC breakpoints for resistance have changed since the trials were performed. COLO/DPI/02/0666 reported both the new (4 mg/l) and old (8 mg/l) breakpoints for resistance, whereas only data for the old breakpoint was available for the EAGER trial. In the COLO/DPI/02/06 trial,66 resistance (4 or 8 mg/l breakpoints) to colistimethate sodium remained very low (≤ 1.1%) throughout the 24-week trial in the colistimethate sodium DPI arm, whereas resistance to tobramycin was reported to not change substantially during the study, with values (CiC information has been removed). In the EAGER trial,65 resistance (8 mg/l breakpoint) to tobramycin started at around 20% (the baseline value was at the end of 28 days off treatment) and was lower at 24 weeks (also at the end of 28 days off treatment). Again, it is unclear if there was a trend of change in resistance over time or merely fluctuations around the mean. The high levels of resistance at baseline (CiC information has been removed) is not surprising as 100% of patients in the COLO/DPI/02/0666 trial and more than 90% in the EAGER had received tobramycin before the trial, and are likely to have already developed some level of resistance. As already discussed elsewhere, the significance of increasing resistance to tobramycin is unclear.

Table 15 shows the results of the sputum density tests in the EAGER trial. 65 These data were not recorded in the COLO/DPI/02/06 trial. 66 Mean change from baseline log₁₀ values show numerically greater reductions in sputum density are achieved with tobramycin DPI at 20 weeks than with nebulised tobramycin. Results at 24 weeks are not reported. These values are in accordance with the slightly greater increase in FEV₁% seen for tobramycin DPI but their statistical and clinical significance is not known.

Exacerbations

Data on protocol-defined acute exacerbations were not reported in a consistent way across the three included trials. COLO/DPI/02/0666 reported time to event data for acute exacerbations. Despite the EMA research guidelines requesting this outcome, Novartis Pharmaceuticals did not provide any acute exacerbation data, although among the AEs reported by Konstan et al. 65 lung disorder is defined as ‘generally reported by the investigator as pulmonary or CF exacerbation’. This is regarded as a proxy for acute exacerbations for the purpose of this assessment, although is clearly not an entirely specific measure and it is unclear what other events may have also been included in this outcome. The EAGER trial65 data were provided as the number of patients experiencing the event, rather than the number of events. Equally, for the time to event data provided by Forest Laboratories for COLO/DPI/02/0666 to be a useful outcome, it would have to be assumed that the time to the first event is directly related to the overall incidence of events. As one patient could theoretically experience more than one exacerbation through the duration of the trial (24 weeks), the Assessment Group requested data relating to the number of events from both manufacturers. Only Forest Laboratories complied with the request; however, the correct interpretation of these data was unclear. Neither manufacturer was able to provide data on the severity of exacerbations. Data relating to this outcome are presented in Table 16.

Numerically more patients on colistimethate sodium DPI experienced protocol-defined acute exacerbations compared with nebulised tobramycin in COLO/DPI/02/0666 (31.1% vs. 26.1%, respectively), although it is unclear if the same trend would be observed for data relating to the number of events. Conversely, the mean duration of use of antibiotics to treat the exacerbation was slightly less at 13.6 days in the colistimethate sodium DPI arm compared with 14.4 days in the nebulised tobramycin arm. Colistimethate sodium DPI had a more favourable exacerbation rate to nebulised tobramycin when non-protocol defined exacerbations were also considered (38% vs. 39%). The number of patients requiring antipseudomonal treatments was similar in both arms. Within the EAGER trial,65 the number of patients experiencing lung disorders was greater in the tobramycin DPI arm (33.8%) than the nebulised tobramycin arm (30.1%). It is unclear if the same trend would be observed for data relating to the number of events. Mean duration of antipseudomonal antibiotic treatment was also slightly shorter in the tobramycin DPI arm than in the nebulised tobramycin arm (30.9 days vs. 33.4 days). The number of patients receiving additional antipseudomonal treatments was higher in the tobramycin DPI arm.

In both trials there is disparity between the number of exacerbations, the number of days on treatment and the number of treatments given (see Table 16). The clinical advisors to the Assessment Group were unsure why this might be. In both trials the number of patients receiving treatment far exceeds the number of exacerbations. One potential explanation is that treatments are given as soon as an exacerbation is suspected, and stopped when tests do not confirm it. As the trial is open label, performance bias (e.g. being more likely to treat a patient’s symptoms as an exacerbation if they are in the DPI arm) and outcome assessment bias (the criteria for an acute exacerbation were subjective to some degree in the COLO/DPI/02/06 trial,66 and to an unknown degree in the EAGER trial65) could affect results. These types of bias could work to increase or decrease the number of exacerbations and number of patients receiving additional antibiotics, and it is unclear to what extent, and in what direction, they may have affected the outcomes in question.

In the case of the outcome ‘mean duration of antibiotic use’, the direction of effect is opposite to the direction of effect indicated by the number of exacerbations in both trials. As this is a mean value, it does not necessarily indicate that fewer days were spent overall on antibiotics or that DPI treatments reduce duration for which antibiotics are needed for any given exacerbation. This may indicate that the exacerbations were, on average, less severe in the DPI groups, or that exacerbations were not confirmed and treatment terminated more often in the DPI arms. Equally, it may indicate that less severe exacerbations were prevented less often, thus bringing down the mean duration of antibiotic administration. Finally, these small differences may reflect variations in clinical practice rather than demonstrating a real difference in antibiotic use between the two groups.

The influence of bias and the lack of consistency in the direction of effect makes the outcomes relating to acute exacerbations difficult to interpret with any certainty.

It is not possible to draw a comparative conclusion as to the relative efficacy between trials in terms of exacerbations, given the difference in the way they have been reported, and the uncertainty about the comparability of the patient data populations and characteristics.

Body mass index

Table 17 shows the BMI measurements from the EAGER65 and COLO/DPI/02/06 trials. 66 The EAGER trial65 states that BMI is an outcome under investigation; however, the BMI data are not provided for any of the time points after baseline. The COLO/DPI/02/06 trial66 reports BMI at every time point. The data presented in Table 17 are for the ITT population and demonstrate very little change in BMI from baseline. BMI data for the COLO/DPI/02/0566 trial are not presented as BMI was not an outcome under investigation in that trial. It is unlikely that changes in BMI would be seen before 6 months, so the lack of change is unsurprising.

Health-related quality of life

Table 18 presents the Cystic Fibrosis Questionnaire-Revised (CFQ-R) results for the HRQoL outcome. The two trials that investigated this outcome were COLO/DPI/02/0666 and COLO/DPI/02/05. 66 For the COLO/DPI/02/0666 trial the data presented were the adjusted means in the numerous domains of the CFQ-R from baseline to week 24. Most of the scores tended to be in favour of the dry powder intervention, although none of the differences were statistically significant. It is interesting to note that one of the few negative results is for the respiratory domain, although this does not reach significance. Although the COLO/DPI/02/0566 trial did also use the CFQ-R throughout the trial, the data were not provided. (CiC information has been removed.) As already noted, this quality-of-life measure has not been validated and is not preference based in line with the NICE reference case. It is therefore difficult to interpret the results in terms of impact on HRQoL and relative weight of the individual items in the measure.

One key argument put forward in favour of the DPI formulations is its ease of use. This may have many benefits, including increased patient satisfaction with treatment, reduced treatment burden, and increased compliance with the medication. The results in Table 18 demonstrate a non-significant trend in improvements in treatment burden for colistimethate sodium DPI compared with nebulised tobramycin. In addition, the Treatment Satisfaction Questionnaire for Medication (TSQM) reported in the EAGER trial65 showed higher values in the tobramycin DPI arm (see Appendix 5 for data). It is worth noting that in both of the trials the comparator was administered using a PARI LC Plus jet nebuliser; this device requires approximately 15 minutes to administer the full dose of tobramycin. Nebulisers with quicker delivery time (around 5 minutes), such as the PARI eFlow jet nebuliser, are now on the market and are in widespread use (Diana Bilton, personal communication). However, these quicker nebulisers may still require time to maintain (cleaning) and assemble. With respect to the relative advantages and disadvantages, it remains unclear whether the reduced treatment burden and improved treatment satisfaction scores would remain significant when compared with the newer, quicker nebulisers.

Adverse events

Table 19 shows the number of AEs by severity across the three trials. The percentage of patients experiencing any AE in all three of the trials is high, although this is to be expected in a patient population with CF, who have a high level of baseline AEs. Mild and moderate AEs appear similar between arms within trials. The EAGER trial65 does not state how many events were severe, whereas both of the colistimethate sodium DPI trials report more severe events in the intervention (DPI) arm. Serious adverse events (SAEs; which are internationally defined as AEs that cause death, are life-threatening, require hospitalisation/prolong hospitalisation or result in disability or birth defect89) appear to occur approximately equally but slightly less frequently in the DPI treatments in both key trials. There appears to be a marked difference between the proportion of serious events reported in the EAGER trial65 compared with the colistimethate sodium trials, even when comparing the nebulised tobramycin arms of the EAGER65 and COLO/DPI/02/06 trials. 66 This difference would appear to be too large to be explained by heterogeneity in study characteristics and study populations. It would seem more likely that this is due to a difference in interpretation of the ‘serious’ criteria between trials, rather than an indication that there is a true difference in number of events.

The number of patients withdrawing from the study due to AEs was higher in the dry powder intervention groups across all the trials. As previously discussed, patients in both trials were largely experienced with nebulised tobramycin, and it is likely that this difference in dropout rates is at least in part due to selection bias of patients tolerant to nebulised tobramycin, and desensitisation to its AEs through prior use.

Table 20 documents the most common AEs (≥ 5% in any group) occurring in any of the three trials. 65,66 The data presented relate to the number of patients who experienced AEs. Data on the actual number of events were available for the COLO/DPI/02/0666 and COLO/DPI/02/0566 trials, but are not presented here. The most common AE in all three trials65,66 is ‘cough’. The percentage of patients experiencing cough was higher in the COLO/DPI/02/0666 and COLO/DPI/02/0566 trials than in the EAGER trial,65 although this may again represent a difference in the definition of cough used in the studies rather than an actual difference in incidence of cough, as the difference persists when comparing the nebulised tobramycin arms of each trial. Cough was more common in the DPI intervention group for all trials. Cough is a known side effect of dry powder formulations and is thought to generally reduce over time, with improved technique, and may be controlled with use of bronchodilators, to some extent, in some patients. Clinical advisors to the Assessment Group were interested to note that haemoptysis did not appear to increase to any great extent in the tobramycin DPI group compared with the nebulised tobramycin group, although there does appear to be a small increase in haemoptysis in the colistimethate sodium DPI group compared with the nebulised tobramycin group. It is unclear whether this difference is clinically or statistically significant.

Although no statistical comparisons have been made, other AEs that appear to be worse in the DPI arm include (CiC information has been removed) chest discomfort and dysphonia in the tobramycin DPI arm in the EAGER trial,65 and throat irritation and dysgeusia in the colistimethate sodium DPI arm in the COLO/DPI/02/06 trial. 66 There are minor improvements in a number of other AEs in the colistimethate sodium DPI arm (see Table 20).

Mortality

Three patients died in the tobramycin DPI group in the EAGER trial65 (Table 21). Two patients died in the nebulised tobramycin group in the COLO/DPI/02/06 trial. 66 None of the deaths is attributed to the study medication. (CiC information has been removed.) For the COLO/DPI/02/0666 trial there were two deaths (both in the nebulised tobramycin group), both of which were assessed as being unrelated to study medication, and were attributed to the underlying disease, although it is unclear if these deaths were also due to acute exacerbations. One clinical advisor to the Assessment Group noted that the number of deaths seemed high for the size of the cohorts and length of the studies, and may indicate that the selected population were not well defined for the purpose of the study. With the small number of events in all arms and the relatively short time horizon of the trials, it is very difficult to draw firm conclusions regarding mortality.

Compliance

Compliance with study medication was reported in both key trials, but it is not clear whether or not the methods and analyses provided are compatible between trials. In the COLO/DPI/02/0666 trial, fewer patients were compliant with medication in the colistimethate sodium DPI arm than in the nebulised tobramycin arm (66.7% vs. 70.7%, respectively, complied with > 75% of doses). The EAGER trial65 did not define how compliance was judged but simply states it was ‘generally high’, with > 90% compliance in both arms. It is not clear if these data include those who withdrew, but seems likely that it does not as the discontinuation rate was 26.7% in the tobramycin DPI arm and 18.2% in the nebulised tobramycin arm, so values for compliance would be expected to be lower if these patients were counted. In comparison, withdrawals in the COLO/DPI/02/0666 were 17.2% in the colistimethate sodium DPI arm and 14.2% in the nebulised tobramycin arm, and it is unclear if these are counted in the compliance figures reported. Results for both DPI formulations do not appear to support the manufacturer’s claim that the improved delivery time would result in better compliance.

Cost-effectiveness review methods

Systematic literature searches were undertaken to identify published economic evaluations of colistimethate sodium and tobramycin for the treatment of CF. Details of the search strategies are reported in Chapter 3 (see Methods for reviewing clinical effectiveness) and the search strategies for the cost-effectiveness review are presented in Appendix 2. Hand-searching of sponsors’ submissions to NICE60,70 was also undertaken in order to identify any further studies missed by the electronic searches. The studies included in the review were critically appraised using the Drummond et al. checklist for economic evaluations. 90

Results of the systematic review

Three published studies were identified by the systematic searches. 5,91,92 None of these three studies relates to either colistimethate sodium or tobramycin in DPI form. However, these studies do provide some information concerning the costs and outcomes of the comparator therapies for this assessment and elucidate some of the key methodological problems surrounding the economic evaluation of treatments for CF. A critical appraisal of these studies is briefly detailed below.

Summary of published economic analyses

The review of the three published economic analyses presented above highlights the lack of relevant economic evidence relating to the cost-effectiveness of tobramycin and colistimethate sodium, in either nebulised or dry powder form, for the treatment of P. aeruginosa in patients with CF. Only one of the three studies is a cost-effectiveness analysis5 and, even in this case, the adopted measure of clinical benefit is difficult to interpret in a policy context. None of the three studies met the NICE reference case owing to their short time horizons. None of the included studies reported final patient outcomes in terms of life-years gained or QALYs gained.

The Novartis Pharmaceuticals submission (tobramycin dry powder for inhalation)

Novartis Pharmaceuticals60 submitted evidence to NICE relating to the clinical effectiveness of tobramycin DPI for the treatment of P. aeruginosa lung infection in patients with CF. The Novartis Pharmaceuticals submission presents the details of a NMA, a discussion of the difficulties of undertaking economic analyses of treatments for CF, and a brief discussion of three previously published economic analyses of CF treatments. 5,92,94 The submission makes particular note that these three economic analyses have deviated considerably from NICE’s reference case95 with respect to the primary health economic outcome measure adopted, which in each case relates to short-term FEV₁ improvements rather than QALYs gained. The Novartis Pharmaceuticals submission does not include any form of de novo economic evaluation. Although the submission states that a cost–utility analysis was explored, this was not pursued owing to data limitations (including the absence of sufficient public domain information relating to the efficacy of colistimethate sodium DPI), a failure to demonstrate statistical significance within the NMA, and the presence of considerable heterogeneity in study design across the trials included in the network. In addition, at the time of writing their submission, Novartis Pharmaceuticals had not proposed a list price, or potential range of list prices, for tobramycin DPI. The submission therefore does not present any economic evidence for tobramycin DPI.

The Forest Laboratories submission (colistimethate sodium dry powder for inhalation)

The Forest Laboratories submission66 reports the methods and results of five clinical studies of colistimethate sodium DPI and an economic analysis of colistimethate sodium DPI compared with nebulised tobramycin using data from the Phase III COLO/DPI/02/06 trial. 96

The review of the Forest Laboratories economic model undertaken by the Assessment Group is divided into three parts: (1) a descriptive exposition of the model’s mathematical structure and the evidence sources used to inform its parameters; (2) a critical appraisal of the Forest Laboratories model including a summary of adherence to, and deviations from, the NICE reference case;95 and (3) a reanalysis of the Forest Laboratories model using assumptions deemed more appropriate by the Assessment Group. This critical review is based on four main evidence sources which were made available to the Assessment Group by Forest Laboratories:

1. a partially executable cost-effectiveness model developed using Microsoft Excel^® version 2010 (Microsoft Corporation, Redmond, WA, USA) and Visual Basic for Applications

2. a written description of the methods and results of the economic analysis presented within the Forest Laboratories submission to NICE66

3. an accompanying mapping report detailing methods to estimate health utilities using data from the COLO/DPI/02/06 trial96

4. a detailed spreadsheet showing the translation of FEV₁% to expected QALY gains (note: this was not included within the original Forest Laboratories submission, but was later provided by Forest Laboratories as part of the clarification process for the appraisal).

In addition, further clarification regarding the methods of the analysis was sought from Forest Laboratories by the Assessment Group over the course of the technology appraisal.

Exposition of the Forest Laboratories model

The Forest Laboratories submission presents a model-based cost–utility analysis of colistimethate sodium DPI compared with nebulised tobramycin from the perspective of the UK NHS. The model time horizon is short, but is unclear and inconsistent between health outcomes and costs. The primary economic outcome is presented in terms of incremental net monetary benefit, assuming a willingness-to-pay threshold of £30,000 per QALY gained. The form of the model would be most accurately described as a cohort-based decision analysis.

The economic analysis includes an estimate of incremental QALY gains accrued over a lifetime horizon, and the short-term costs associated with antibiotic drug acquisition and the management of exacerbations in each treatment group. The economic analysis draws on seven evidence sources: (1) patient-level data from the COLO/DPI/02/06 trial;66 (2) patient-level data from a study in which individuals completed both the child-friendly EQ-SD, Youth version (EQ-5D-Y) and CFQ-R;96 (3) a mapping study used to map from the CFQ-R instrument to the EQ-5D-Y;97 (4) observational data relating to an assumed relationship between 1- and 2-year mortality risk and FEV₁% predicted;43 (5) a fixed estimate of life expectancy from the Cystic Fibrosis Foundation;98 (6) 2009–10 NHS reference costs;99 and (7) the BNF. 100

The general derivation of estimated QALYs for patients receiving either colistimethate sodium DPI or nebulised tobramycin within the Forest Laboratories model is summarised in Figure 8. This approach is based on three mapped relationships: (1) the translation of FEV₁% predicted to the probability of mortality at 1 or 2 years; (2) the estimation of remaining life expectancy given the individual patient’s age; and (3) the translation of the CFQ-R to the EQ-5D-Y.

FIGURE 8.

Quality-adjusted life-year derivation within the Forest Laboratories model.

Predicted mortality differences between colistimethate sodium DPI and nebulised tobramycin were estimated by deriving regression equations for mortality at 1 year and 2 years using reported data on FEV₁% predicted and death from a retrospective analysis of the risk of mortality by FEV₁, FVC, arterial oxygen pressure (PaO₂), arterial carbon dioxide tension (PaCO₂), sex, weight and height. 43 Forest Laboratories fitted polynomial regression equations to the Kerem et al. data43 for 1 or 2 years by FEV₁% group. The derived mortality risk equations are as follows:

⁽¹⁾ Academic-in-confidence information has been removed

⁽²⁾ Academic-in-confidence information has been removed

All patients in the analysis are assumed to have a fixed survival duration of 37.4 years, based on an estimate from the Cystic Fibrosis Foundation. Remaining expected life-years for each individual patient were calculated as the patient’s remaining life expectancy (maximum survival − current patient age) multiplied by the probability of surviving beyond 1 or 2 years based on the regression equation.

Preference-based health utilities were not collected within the COLO/DPI/02/06 trial. 66 Forest Laboratories undertook a mapping exercise to cross-walk from the CFQ-R to the EQ-5D-Y using patient data from a German study reported by Eidt-Koch et al. 96 This mapping exercise produced a single utility value for patients with CF with chronic P. aeruginosa lung infection within the COLO/DPI/02/06 trial. 66 The calculations used to estimate the total QALYs gained are not entirely clear from the Forest Laboratories submission but appear to adopt the following general logic:

1. The mortality risk equation based on Kerem et al. 43 was applied to the individual patient’s FEV₁% predicted score at baseline within COLO/DPI/02/0666 (Visit 0).

2. The individual patient’s remaining life expectancy was calculated as the difference between a fixed life expectancy of 37.4 years and the patient’s current age.

3. Expected QALYs before treatment were calculated as the probability of surviving at 1 or 2 years multiplied by the patient’s remaining life expectancy multiplied by a fixed utility score for patients with CF.

4. The Kerem et al. 43 mortality risk equation was applied to the individual patient’s FEV₁% predicted score at 24 weeks (Visit 6).

5. The individual patient’s remaining life expectancy was calculated as the difference between a fixed life expectancy of 37.4 years and the patient’s current age.

6. Expected QALYs after treatment were calculated as the probability of surviving at 1 or 2 years multiplied by the patient’s remaining life expectancy multiplied by a fixed utility score for patients with CF.

7. The mean QALY change for patients receiving a given treatment was calculated as the difference between mean predicted QALYs before treatment and mean predicted QALYs after 24 weeks of treatment (i.e. the difference between Step 6 and Step 3).

8. Steps 1 to 7 were undertaken separately for colistimethate sodium DPI and nebulised tobramycin. Incremental QALYs between the colistimethate sodium DPI and the nebulised tobramycin groups were calculated as the difference in mean QALY change within each treatment group.

Remaining survival was discounted at a rate of 3.5%. The FEV₁→QALY analysis excludes all patients already aged > 37.4 years and those for whom FEV₁% predicted estimates were not available at both Visit 0 and Visit 6 within COLO/DPI/02/06. 66

Acquisition costs for nebulised tobramycin were taken from the BNF. 100 The model assumes a cost per dose of £21.20. The Forest Laboratories submission states that the annual cost of tobramycin is £7738, which corresponds to a regimen in which two doses of nebulised tobramycin are used each day, and each 28-day treatment period is followed by 28 days without nebulised tobramycin (Forest Laboratories’ submission,66 see Table 8).

Acquisition costs for colistimethate sodium DPI are not yet listed within the BNF. The Forest Laboratories submission states that if colistimethate sodium DPI was priced at parity with nebulised tobramycin it would cost £7738.00 per year (Forest Laboratories’ submission, p. 32). 66 The model, however, includes a parameter called ‘Colobreathe price’, which has a value of £21.20 per dose, which, if used twice daily on a continuous basis, as per the COLO/DPI/02/06 trial,70 would imply an annual cost of £15,476.00 (2 × £21.20 × 365). The Forest Laboratories submission later states that the proposed unit cost for colistimethate sodium DPI is £39.29 per dose (Forest Laboratories’ submission, page 33);66 if used continuously, this would imply an annual treatment cost of £28,681.70 (2 × 39.29 × 365). Forest Laboratories later stated via personal correspondence that the inclusion of this price within the submission was a mistake but is included here for the sake of transparency. Towards the end of the appraisal process, Forest Laboratories stated that their anticipated price for colistimethate sodium DPI was in the range £510.00 and £1100.00 for 56 doses. The range of potential prices for colistimethate sodium DPI are summarised in Table 26. Importantly, the costs of antibiotic treatment are not actually included in Forest Laboratories’ calculations of incremental net benefit. During the peer review process for this assessment (and following EMA approval), Forest Laboratories put forward an anticipated list price of £895 per 56 doses; this corresponds to an annual cost of £11,666.96 per patient.

TABLE 26 - Proposed price summary for colistimethate sodium DPI

Implied price per year/dose.

No.	Price statement	Annual cost (£)	Cost per dose (£)	Source
1	Lower end of anticipated price range	6648.21a	9.11	Correspondence with Forest Laboratories during appraisal
2	Parity with tobramycin	7738.00	10.60a	Forest Laboratories’ submission, p. 32
3	Anticipated list price following EMA approval	11,666.96a	15.98	Correspondence with NICE during peer review
4	Upper end of anticipated price range	14,339.29	19.64	Correspondence with Forest Laboratories during appraisal
5	Cost per dose in model	15,476.00a	21.20	Forest Laboratories’ model parameter
6	Initial price	28,681.70a	39.29	Forest Laboratories’ submission, table 9

Rates of CF exacerbations were estimated by calculating the mean time to exacerbation in each treatment group and converting this to the mean number of exacerbations within a 1-year period, thereby assuming a constant exacerbation rate. For colistimethate sodium DPI, the mean time to first exacerbation was estimated to be 63.6 days; the mean number of exacerbations within 1 year was then calculated as 365/63.6 = 5.74 exacerbations. For nebulised tobramycin, the mean time to first exacerbation was estimated to be 59.4 days; the corresponding mean number of exacerbations within 1 year was calculated as 365/59.4 = 6.14 exacerbations. Each exacerbation was assumed to cost £2587 based on NHS reference cost tariffs for the management of asthma with major comorbidities and complications without intubation.

All parameter values used in the Forest Laboratories model are summarised in Table 27.

TABLE 27 - Parameter values used in the Forest Laboratories model

Model parameter	Value	Source
Health outcomes/treatment effectiveness
Health utility for CF	0.68	Jointly derived using data from the Cystic Fibrosis Foundation,98 Rowen and Brazier 201097 Kerem et al. 199243 and the COLO/DPI/02/06 trial66
QALYs
Colistimethate sodium DPI, 1-year mortality model	0.194
Colistimethate sodium DPI, 2-year mortality model	0.209
Nebulised tobramycin, 1-year mortality model	0.163
Nebulised tobramycin, 2-year mortality model	0.168
Time to exacerbation: colistimethate sodium DPI	63.6	COLO/DPI/02/0666
Time to exacerbation: nebulised tobramycin	59.4
Resource costs (£)
Cost of managing an exacerbation	2587.00	NHS reference costs 2009–1099
Assumed willingness-to-pay threshold (λ)	30,000	Forest Laboratories’ submission66

The incremental net benefit for colistimethate sodium DPI compared with nebulised tobramycin is simply calculated as:

⁽³⁾ ( QALYS Coli − QALYS Tobi neb × λ ) − ( costs Coli exacerbations − cost Tobi neb exacerbations )

where λ = assumed willingness-to-pay threshold; Coli = colistimethate sodium, and Tobi neb = nebulised tobramycin.

Critical appraisal of the Forest Laboratories model

This section presents a detailed critical appraisal of the Forest Laboratories model. This critical appraisal should be interpreted in light of the limitations of the available evidence base surrounding the effectiveness of colistimethate sodium DPI as compared against other antibiotics for the treatment of P. aeruginosa lung infection as well as the context of care within which these treatments are used. Most patients with chronic P. aeruginosa will receive antibiotics for the rest of their lives. However, there is no long-term evidence to demonstrate the efficacy of colistimethate sodium DPI or tobramycin DPI beyond a maximum 24-week trial follow-up period (see Chapter 5) and the short-term trial evidence that is available does not include the direct measurement of HRQoL using a preference-based instrument (e.g. the EQ-5D). There is also only very limited evidence relating to survival benefits for either colistimethate sodium DPI or tobramycin DPI. The implications of these problems are discussed further below (see Methodological issues surrounding the economic evaluation of cystic fibrosis treatments). The use of modelling as a means of translating from intermediate end points to final outcomes, and/or for projecting beyond the termination of a trial, is not a substitute for empirical evidence and should thus be interpreted with an appropriate degree of caution. Given these limitations in the available evidence, the appropriate handling of uncertainty should therefore be considered key.

Despite the limitations of the evidence base, the Forest Laboratories model is subject to a number of methodological problems that are likely to produce considerable bias in the results. These concerns, limitations and biases are summarised in Box 1; specific issues are then discussed in more detail below.

BOX 1 - Summary of key problems within the Forest Laboratories model

Multiple deviations from the NICE reference case.

Conceptually inconsistent time horizon for costs and health outcomes.

Assumption of intermittent treatment using colistimethate sodium DPI.

Limitations of the CFQ-R→EQ-5D-Y mapping exercise.

Questionable validity of methods for estimating mortality benefits.

Incremental net benefit estimates may not reflect the proposed price of colistimethate sodium DPI.

Potential biases in modelling of exacerbation rates.

Omission of relevant costs and health impacts.

Incorrect application of discounting formula applied to future health gains.

Limited justification of modelling methods and identification, selection and use of evidence.

Multiple deviations from the National Institute for Health and Care Excellence reference case

Table 30 shows the extent to which the Forest Laboratories model adheres to the NICE reference case. The perspective of the economic analysis, namely that of the NHS, is appropriate. The use of discounting is however partial. No discounting is undertaken for costs owing to the short time horizon considered. Future QALY gains were discounted. The justification for presenting economic results in terms of incremental net benefit rather than the incremental cost per QALY gained is unclear. Further, the Forest Laboratories model is entirely deterministic and the submission report does not include any probabilistic sensitivity analysis (PSA). No justification is given regarding this exclusion. Simple sensitivity analysis is presented but this is limited to examining the differential impact of using 1- or 2-year mortality predictions.

TABLE 30 - Adherence to the NICE reference case

PSS, Personal Social Services.

Element of economic analysis	Reference case	Comments
Defining the decision problem	The scope developed by the Institute	The submission report does not include a description of the scope of the decision problem to be addressed. The scope of the Forest Laboratories economic analysis is narrower than the scope of the appraisal.62 Only colistimethate sodium DPI is included as an intervention
Comparator	Therapies routinely used in the NHS, including technologies regarded as current best practice	Nebulised tobramycin is the sole treatment comparator considered within the analysis. The relative cost-effectiveness of colistimethate sodium DPI compared with nebulised colistimethate sodium or combination (cyclical switching) strategies is not considered
Perspective on costs	NHS and PSS	An NHS perspective was adopted. Only exacerbation costs over a 1-year period are included within the incremental net benefit calculation
Perspective on outcomes	All health effects on individuals	Health benefits for NHS patients are included. Short-term FEV1 changes are translated into QALY gains accruing over the patient’s estimated remaining lifetime
Type of economic evaluation	Cost-effectiveness analysis	The economic analysis takes the form of a cost-effectiveness analysis. Economic outcomes are expressed in terms of incremental net monetary benefit rather than the incremental cost per QALY gained
Synthesis of evidence on outcomes	Based on a systematic review	The economic analysis is based on one RCT (COLO/DPI/02/06) and other indirect evidence43,66,97,98
Measure of health effects	QALYs	Health outcomes are valued in terms of QALYs gained
Source of data for measurement of HRQoL	Reported directly by patients and/or carers	Health utilities were derived from a mapping study to translate the CFQ-R to the child-friendly EQ-5D-Y.97 Preferences were valued using the adult EQ-5D tariff. Carer QALYs and process utilities associated with more convenient treatment are discussed but not included in the analysis
Source of preference data for valuation of changes in HRQoL	Representative sample of the public
Discount rate	An annual rate of 3.5% on both costs and health effects	Costs were not discounted. Future QALY gains were discounted although the discount rate applied is incorrect
Equity weighting	An additional QALY has the same weight regardless of the other characteristics of the individuals receiving the health benefit	No additional equity weighting is applied to the modelled QALY gains

Conceptually inconsistent time horizon for costs and health outcomes

As noted above, there currently exists no evidence relating to the long-term costs or health outcomes associated with colistimethate sodium DPI or nebulised tobramycin for the treatment of P. aeruginosa lung infection in patients with CF. The Forest Laboratories submission does not state the intended time horizon for their economic analysis. However, it is evident from the model exposition presented above that the adopted time horizon is inconsistent in terms of the time period considered for costs and that considered for health outcomes. The model uses changes in FEV₁% predicted measured from baseline to 24 weeks within the COLO/DPI/02/06 trial66 and ‘extrapolates’ the impact of this shift in FEV₁ to lifetime QALY benefits. The economic model therefore reflects the predicted long-term mortality benefits associated with 24 weeks of treatment only. In direct contradiction to this, exacerbation costs are arbitrarily modelled over a 1-year period, without any consideration of longer-term costs. As acquisition costs are excluded entirely from the incremental net benefit calculation, there is an implication underlying the economic analysis that treatment in the intervention and control groups is the same after the first 24 weeks. Given this mismatch between the time horizon for costs and outcomes, it is conceptually unclear how the time horizon for incremental net benefit should be interpreted. The Forest Laboratories submission provides no discussion or justification of this issue.

Assumption of intermittent treatment using colistimethate sodium dry powder for inhalation

There is a significant bias with respect to the options assessed within the Forest Laboratories model. As the model includes equal cost-per-dose parameters for colistimethate sodium DPI and nebulised tobramycin, the model appears to assume that colistimethate sodium DPI is used according to the same treatment schedule as nebulised tobramycin, that is, each 28-day treatment period is followed by a 28-day period without treatment. This reflects the licensed indication for nebulised tobramycin, but does not reflect either the protocol or practice of the COLO/DPI/02/06 trial or the licensed indication for colistimethate sodium DPI. Within the COLO/DPI/02/06 trial, patients allocated to the colistimethate sodium DPI group received treatment on a continuous basis. 70 The consequence of assuming a ‘cycle-on, cycle-off’ regimen is that the modelled treatment benefits reflect those associated with the continuous use of colistimethate sodium DPI at only half of the cost of generating these benefits. Unless colistimethate sodium DPI is priced at parity with the annual cost of nebulised tobramycin, this is inappropriate and produces a substantial bias in favour of colistimethate sodium DPI.

Limitations of the Cystic Fibrosis Questionnaire-Revised to EQ-SD-Y mapping exercise

The COLO/DPI/02/06 trial66 did not include the collection of data on HRQoL using a preference-based instrument. As a consequence, no data were available from the clinical trial to produce direct estimates of QALYs gained for colistimethate sodium DPI or nebulised tobramycin. However, the COLO/DPI/02/06 trial66 did include the use of a disease-specific measure: the CFQ-R. To estimate health utilities associated with colistimethate sodium DPI and nebulised tobramycin, Forest Laboratories undertook a mapping exercise using patient-level data from a published supplementary study in patients with CF. 96 The mapping exercise relates to two distinct patient cohorts: (1) an ‘estimation data set’ – Eidt-Koch et al. 96 and (2) an ‘application data set’ – the COLO/DPI/02/06 trial. 66 Collection of data for the estimation data set96 was undertaken in 2006 across four CF centres in Germany. Within this study, a cohort of 96 patients with CF completed both the German version101 of the CFQ49,53 and the EQ-5D-Y. 102 Patients included in this study were generally of young age (mean = approximately 13 years, range 8–17 years) and mean FEV₁% predicted was generally high both in the children and adolescent groups (93.6% and 90.7%, respectively). Patient-level data from the estimation data set were used to produce a series of regression equations to ‘cross-walk’ from the CFQ to the EQ-5D-Y. The selected regression equation was then applied to patient-level data from the COLO/DPI/02/06 trial66 in which quality-of-life data were collected only using the CFQ-R instrument. Details relating to the estimation of alternative mapping functions in the child and adolescent populations were made available by Forest Laboratories as a technical appendix to the main submission. 97

There are a number of problems associated with the mapping exercise and its use within the colistimethate sodium DPI model; these are detailed below.

Comparability of populations within the estimation data set and the estimation data set

The NICE Decision Support Unit technical support document103 on the use of mapping in health technology appraisals states the following:

The characteristics of the estimation sample should be similar to the target sample for the mapping analysis, and should contain all variables from the target sample or included in the economic model that are thought to impact on EQ-5D scores. Under some circumstances, it may be appropriate for the estimation sample to include a broader range of people, providing that the target sample is sufficiently represented.

The comparability of the estimation data set96 and the application data set (COLO/DPI/02/06)66 appears to be subject to certain potentially important heterogeneities in terms of basic demographic and clinical variables. A crude comparison of patient characteristics within the Eidt-Koch et al. cohort96 and the baseline characteristics of patients recruited to COLO/DPI/02/0666 is presented in Table 31. In particular, only 65 patients (67.7%) included in Eidt-Koch et al. 96 had a bacterial colonisation of the lung, although it is unclear what proportion of these patients had chronic P. aeruginosa or an alternative type of bacterial infection. In addition, there are noticeable differences in terms of patient age and baseline FEV₁ lung function.

TABLE 31 - Comparison of demographic and clinical variables in the estimation and application data sets

ABPA, allergic bronchopulmonary aspergillosis; MEF, maximum expiratory flow; NR, not reported.

Variable	Estimation data set (Eidt-Koch et al.96)		COLO/DPI/02/06 at baseline66
	Children (8–13 years)	Adolescents (14–17 years)	ITT population (6–56 years)
Sex (male)	43.6% (n = 24)	58.5% (n = 24)	54.5% (n = 374)
Age (mean/SD)	10.8/1.7	15.9/1.80	21.1/9.49 58.8% patients were aged > 18 years
% vital capacity (mean/SD)	92.5/11.9 (n = 47)	97.2/13.1 (n = 34)	NR
%FEV1 (mean/SD)	93.6/15.2 (n = 47)	90.7/20.3 (n = 34)	Precise values not reported. FEV1 range 25–75% predicted required for eligibility
%MEF25 (mean/SD)	68.4/41.7 (n = 47)	58.9/37.5 (n = 34)	NR
Bacterial colonisation of the lung (%)	63.6 (n = 35)	73.2 (n = 30)	100% infected with chronic P. aeruginosa
Pneumothorax (%)	1.8 (n = 1)	0 (n = 0)	NR
ABPA (%)	3.6 (n = 2)	12.2 (n = 5)	Exclusion criteria within trial
Pancreatic insufficiency (%)	80.0 (n = 44)	78.1 (n = 32)	NR
Hepatobiliary complications (%)	23.6 (n = 13)	26.8 (n = 11)	33.3% in the colistimethate sodium DPI group; 40.3% in the nebulised tobramycin group
Distal intestinal obstruction (%)	7.3 (n = 4)	0 (n = 0)	NR
Diabetes mellitus (%)	0 (n = 0)	7.3 (n = 3)	NR
Nasal polyp (%)	10.9 (n = 6)	17.1 (n = 7)	NR
Isolation obligation for patient (%)	1.8 (n = 1)	9.8 (n = 4)	NR

Limited sample size within the estimation data set

The Eidt-Koch et al. study96 recruited only a small number of patients (n = 96). Of these, 93 patients completed both the CFQ and the EQ-5D-Y, and 93 patients were included in the mapping exercise. 97 Inevitably, this leads to considerable uncertainty surrounding the use of the mapping function, none of which is addressed in the health economic analysis.

Range of state space captured within the estimation data set

The Eidt-Koch et al. 96 publication states that 44.6% patients had no problems on any of the dimensions of the EQ-5D-Y. In other words, nearly half of the estimation data set cohort reported an EQ-5D-Y profile of (1,1,1,1,1), which represents a notional state of ‘perfect health’ (health utility = 1.0). This can be a common problem in utility mapping exercises, but is further compounded here by the small sample size of the estimation data set. At the lower ends of the scale, Eidt-Koch et al. 96 report that only ‘one or two patients reported extreme problems (level 3)’ on at least one of the dimensions of the EQ-5D-Y. As a consequence, the limited coverage of the EQ-5D state space within the estimation data set may call to question the validity of applying the mapping function to a cohort of patients with a generally higher level of disease activity.

Valuation of the EQ-SD-Y

Eidt-Koch et al. 96 used the child-friendly EQ-5D-Y (see Wille et al. 102). Within the mapping exercise, responses were valued using the UK adult EQ-5D tariff reported by Kind et al. 104 in which the lowest age of respondents was 18 years. A valuation tariff for children below this age does not currently exist.

Ambiguity regarding the selection and justification of the statistical mapping function

The Forest Laboratories submission66 presents 12 mapping functions including ordinary least squares (OLS), Tobit regressions and Censored Least Absolute Deviations (CLAD) forms. Seven alternative regression models are presented for children (aged 8–13 years) and five alternative regression models are presented for adolescents and adults (aged 14–17 years). The Forest Laboratories submission states ‘The preferred model was chosen using root mean squared error, mean squared error and mean absolute error’. These selection criteria are appropriate. 103 However, neither the submission report nor the accompanying appendices state which mapping function was actually selected for use in the health economic model analysis. Further, while the Forest Laboratories submission claims favourable benefits in terms of improved ease of use and improved sensitivity for colistimethate sodium DPI, the use of a single health utility score within the model indicates that such potential benefits are not captured in the economic model.

Questionable validity of methods for estimating mortality benefits

Although mortality was recorded within the COLO/DPI/02/06 trial66 as a safety end point, the study was not powered to demonstrate a treatment benefit in terms of survival. Within the COLO/DPI/02/06 trial,66 two patients died during the study follow-up period, both of whom were allocated to the nebulised tobramycin group (see Table 21). Both of these deaths were reported to be unrelated to the study drug and were instead attributed to the underlying disease. Within the economic analysis, modelled differences in survival are captured by deriving and applying regression equations describing a potential relationship between FEV₁% predicted and mortality at 1 year and 2 years from a retrospective analysis of the risk of mortality by FEV₁%, PaO₂, FVC, weight and height43 to patient-level changes in FEV₁% observed within the COLO/DPI/02/06 trial. 66 This change in predicted survival is then weighted by a single utility score, discounted, and compared incrementally between treatments. There are a number of problems with this approach, as detailed below.

Assumption of a single fixed life expectancy

The Forest Laboratories analysis assumes that all patients have a fixed maximum life expectancy of 37.4 years. In reality, the trial cohort would be expected to follow a survival distribution. Furthermore, the potential QALY gains of individuals with an age greater than 37.4 years within COLO/DPI/02/0666 were excluded from the analysis (n excluded = 32). The impact of this bias on the cost-effectiveness of colistimethate sodium DPI is unclear.

Validity of the relationship between FEV₁% predicted and mortality

The long-term mortality benefits included in the model are based on data presented within two figures reported by Kerem et al. 43 The Forest Laboratories submission itself notes that this assumed relationship is only ‘a suggestion’. 66 As the model applies a common health utility score for all patients irrespective of treatment group, this predicted survival benefit drives the entire QALY gain attributed to colistimethate sodium DPI. However, scrutiny of the Kerem et al. 43 publication indicates that increased mortality risk was also associated with decreasing PaO₂, increasing PaCO₂, increasing weight-for-height, and increasing age. Further, a multivariate regression presented within Kerem et al. 43 indicates that all variables except sex were statistically significant at the 5% level. It is therefore reasonable to suggest that the potential for confounding within the proposed FEV₁% mortality relationship is substantial. The validity of using FEV₁% as a single independent surrogate for mortality is not explored, justified or discussed within the submission.

Questionable value of the regression equation

Although Kerem et al. 43 clearly report categorical data, Forest Laboratories fitted their 1- and 2-year regression equations to the mid-point of each FEV₁% category (Figures 9 and 10; refitted by the Assessment Group), thereby inappropriately treating categorical data as if they were continuous. It is unclear why Forest Laboratories needed to apply a regression equation (which, in itself, is an approximation), as it should have been possible to directly apply the Kerem et al. 43 mortality probabilities to the categorical FEV₁% bands from COLO/DPI/02/06. 66 The value of the regression equation is thus unclear and is not justified within the submission.

It should also be noted that the FEV₁% mean change from baseline value reported in COLO/DPI/02/0666 did not show that colistimethate sodium DPI was superior to nebulised tobramycin. A crude analysis of the raw trial data indicates undertaken by the Assessment Group indicates that the mean FEV₁% change is actually more favourable for nebulised tobramycin than colistimethate sodium DPI. It therefore appears counterintuitive that a less favourable FEV₁% improvement may lead to a more favourable estimate of QALY gain for colistimethate sodium DPI than with nebulised tobramycin.

FIGURE 9.

Fitted mortality probabilities predicted at 1 year by FEV₁% category.

FIGURE 10.

Fitted mortality probabilities predicted at 2 years by FEV₁% category.

Incremental net benefit estimates do not reflect the price of colistimethate sodium dry powder for inhalation

On p. 32 of the Forest Laboratories submission, there is a suggestion that colistimethate sodium DPI will be priced at parity with nebulised tobramycin on an annual basis (£7738). 66 However, the economic model includes a parameter that indicates that the unit cost per dose of colistimethate sodium DPI is £21.20, which indicates price parity with tobramycin on a per-dose basis. As nebulised tobramycin is used on a ‘cycle-on, cycle-off’ basis, but colistimethate sodium DPI is not, the actual cost of colistimethate sodium priced on this basis would be double that of nebulised tobramycin over a 1-year period. The submission later states that as the model does not capture other benefits of colistimethate sodium DPI (more favourable performance regarding antimicrobial resistance, reduced costs of devices and consumables, reductions in carer time and ease of use), a per-dose price of £39.29 is proposed. This would obviously lead to a higher incremental cost for colistimethate sodium DPI than with nebulised tobramycin. Crucially, the positive incremental net benefits claimed within the submission do not therefore reflect either the appropriate dosage regimen or the actual proposed price of colistimethate sodium DPI; the results presented by Forest Laboratories therefore only reflect the scenario in which colistimethate sodium DPI is priced at parity with tobramycin on an annual basis. This is the lowest price suggested by Forest Laboratories (£9.11 per dose). As noted above, Forest Laboratories later proposed a cost of £895 for 56 doses of colistimethate sodium DPI (see Table 26).

Potential biases in modelling exacerbation rates

The Forest Laboratories model uses data on time to exacerbation for colistimethate sodium DPI and nebulised tobramycin from COLO/DPI/02/0666 to estimate the mean number of expected exacerbations over a 1-year period assuming a constant rate in each group. As the acquisition costs of the intervention and the comparator are not included in the incremental net benefit calculation, all predicted cost savings are driven by this element of the model. In order to produce these time-to-event estimates, Forest Laboratories must have used data on the observed number of exacerbations in each group together with the time at which they occurred. Therefore, the analysis takes the observed number and timing of exacerbations, converts these into time-to-event estimates, and then converts them back to the estimated number of exacerbations over a one year period. No justification of this approach is provided within the Forest Laboratories submission. It would have been more appropriate simply to use annualised exacerbation rates (taking into account censored observations) using the COLO/DPI/02/06 trial data,66 and a relative risk to reflect the differences between treatment groups.

Omission of relevant costs and health impacts

The systematic review of clinical effectiveness (see Chapter 3) reported that across many AEs, incidence was higher for colistimethate sodium DPI than nebulised tobramycin. Many of these AEs may be self-limiting and transient in nature, and the costs of managing them may be minimal. However, the Forest Laboratories model does not include any consideration of these costs or potential impacts on HRQoL and the submission report does not discuss or justify their exclusion.

Incorrect discounting formulae applied to future health benefits

Although the Forest Laboratories analysis includes discounting of future health benefits, the discounting formulae have been applied incorrectly. Within the Forest Laboratories FEV₁% → QALY analysis, the discount weight is calculated using the following formula for each additional year of survival:

⁽⁴⁾ Discounted utility in a given year = UtilityValue . exp ( − DiscountRate .year )

If this method is used to calculate discount weights, the discount rate r should be converted to (log[1 + r]). This error produces only a minor bias in the results.

Limited justification of modelling methods and identification, selection and use of evidence

The Forest Laboratories submission presents very little justification for the modelling approach adopted. The methods used to identify, select or use particular sources of evidence (e.g. Kerem et al. 43) are not discussed within the Forest Laboratories submission.

Reanalysis of the Forest Laboratories model by the Assessment Group

This section presents some simple reanalyses of the Forest Laboratories model to demonstrate the impact of some of the biases detailed above. These analyses are presented as detailed in Table 32 for both the 1- and 2-year mortality models.

The results of these alternative analyses are presented in Table 33. It should be noted that this reanalysis does not fully resolve the problems regarding the model time horizon, the health impact of AEs, or the considerable uncertainty surrounding the estimation of QALY benefits for colistimethate sodium DPI. The results of the analysis suggest that the price of colistimethate sodium DPI and the time horizon for costs are highly sensitive within the analysis. The reanalysis suggests that if colistimethate sodium DPI is priced lower than nebulised tobramycin per annum, it may dominate due to modelled cost savings associated with avoided exacerbations and the estimated incremental QALY gains. For the range of higher prices per dose administered, the incremental cost-effectiveness of colistimethate sodium DPI compared with nebulised tobramycin is in the range £42,872–485,550 per QALY gained depending on assumptions regarding time horizon, mortality estimates and drug acquisition costs.

Discussion of available economic evidence

There is clearly considerable uncertainty surrounding the cost-effectiveness of alternative antibiotics for the treatment of P. aeruginosa in patients with CF. The review of published economic evaluations did not identify any directly relevant studies that report on the cost-effectiveness of colistimethate sodium DPI or tobramycin DPI compared with current standard treatments.

The Novartis Pharmaceuticals submission did not report any economic results for tobramycin DPI within their submission.

Forest Laboratories did present a simple economic analysis; however, this is subject to a number of methodological weaknesses, as detailed above. The majority of these weaknesses cannot be easily rectified given Forest Laboratories’ adopted model structure. One of the most significant problems within the Forest Laboratories model relates to the direct contradiction between the lifetime context of care and the apparently short time horizon adopted. As a consequence it is unclear how the results of the net benefit analysis should be interpreted. There is no obvious reason why colistimethate sodium DPI would be stopped after 24 weeks, yet Forest Laboratories’ economic analysis appears to imply the use of a ‘stopping rule’ at this point. This reflects the limitations of the trial evidence and the methods for estimating QALYs rather than what would be considered reasonable clinical practice. The disparity between the time horizon for clinical benefit, the cost of managing exacerbations and drug costs mean that it is impossible to interpret Forest Laboratories’ economic analysis in a meaningful way. Further, the methods for translating a lower level of FEV₁ benefit for colistimethate sodium DPI into a greater number of QALYs than nebulised tobramycin remains counterintuitive. Given Forest Laboratories’ model structure, if the selected price of colistimethate sodium DPI is set to one of the prices which is higher than nebulised tobramycin, the cost per QALY gained for colistimethate sodium DPI compared with nebulised tobramycin is expected to be in the range £42,872–485,550.

The next section discusses the difficulties in undertaking a robust economic evaluation of colistimethate sodium DPI and tobramycin DPI in order to justify the modelling approach adopted by the Assessment Group and to highlight the uncertainties surrounding the results of the de novo analysis.

Absence of any direct evidence of the impact of treatment on health-related quality of life

Within the pivotal trials of colistimethate sodium and tobramycin DPI, HRQoL was not directly assessed using a preference-based health utility instrument. As such, it is not possible to directly estimate health utilities for each competing treatment option from these sources. Whilst Forest Laboratories reported a mapping exercise to translate the CFQ-R to the EQ-5D,97 the resulting estimates of health utility were not differentiated by treatment; instead a common mean value was applied to both treatments. If it is plausible that DPI treatment influences quality of life, the only means of quantifying this is by assuming some relationship between other clinical end points measured within the clinical trials and their impact on HRQoL.

The use of a short time horizon within the pivotal trials of colistimethate sodium dry powder for inhalation and tobramycin dry powder for inhalation

Research recommendations from the EMA CHMP state that for interventions which are intended to slow or stop pulmonary disease progression, a 12-month FEV₁ end point should be used. 18 Although this 12-month end point would represent only the impact of treatment within a limited proportion of a patient’s lifetime, neither pivotal trial of colistimethate sodium DPI or tobramycin DPI met this criterion, as both trials were less than 6 months in duration. The adoption of such short study durations has three negative consequences: (1) the trial durations are insufficient to assess any treatment benefits in terms of potential mortality reduction; (2) uncertainties surrounding the relevance of intermediate outcome measures such as FEV₁% predicted and final end points such as mortality are inflated by the absence of long-term evidence; and (3) evidence surrounding long-term AEs and treatment compliance is absent.

Owing to the short study durations adopted within the EAGER trial65 and the COLO/DPI/02/06 trial,66 mortality estimates are subject to very high levels of censoring (approximately 99% in each trial). Within the COL/DPI/02/06 trial, no patients died in the colistimethate sodium DPI arm, whereas two patients died within the tobramycin arm. 70 Within the EAGER trial,65 three patients died in the tobramycin DPI arm, whereas no patients died within the nebulised tobramycin arm. These event numbers are insufficient for comparative survival extrapolation over a lifetime horizon.

Validity of relationships between intermediate and final end points

As a consequence of the absence of direct comparative evidence of HRQoL impacts and the limited evidence of survival benefits, the economic analysis of treatments for P. aeruginosa requires some proposition and quantification of relationships between other clinical end points which may impact on HRQoL and/or survival. In order for an intermediate end point to be useful, it must represent an end point that can substitute for and be predictive of a final patient relevant clinical outcome. 105 Judgements about the credibility and validity of such relationships may be made on the basis of a range of evidence and may be interpreted within a hierarchy, as suggested by Taylor and Elston105 (Table 34).

Potential intermediate outcome measures include one or more of the following: FEV₁% predicted, exacerbation rates and the incidence and duration of other AEs. 18 The plausibility and methodological problems of using these relationships to estimate the QALY gains associated with DPI treatment are considered below.

Limited availability of evidence on clinical outcome measures for all relevant treatments

A comprehensive economic analysis of CF treatments would synthesise all relevant evidence on treatment effects within a meta-analytic framework. 103 However, from the perspective of health economic evaluation, this type of evidence synthesis would be useful only if a plausible and quantifiable relationship exists between FEV₁%, or other intermediate clinical end points, and HRQoL and/or survival. The majority of clinical trials of colistimethate sodium and tobramycin (in either dry powder or nebulised form) report mean change in FEV₁% within the trial cohorts, and very few report FEV₁% outcomes beyond 4 weeks. Given the concerns regarding the validity of the relationships between FEV₁% and mortality and HRQoL outlined above, this would not be useful, as it would require that the translated relationship has interval properties (e.g. x% change in FEV₁% leads to y% change in HRQoL). Further, the systematic reviews presented above suggest that this type of relationship is unlikely to hold; the value of a NMA based on summary data is therefore questionable in this context.

Scope of the economic analysis

A number of potential options are relevant to the economic analysis of antibiotic treatments for P. aeruginosa. These include:

1. colistimethate sodium DPI

2. tobramycin DPI

3. colistimethate sodium nebulised

4. tobramycin nebulised

5. aztreonam.

Some patients may switch between tobramycin and colistimethate sodium at some point in their lives. This may be happen due to apparent treatment failure on the current drug, or may be part of a planned treatment regimen whereby colistimethate sodium and tobramycin are alternated every 28 days.

The Assessment Group developed a de novo health economic model to assess the cost-effectiveness of two competing treatment options: (1) colistimethate sodium DPI compared with (2) nebulised tobramycin for the treatment of chronic P. aeruginosa in patients with CF. A number of potentially relevant interventions and comparators were therefore excluded from the analysis (Table 40). In addition, a crude threshold analysis is presented to compare tobramycin DPI with nebulised tobramycin.

Model structure

The model estimates the expected costs and QALY gains associated with colistimethate sodium DPI compared with nebulised tobramycin. The analysis adopts an NHS perspective over a lifetime horizon. The primary economic outcome measure for the analysis is the incremental cost per QALY gained. All costs and health outcomes within the model were discounted using the standard approach at a rate of 3.5%. Costs were valued at 2011 prices.

The model takes the form of a state transition model to estimate transitions between three FEV₁% strata [(1) FEV₁ 70–99%; (2) FEV₁ 40–69% and (3) FEV₁ < 40%]. Twenty-four week transition probabilities are estimated, based on those observed within COL/DPI/02/0666. Different levels of HRQoL are assumed for each health state. Treatment duration, which is assumed to be directly related to survival duration, is assumed to be exactly equivalent between the competing treatment options. During each cycle, patients may remain in their current FEV₁% state, transit to an improved or worsened FEV₁% state or die. Patients with FEV₁ of < 40% may undergo lung transplantation and do not subsequently receive further treatment with colistimethate sodium DPI or tobramycin; other treatments received by these patients are assumed to be identical irrespective of previous antibiotic treatments received. Additional HRQoL decrements are applied for minor and major exacerbations based on treatment-specific rates and data relating to the mean time receiving i.v. antibiotics. Total QALYs are calculated as the total sojourn time in each health state weighted by the respective utility for that health state, less any QALY losses resulting from exacerbations. Costs within each treatment group include drug acquisition costs and the costs of managing exacerbations (either in hospital or at home). Potential cost savings associated with reduced maintenance of nebulisers are also included in the economic analysis. Costs associated with follow-up and concomitant medications are assumed to be related only to treatment time and are therefore assumed to be equivalent between treatment groups. A conceptual form of the implemented health economic model is presented in Figure 11.

FIGURE 11.

Conceptual form of the implemented economic model.

Given the considerable uncertainty surrounding the extrapolation of the 24-week efficacy data to a lifetime horizon, two separate analyses are presented:

1. a reference case analysis based on FEV₁% extrapolation over a lifetime horizon

2. a ‘within-trial’ analysis that does not include any extrapolation.

Separate analyses are presented for each of the six prices presented in Table 26.

Evidence used to inform the model parameters

The sources of evidence used to inform the model parameters are detailed below. For the most part, the model parameters have been informed directly using data from the COLO/DPI/02/06 trial;66 these have been augmented using other data from external sources. A summary of model parameter values and distributions used in the base-case analysis is presented in Table 41.

Patient survival

Mean survival for patients with CF was estimated using data reported by Dodge et al. 22 This study reported survival data up to the end of 2003 for all subjects with CF born in the UK in the period 1968–92, collated via active enquiry of CF clinics and other hospital consultants. Survival curves are reported within this paper separately for males and females (see Figure 2). Data are not available on the number of patients at risk over time.

Engauge^® digitising software version 5 (http://digitizer.sourceforge.net/) was used to replicate the published survival data from the graphs assuming a 50 : 50 split between males and females. Parametric survival curves were fitted to these data in order to estimate the mean durations of survival within the cohorts. Exponential, Weibull, linear, Gompertz and log-logistic curves were fitted to the empirical survival curve data. Each of these curves result in different distributions of survival. Information regarding the number of patients at risk and the number of events was not available from Dodge et al. 22 hence curves were fitted using Solver add-in within Microsoft Excel. Each curve was inspected visually with respect to how well the distribution fitted the observed data. The plausibility of the unobserved portion of each curve was considered by comparing the median survival of the fitted curve against the predicted median survival from the 2010 CF Registry report (Table 42).

TABLE 42 - Median predicted survival from the CF Registry6

Year	2007	2008	2009	2010
Median predicted survival: years (95% CI)	35.2 (31.0 to 42.6)	38.8 (34.2 to 47.3)	34.4 (30.7 to 37.0)	41.4 (36.8 to 46.7)

Figure 12 presents the actual and predicted survival using a range of different curves. The Weibull and log-logistic models appear to provide the best fit to the data. Both of these curves provide a reasonable fit to the median survival as well as the overall distribution. However, the tail of the log-logistic distribution appears to overestimate survival during the later decades of life. Therefore, the Weibull curve was used in the base-case economic analysis.

FIGURE 12.

Long-term survival estimated from Dodge et al. 22

Uncertainty surrounding the two parameters of the Weibull survivor function was modelled using a multivariate normal distribution. As patient-level data were not available, the variance and covariance of the parameters was assumed rather than estimated. These were fitted against the maximum and minimum median predicted survival data from the CF Registry reports for the years 2007–10.

Key assumptions within the de novo economic analysis

The model makes the following key assumptions

1. FEV₁ measurements are stable and not subject to measurement error.

2. HRQoL is assumed to differ by FEV₁% strata.

3. Transitions between FEV₁% strata over time are assumed to be independent of patient’s previous transitions.

4. Colistimethate sodium DPI has no additional benefit over nebulised tobramycin in terms of patient survival.

5. The costs of follow-up and concomitant medication are equivalent between colistimethate sodium DPI and nebulised tobramycin.

Uncertainty analysis

The model is fully probabilistic. Monte Carlo sampling was used to propagate uncertainty through the model in order to produce distributions of expected costs and outcomes for each treatment option. The model was run over 5000 Monte Carlo samples. The results of the PSA are presented in terms of incremental cost-effectiveness planes and CEACs. Simple sensitivity analyses were undertaken to examine the impact of parameter uncertainty and structural uncertainty on the model results. The following analyses were explored:

• Scenario 1. The analysis was run using point estimates of parameters rather than the probabilistic means.

• Scenarios 2–6. Secondary analyses are presented using alternative FEV₁% utility estimates reported by Yi et al. 110 and Stahl et al. 109 It should be noted that these studies report health utility using different categories of FEV₁% bands (FEV₁ > 79%, FEV₁ 60–79%, FEV₁ 40–59% and FEV₁ < 40%). As such, it was necessary to redefine the structure of the model and re-estimate the transition probabilities between four instead of three states. The general logic of the model, however, remains the same as the base-case analysis.

• Scenario 7. FEV₁% transition probabilities for the nebulised tobramycin group were set equal to those for the colistimethate sodium DPI group.

• Scenario 8. The HRQoL decrement associated with minor and major exacerbations was doubled.

• Scenario 9. The cost of hospitalisation for major exacerbations was doubled.

Model validation and verification methods

A number of measures were taken to ensure that the Assessment Group model was credible and not subject to computational errors. First, the methods and results of the health economic model were peer reviewed by three clinical advisors to the project (see Acknowledgements). The executable model and its underlying logic were checked by the model authors and a third modeller who was not involved in its development. The expectations of each model parameter distribution were compared against their deterministic counterpart. All model input parameters were double-checked against the sources from which they were derived. The plausibility of the model results were considered against the model developers’ expectations of those results prior to model development.

In addition to the above activities, a validation exercise was undertaken to examine the plausibility of the extrapolated Markov trace based on the COLO/DPI/02/06 trial66 by deriving equivalent transition matrices using longitudinal panel data from the CF Registry for the period 1997–2008. Transition matrices were generated as follows:

• The first FEV₁% measurement each calendar year was taken as each patient’s observation for year t.

• As patients entered the registry during different calendar years, the first observation for each patient was transposed to a common starting year.

• Missing data between observations were imputed according to a LOCF rule (final observations were not imputed).

• FEV₁ scores were mapped to three FEV₁% bands (FEV₁ > 70%, FEV₁ 40–69%, FEV₁ < 40%).

• Transition matrices for year t were calculated based on the number of patients transiting between each state between years t and year t + 1.

These transition matrices were then applied to the initial distribution of patients in the model and the resulting Markov trace was compared against the Markov trace for the tobramycin group. The results of this analysis are shown in Figure 13.

FIGURE 13.

Comparison of registry- and trial-derived Markov trace.

This analysis shows that the registry-derived transition matrices suggest a similar shape in the Markov trace as those derived from the COLO/DPI/02/06 trial. 66 There are clearly some differences between the traces generated using the registry data and the trial; however, the FEV₁ < 40% state population, which has the greatest impact on HRQoL, appears fairly similar between the two sources. Some of this discrepancy may be caused by differences between the registry and the trial in terms of patient characteristics, for example the registry cohort does not exclusively include those patients with P. aeruginosa. This analysis lends some weight to the credibility of the trial extrapolation.

Simplifications and exclusions from the economic analysis

Health economic results

Uncertainty analysis

Results of the long-term reference case economic analysis

Figures 14–25 present cost-effectiveness planes and CEACs for the long-term reference case model over the range of pricing scenarios for colistimethate sodium DPI.

FIGURE 14.

Reference case model: cost-effectiveness plane (price per dose = £9.11).

Assuming a price per dose of £9.11, the long-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £20,000 per QALY gained is around 0.32. The probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is also approximately 0.32. The probability that colistimethate sodium DPI is dominated by nebulised tobramycin is also approximately zero.

FIGURE 15.

Reference case model: CEAC (price per dose = £9.11).

Assuming a willingness-to-pay threshold of £20,000 per QALY gained and a price per dose of £9.11, the probability that colistimethate sodium DPI is optimal is approximately 0.98. At a willingness-to-pay threshold of £30,000 per QALY gained, the probability that colistimethate sodium DPI is optimal is approximately 0.92.

FIGURE 16.

Reference case model: cost-effectiveness plane (price per dose = £10.60).

Assuming a price per dose of £10.60, the long-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £20,000 per QALY gained is around 0.32. The probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is also approximately 0.32. The probability that colistimethate sodium DPI is dominated by nebulised tobramycin is also approximately 0.01.

FIGURE 17.

Reference case model: CEAC (price per dose = £10.60).

Assuming a willingness-to-pay threshold of £20,000 per QALY gained and a price per dose of £10.60, the probability that colistimethate sodium DPI is optimal is approximately 0.55. At a willingness-to-pay threshold of £30,000 per QALY gained, the probability that colistimethate sodium DPI is optimal is approximately 0.48.

FIGURE 18.

Reference case model: cost-effectiveness plane (price per dose = £15.98).

Assuming a price per dose of £15.98, the long-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is zero. The probability that colistimethate sodium DPI is dominated is approximately 0.68.

FIGURE 19.

Reference case model: CEAC (price per dose = £15.98).

Assuming a willingness-to-pay threshold of £20,000 per QALY gained and a price per dose of £15.98, the probability that colistimethate sodium DPI is optimal is zero. At a willingness-to-pay threshold of £30,000 per QALY gained, the probability that colistimethate sodium DPI is optimal is also zero.

FIGURE 20.

Reference case model: cost-effectiveness plane (price per dose = £19.64).

Assuming a price per dose of £19.64, the long-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is zero. The probability that colistimethate sodium DPI is dominated is approximately 0.68.

FIGURE 21.

Reference case model: CEAC (price per dose = £19.64).

Assuming a willingness-to-pay threshold of £20,000 per QALY gained and a price per dose of £19.64, the probability that colistimethate sodium DPI is optimal is zero. At a willingness-to-pay threshold of £30,000 per QALY gained, the probability that colistimethate sodium DPI is optimal is also zero.

FIGURE 22.

Reference case model: cost-effectiveness plane (price per dose = £21.20).

Assuming a price per dose of £21.20, the long-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is zero. The probability that colistimethate sodium DPI is dominated is approximately 0.68.

FIGURE 23.

Reference case model: CEAC (price per dose = £21.20).

Assuming a willingness-to-pay threshold of £30,000 per QALY gained and a price per dose of £21.20, the probability that colistimethate sodium DPI is optimal is zero.

FIGURE 24.

Reference case model: cost-effectiveness plane (price per dose = £39.29).

Assuming a price per dose of £39.29, the long-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is zero. The probability that colistimethate sodium DPI is dominated is approximately 0.68.

FIGURE 25.

Reference case model: CEAC (price per dose = £39.29).

Assuming a willingness-to-pay threshold of £30,000 per QALY gained and a price per dose of £39.29, the probability that colistimethate sodium DPI is optimal is zero.

Results of the short-term ‘within-trial’ economic analysis (excluding any extrapolation)

Figures 26–37 present cost-effectiveness planes and CEACs for the short-term model over the six pricing scenarios for colistimethate sodium DPI.

FIGURE 26.

Short-term model: cost-effectiveness plane (price per dose = £9.11).

Assuming a price per dose of £9.11, the short-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £20,000 per QALY gained is around 0.23. The probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is also approximately 0.23. The probability that colistimethate sodium DPI is dominated is approximately zero.

FIGURE 27.

Short-term model: CEAC (price per dose = £9.11).

Assuming a willingness-to-pay threshold of £20,000 per QALY gained and a price per dose of £9.11, the probability that colistimethate sodium DPI is optimal within the short-term model is approximately 1.0. At a willingness-to-pay threshold of £30,000 per QALY gained, the probability that colistimethate sodium DPI is optimal is also approximately 1.0.

FIGURE 28.

Short-term model: cost-effectiveness plane (price per dose = £10.60).

Assuming a price per dose of £10.60, the short-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £20,000 per QALY gained is around 0.23. The probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is also approximately 0.23. The probability that colistimethate sodium DPI is dominated is approximately 0.03.

FIGURE 29.

Short-term model: CEAC (price per dose = £10.60).

Assuming a willingness-to-pay threshold of £20,000 per QALY gained and a price per dose of £10.60, the probability that colistimethate sodium DPI is optimal within the short-term model is approximately 0.77. At a willingness-to-pay threshold of £30,000 per QALY gained, the probability that colistimethate sodium DPI is optimal is approximately 0.65.

FIGURE 30.

Short-term model: cost-effectiveness plane (price per dose = £15.98).

Assuming a price per dose of £15.98, the short-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is zero. The probability that colistimethate sodium DPI is dominated is approximately 0.77.

FIGURE 31.

Short-term model: CEAC (price per dose = £15.98).

Assuming a willingness-to-pay threshold of £30,000 per QALY gained and a price per dose of £15.98, the probability that colistimethate sodium DPI is optimal within the short-term model is zero.

FIGURE 32.

Short-term model: cost-effectiveness plane (price per dose = £19.64).

Assuming a price per dose of £19.64, the short-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is zero. The probability that colistimethate sodium DPI is dominated is approximately 0.77.

FIGURE 33.

Short-term model: CEAC (price per dose = £19.64).

Assuming a willingness-to-pay threshold of £30,000 per QALY gained and a price per dose of £19.64, the probability that colistimethate sodium DPI is optimal within the short-term model is zero.

FIGURE 34.

Short-term model: cost-effectiveness plane (price per dose = £21.20).

Assuming a price per dose of £21.20, the short-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is zero. The probability that colistimethate sodium DPI is dominated is approximately 0.77.

FIGURE 35.

Short-term model: CEAC (price per dose = £21.20).

Assuming a willingness-to-pay threshold of £30,000 per QALY gained and a price per dose of £21.20, the probability that colistimethate sodium DPI is optimal within the short-term model is zero.

FIGURE 36.

Short-term model: cost-effectiveness plane (price per dose = £39.29).

Assuming a price per dose of £39.29, the short-term model suggests that the probability that colistimethate sodium DPI produces a positive QALY gain and has an ICER that is better than £30,000 per QALY gained is zero. The probability that colistimethate sodium DPI is dominated is approximately 0.77.

FIGURE 37.

Short-term model: CEAC (price per dose = £39.29).

Assuming a willingness-to-pay threshold of £30,000 per QALY gained and a price per dose of £39.29, the probability that colistimethate sodium DPI is optimal within the short-term model is approximately zero.

The results of the PSA highlight an important aspect of the model: although there is clearly considerable uncertainty surrounding the extrapolation of the COLO/DPI/02/06 trial data,66 this element of the model has virtually no bearing on the economic conclusions of the model, as colistimethate sodium DPI remains dominated when the price is set at one of the higher prices than that of nebulised tobramycin.

Simple (deterministic) sensitivity analysis

Table 47 presents the results of the simple sensitivity analysis; this is presented only for the long-term model. It should be noted that this analysis is deterministic and uses the point estimates of each parameter rather than the expectation of the mean (although the transition probabilities also include the weak priors to allow for comparison against the probabilistic results).

The results of the deterministic sensitivity analysis presented in Table 47 show that the results are particularly sensitive to the choice of utility values used within the model. Where colistimethate sodium DPI produces a positive QALY gain, this is very small and results in ICER ranging from dominating to in excess of £121M per QALY gained.

TABLE 47 - Simple sensitivity analysis (long-term model)

Colistimethate sodium DPI price (£)	QALYs			Costs (£)			ICER
	Colistimethate sodium DPI	Tobramycin nebulised	Incremental	Colistimethate sodium DPI	Tobramycin nebulised	Incremental
1. Deterministic point estimates for parameters
9.11	9.46	9.57	−0.12	93,720.31	110,278.31	−16,558.00	143,325
10.60	9.46	9.57	−0.12	107,166.44	110,278.31	−3111.87	26,936.14
15.98	9.46	9.57	−0.12	155,716.90	110,278.31	45,438.59	Dominated
19.64	9.46	9.57	−0.12	188,745.65	110,278.31	78,467.34	Dominated
21.20	9.46	9.57	−0.12	202,823.48	110,278.31	92,545.17	Dominated
39.29	9.46	9.57	−0.12	366,072.13	110,278.31	255,793.82	Dominated
2. TTO utility values from Yi et al.110
9.11	11.83	11.83	0.00	93,651.08	110,086.92	−16,435.84	Dominating
10.60	11.83	11.83	0.00	107,087.28	110,086.92	−2999.64	Dominating
15.98	11.83	11.83	0.00	155,601.87	110,086.92	45,514.96	21,699,661
19.64	11.83	11.83	0.00	188,606.22	110,086.92	78,519.31	37,434,780
21.20	11.83	11.83	0.00	202,673.65	110,086.92	92,586.74	44,141,552
39.29	11.83	11.83	0.00	365,801.72	110,086.92	255,714.80	121,914,312
3. SG utility values from Yi et al.110
9.11	11.15	11.15	0.00	93,651.08	110,086.92	−16,435.84	Dominating
10.60	11.15	11.15	0.00	107,087.28	110,086.92	−2999.64	Dominating
15.98	11.15	11.15	0.00	155,601.87	110,086.92	45,514.96	13,458,617
19.64	11.15	11.15	0.00	188,606.22	110,086.92	78,519.31	23,217,891
21.20	11.15	11.15	0.00	202,673.65	110,086.92	92,586.74	27,377,581
39.29	11.15	11.15	0.00	365,801.72	110,086.92	255,714.80	75,613,992
4. HUI-2 utility values from Yi et al.110
9.11	10.25	10.24	0.01	93,651.08	110,086.92	−16,435.84	Dominating
10.60	10.25	10.24	0.01	107,087.28	110,086.92	−2999.64	Dominating
15.98	10.25	10.24	0.01	155,601.87	110,086.92	45,514.96	4,030,094
19.64	10.25	10.24	0.01	188,606.22	110,086.92	78,519.31	6,952,444
21.20	10.25	10.24	0.01	202,673.65	110,086.92	92,586.74	8,198,036
39.29	10.25	10.24	0.01	365,801.72	110,086.92	255,714.80	22,642,110
5. EQ-5D values from Stahl et al.109 (GOLD criteria)
9.11	8.28	8.36	−0.08	93,651.08	110,086.92	−16,435.84	213,185
10.60	8.28	8.36	−0.08	107,087.28	110,086.92	−2999.64	38,907
15.98	8.28	8.36	−0.08	155,601.87	110,086.92	45,514.96	Dominated
19.64	8.28	8.36	−0.08	188,606.22	110,086.92	78,519.31	Dominated
21.20	8.28	8.36	−0.08	202,673.65	110,086.92	92,586.74	Dominated
39.29	8.28	8.36	−0.08	365,801.72	110,086.92	255,714.80	Dominated
6. EQ-5D values from Stahl et al.109 (BTS criteria)
9.11	8.74	8.81	−0.06	93,651.08	110,086.92	−16,435.84	264,903
10.60	8.74	8.81	−0.06	107,087.28	110,086.92	−2999.64	48,346
15.98	8.74	8.81	−0.06	155,601.87	110,086.92	45,514.96	Dominated
19.64	8.74	8.81	−0.06	188,606.22	110,086.92	78,519.31	Dominated
21.20	8.74	8.81	−0.06	202,673.65	110,086.92	92,586.74	Dominated
39.29	8.74	8.81	−0.06	365,801.72	110,086.92	255,714.80	Dominated
7. Transition probabilities for nebulised tobramycin set equal to those for colistimethate sodium DPI
9.11	9.46	9.46	0.00	93,724.58	110,118.99	−16,394.41	Dominating
10.60	9.46	9.46	0.00	107,171.33	110,118.99	−2947.67	Dominating
15.98	9.46	9.46	0.00	155,724.00	110,118.99	45,605.00	9,793,372
19.64	9.46	9.46	0.00	188,754.25	110,118.99	78,635.26	16,886,400
21.20	9.46	9.46	0.00	202,832.72	110,118.99	92,713.73	19,909,658
39.29	9.46	9.46	0.00	366,088.82	110,118.99	255,969.82	54,967,822
8. Utility decrement for exacerbations doubled
9.11	9.41	9.52	−0.11	93,720.31	110,278.31	−16,558.00	149,445
10.60	9.41	9.52	−0.11	107,166.44	110,278.31	−3111.87	28,086
15.98	9.41	9.52	−0.11	155,716.90	110,278.31	45,438.59	Dominated
19.64	9.41	9.52	−0.11	188,745.65	110,278.31	78,467.34	Dominated
21.20	9.41	9.52	−0.11	202,823.48	110,278.31	92,545.17	Dominated
39.29	9.41	9.52	−0.11	366,072.13	110,278.31	255,793.82	Dominated
9. Cost of hospitalisation doubled
9.11	9.46	9.57	−0.12	103,782.02	120,772.51	−16,990.49	147,069
10.60	9.46	9.57	−0.12	117,228.15	120,772.51	−3544.36	30,680
15.98	9.46	9.57	−0.12	165,778.61	120,772.51	45,006.10	Dominated
19.64	9.46	9.57	−0.12	198,807.36	120,772.51	78,034.85	Dominated
21.20	9.46	9.57	−0.12	212,885.19	120,772.51	92,112.68	Dominated
39.29	9.46	9.57	−0.12	376,133.84	120,772.51	255,361.33	Dominated

Commentary on the cost-effectiveness of tobramycin dry powder for inhalation

The de novo Assessment Group model explicitly excludes tobramycin DPI. This decision was taken by the Assessment Group, and NICE were informed of this in December 2011. This exclusion reflects the absence of suitable FEV₁% data for tobramycin DPI and the absence of a health economic model within the Novartis Pharmaceuticals submission to NICE. 60 Despite the absence of a model, it is possible to crudely postulate the likely incremental cost-effectiveness of tobramycin DPI compared with nebulised tobramycin on the basis of the following observations regarding the available evidence base.

• There is no empirical evidence that tobramycin DPI provides an improved or equivalent level of HRQoL as nebulised tobramycin.

• Within the EAGER trial,65 study follow-up was insufficient to assess any potential benefit in survival duration for tobramycin DPI. Three deaths occurred, all of which were in the tobramycin DPI group. 65

• There appears to be a small incremental FEV₁% predicted associated with tobramycin DPI (see Table 12). However, as above (see Methodological issues surrounding the economic evaluation of CF treatments), assumptions of a simple independent relationship between FEV₁% predicted and mortality should be interpreted with caution.

• Although Novartis Pharmaceuticals60 did not provide data on exacerbations requested by the Assessment Group, results reported by Konstan et al. 65 suggest that the incidence of lung disorder, which remains the best available proxy for exacerbation incidence, was higher in the tobramycin DPI group (relative risk = 1.12). It is likely that the cost of managing exacerbations would therefore be higher for tobramycin DPI than nebulised tobramycin. This would also likely result in a small QALY loss.

• The EAGER trial65 suggests a less favourable profile for tobramycin DPI across almost all of the common AEs (especially cough and dysphonia) compared with nebulised tobramycin.

• The incremental drug cost per 28-day treatment cycle for tobramycin DPI is £602.80 higher than that for nebulised tobramycin. Based on the treatment time within the Assessment Group model, this would result in a discounted lifetime drug and nebuliser cost of around £144,442 per patient. Compared against nebulised tobramycin, the discounted lifetime incremental drug cost of tobramycin DPI is around £46,168 per patient. This explicitly excludes any cost disadvantage associated with the apparently higher exacerbation rate for tobramycin DPI.

Given the incremental cost of tobramycin DPI, tobramycin DPI would have to produce 1.54 additional discounted QALYs compared with nebulised tobramycin to achieve a cost–utility ratio of £30,000 per QALY gained. In order to achieve a cost per QALY ratio of £20,000, tobramycin DPI would have to produce 2.31 additional discounted QALYs compared with nebulised tobramycin. Given the nebulised tobramycin transition matrix from COLO/DPI/02/0666 for the comparator, the assumed treatment starting age (21 years) and the use of EQ-5D values for alternative FEV₁% bands,60,111 neither of these incremental QALY thresholds is actually possible within the Assessment Group model.

As noted above, there may be a process-related utility benefit associated with tobramycin DPI owing to treatment convenience that has not been considered within the above analysis. However, the trial investigators did not collect any information relating to HRQoL and therefore the plausibility of such an argument cannot be demonstrated empirically.

Following this assessment, the Assessment Group undertook further economic analyses of both colistimethate sodium DPI and tobramycin DPI compared with nebulised tobraymcin. These analyses included Patient Access Schemes for both DPI products. These analyses are not presented here.

Summary of available evidence

There is a dearth of economic evidence relating to the cost-effectiveness of colistimethate sodium DPI and tobramycin DPI for the treatment of P. aeruginosa in patients with CF. The literature review did not identify any published economic analyses of colistimethate sodium DPI or tobramycin DPI. Novartis Pharmaceuticals60 did not submit any economic evidence relating to the cost-effectiveness of tobramycin DPI. Forest Laboratories did submit an economic model to assess colistimethate sodium DPI compared with nebulised tobramycin, which suggests that colistimethate sodium DPI is expected to dominate nebulised tobramycin. However, this was subject to a number of methodological problems and biases that are likely to produce overly favourable estimates of cost-effectiveness. The basis of this model assumes that an absolute FEV₁ measurement is directly associated with survival duration. A review of the literature suggests that the validity of this relationship is dubious and is likely to be subject to considerable confounding. A reanalysis of the Forest Laboratories model using more plausible assumptions suggests that the cost-effectiveness of colistimethate sodium DPI compared with nebulised tobramycin is expected to range from dominating to £485,550 per QALY gained, depending on the price of the intervention.

Summary of the economic analysis undertaken by the Assessment Group

The Assessment Group developed a de novo health economic model based on patient-level data from the COLO/DPI/02/06 trial66 augmented using external sources. This model extrapolates 24-week FEV₁% to a lifetime horizon. Although this extrapolation is clearly subject to considerable uncertainty, the conclusions of the analysis appear robust as the short-term 24-week analysis of the COLO/DPI/02/06 trial66 produces consistent results to the lifetime model. An analysis of longitudinal patient-level data from the CF Registry suggests that the probabilities of transition between FEV₁% are relatively stable over time, which lends some weight to the credibility of the trial extrapolation.

The results of this analysis suggest that colistimethate sodium DPI is expected to produce fewer QALYs than nebulised tobramycin, both in the short-term and over a lifetime horizon. If the price of colistimethate sodium DPI is set at one of the prices which is higher than that of nebulised tobramycin, it is expected to be more expensive and hence dominated by nebulised tobramycin. If the price of colistimethate sodium DPI is set at £9.11, the incremental cost-effectiveness of nebulised tobramycin compared with colistimethate sodium DPI is expected to be in the range £126,000–277,000 per QALY gained. If the price of colistimethate sodium DPI is set at £10.60, the incremental cost-effectiveness of nebulised tobramycin compared with colistimethate sodium DPI is expected to be in the range £24,000–50,000 per QALY gained.

Insufficient data were available, at the time of the assessment, to produce a full economic evaluation of tobramycin DPI compared with any comparator. Instead, a crude threshold analysis is presented to estimate the necessary QALY gain that tobramycin DPI would need to produce given its incremental lifetime cost. The model structure suggests that given its acquisition cost, it is not possible for tobramycin to have a cost-effectiveness ratio that is better than £30,000 per QALY gained.

Principal findings: clinical effectiveness

Three trials were included in the review. 65,66 Both colistimethate sodium DPI and nebulised tobramycin DPI were reported to be non-inferior to nebulised tobramycin in pivotal Phase III non-inferiority trials, for the outcome FEV₁%, based on information from two of the trials. However, there are problems with the trials, which indicate that the results should be judged with caution. None of the trials complied with the time horizon of 12 months’ follow-up recommended by the EMA for efficacy trials, with both following patients for 24 weeks only. As such, the existing evidence base does not include information about the long-term efficacy and safety of these treatments. Both of the large trials65,66 could also be criticised for the way they analysed the results, with the COLO/DPI/02/06 trial66 of colistimethate sodium DPI only reaching non-inferiority when analysed non-parametrically, and the EAGER trial65 of tobramycin DPI only presenting results without imputation. The COLO/DPI/02/0566 trial was not powered to detect an effect in FEV₁%, was only 4 weeks in duration (prior to crossover), and reported no significant differences in FEV₁% between arms and from baseline. It was not possible to draw any firm conclusions as to the relative efficacy as measured by FEV₁% of any intervention compared with any other intervention (except nebulised tobramycin) owing to missing data, uncertain comparability of patient characteristics and incompatible methods of analysing the data.

As FEV₁% is a surrogate outcome, the EMA recommends that it should be considered alongside ‘harder’ outcomes such as exacerbations, and should be supported with microbiological data. Sputum density data for tobramycin DPI supported the FEV₁% values seen with a decrease at week 20. Data on sputum density outcomes were not available for colistimethate sodium DPI. Resistance of around 20% was reported for the tobramycin arms across both Phase III trials, and of ≤ 1.1% for colistimethate sodium DPI. Both tobramycin DPI and colistimethate sodium DPI appeared to be less effective in reducing the frequency of exacerbations, but patients treated with DPIs spent less time on antibiotics. AEs were mostly similar between arms within trials, except for cough, which was higher in both DPI intervention arms. More patients in the DPI intervention arms withdrew owing to AEs in both trials. 65,66 The statistical and clinical significance of differences in sputum density, resistance data, exacerbations and AE data is not known. Insufficient mortality events were recorded and the study follow-up was not long enough to draw conclusions as to the effect of DPI formulations on mortality in comparison with nebulised tobramycin.

Principal findings: cost-effectiveness

The cost-effectiveness of colistimethate sodium DPI and tobramycin DPI are subject to considerable uncertainty. This is driven by a number of factors including (1) an absence of any direct method of HRQoL elicitation within the pivotal clinical trials; (2) the short-term nature of follow-up within these studies and the absence of sufficient survival data to allow extrapolation; (3) the questionable validity of absolute measures of FEV₁ as an independent predictor of CF mortality; and (4) gaps in the evidence base concerning the relative effectiveness of competing treatments for P. aeruginosa lung infection. Given current evidence, questions relating to the long-term cost-effectiveness of the colistimethate sodium DPI and tobramycin are therefore inevitably hinged on the credibility of relationships between intermediate and final outcomes.

A systematic review of existing cost-effectiveness studies did not identify any full economic evaluations of colistimethate sodium DPI or tobramycin DPI. Previous analyses were short term and did not involve extrapolation to more relevant time horizons, nor did they involve the translation of intermediate outcomes to more policy-relevant economic outcome measures.

Two submissions were received from the manufacturers of colistimethate sodium DPI and tobramycin DPI. Novartis Pharmaceuticals60 did not submit any economic evidence to support the argument that tobramycin DPI represents a cost-effective use of resources. Forest Laboratories66 did submit an economic model to assess colistimethate sodium DPI compared with nebulised tobramycin, which suggests that colistimethate sodium DPI is expected to dominate nebulised tobramycin. However, this was subject to a number of methodological problems and biases that are likely to produce overly favourable estimates of cost-effectiveness. The Forest Laboratories submission was ambiguous with respect to the actual proposed price of colistimethate sodium DPI, and the net benefit estimates produced from the economic model did not include the acquisition costs of either the intervention or the comparator. This model is underpinned by the assumption that an absolute FEV₁ measurement is directly associated with survival duration (either at 1 year or 2 years post measurement). A review of the available literature suggests that the validity of this relationship is dubious and is likely to be subject to confounding owing to other clinically relevant variables. Even if this relationship is considered plausible, and the methods of prediction are considered accurate, a reanalysis of the Forest Laboratories model using more plausible assumptions suggests that the incremental cost–utility of colistimethate sodium DPI compared with nebulised tobramycin is expected to range from dominating to £485,550 per QALY gained, depending on the proposed price of the intervention.

The de novo health economic model developed by the Assessment Group is based on patient-level data from the COLO/DPI/02/06 trial66 augmented using external sources. This model defines differential states of HRQoL by FEV₁ strata, and extrapolates the observed FEV₁ transitions within COLO/DPI/02/06 trial to a lifetime horizon. No additional survival benefit is assumed. Although this extrapolation is clearly subject to considerable uncertainty, the conclusions of the analysis appear robust as the short-term 24-week analysis of the COLO/DPI/06 trial66 produces consistent results to the lifetime model. The results of this economic analysis suggest that colistimethate sodium DPI is expected to produce fewer QALYs than nebulised tobramycin, both in the short term and over a lifetime horizon. If the price of colistimethate sodium DPI set at one of the prices that is higher than that of nebulised tobramycin, it is expected to be more expensive and hence dominated by nebulised tobramycin. If the price of colistimethate sodium DPI is set at £9.11, the incremental cost-effectiveness of nebulised tobramycin compared with colistimethate sodium DPI is expected to be in the range £126,000–277,000 per QALY gained. If the price of colistimethate sodium DPI is set at £10.60, the incremental cost-effectiveness of nebulised tobramycin compared with colistimethate sodium DPI is expected to be in the range £24,000–50,000 per QALY gained.

At the time of the assessment, insufficient data were available to produce a full economic evaluation of tobramycin DPI compared with any relevant comparator. Instead, a crude threshold analysis is presented to estimate the necessary QALY gain that tobramycin DPI would need to produce in order to achieve a particular cost–utility ratio, given its incremental lifetime cost. In order to achieve a cost–utility ratio of £30,000 per QALY gained, tobramycin DPI would need to produce an estimated 1.54 additional discounted QALYs compared with nebulised tobramycin. In order to achieve a cost per QALY ratio of £20,000, tobramycin DPI would need to produce an estimated 2.31 additional discounted QALYs compared with nebulised tobramycin. The model structure suggests that neither of these QALY thresholds is achievable given the price of tobramycin DPI.

Following this assessment, the Assessment Group undertook further economic analyses of both colistimethate sodium DPI and tobramycin DPI compared with nebulised tobraymcin. These analyses included Patient Access Schemes for both DPI products. These analyses are not presented here.