Notes
Article history
The research reported in this article of the journal supplement was commissioned and funded by the HTA programme on behalf of NICE as project number 06/50/01. The assessment report began editorial review in October 2008 and was accepted for publication in March 2009. See the HTA programme web site for further project information (www.hta.ac.uk). This summary of the ERG report was compiled after the Appraisal Committee’s review. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report. The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.
Declared competing interests of authors
the authors are not aware of any competing interests
Permissions
Copyright statement
© 2009 Queen’s Printer and Controller of HMSO. This monograph may be freely reproduced for the purposes of private research and study and may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
2009 Queen’s Printer and Controller of HMSO
This paper presents a summary of the submission’s evidence for the clinical effectiveness and cost-effectiveness of varenicline for smoking cessation included four studies of varenicline (one of which was commercial-in-confidence) and a meta-analysis of varenicline versus nicotine replacement therapy (NRT), bupropion and placebo. Two controlled trials of 12 weeks of varenicline versus sustained-release bupropion and placebo suggested that varenicline results in a statistically significant improvement in the odds of quitting at 12 weeks [odds ratio (OR) for quit rate during last 4 weeks of the study: 1.90–1.93 (p < 0.001) varenicline versus bupropion; 3.85 (p < 0.001) varenicline versus placebo). The ORs for sustained abstinence (weeks 9–52) for varenicline versus bupropion were 1.77 (p = 0.004) and 1.46 (p = 0.057), and for varenicline versus placebo were 2.66–3.09 (p < 0.01). A placebo-controlled maintenance trial examined whether a further 12 weeks of varenicline would maintain the rate of abstinence among those successfully treated on one 12-week course [OR = 2.48 at week 24 for varenicline versus placebo (p < 0.001)]. The meta-analysis suggested that varenicline was superior to placebo and bupropion at 1 year and 3 months. Based on indirect comparisons, varenicline was reported to be superior to NRT when compared with placebo or all controls at 1 year and 3 months. The submission presented a state transition model to estimate the incremental cost-effectiveness of varenicline compared with bupropion, NRT and placebo. The model suggests that varenicline dominates bupropion, NRT and placebo.Treatment efficacy was based on a pooled analysis of 1-year quit rates from the varenicline clinical trials. Assuming a willingness-to-pay threshold range of £20,000–30,000 per quality-adjusted life-year gained, the probabilistic sensitivity analysis suggests that the probability that varenicline produces the greatest amount of net benefit is 0.70. Weaknesses of the manufacturer’s submission include the assumption that only a single quit attempt using a single smoking cessation intervention is made, the presence of multiple computational errors and a limited sensitivity analysis. In conclusion, varenicline is likely to be clinically and costeffective for smoking cessation assuming that each user makes a single quit attempt. The key area of uncertainty concerns the long-term experience of subjects who have remained abstinent from smoking beyond 12 months. The guidance issued by the National Institute for Health and Clinical Excellence in July 2007 states that varenicline is recommended within its licensed indications as an option for smokers who have expressed a desire to quit smoking and that varenicline should normally be prescribed only as part of a programme of behavioral support.
Introduction
The National Institute for Health and Clinical Excellence (NICE) is an independent organisation within the NHS that is responsible for providing national guidance on the treatment and care of people using the NHS in England and Wales. One of the responsibilities of NICE is to provide guidance to the NHS on the use of selected new and established health technologies, based on an appraisal of those technologies
NICE’s single technology appraisal (STA) process is specifically designed for the appraisal of a single product, device or other technology, with a single indication, for which most of the relevant evidence lies with one manufacturer or sponsor (Pfizer). Typically, it is used for new pharmaceutical products close to launch. The principal evidence for an STA is derived from a submission by the manufacturer/sponsor of the technology. In addition, a report reviewing the evidence submission is submitted by the evidence review group (ERG), an external organisation independent of NICE. This paper presents a summary of the ERG report for the STA of varenicline for smoking cessation.
Description of the underlying health problem
Three million deaths a year worldwide can be attributed to smoking,1 and it is a major etiological factor for lung cancer, cardiovascular disease and peripheral vascular disease. Smoking also causes respiratory disease, such as chronic obstructive pulmonary disease (COPD), including bronchitis and emphysema. Half of all smokers in the UK die prematurely of a smoking-related ailment, with the decrease in life expectancy for regular smokers under the age of 35 years who continue to smoke estimated to be about 8 years (www.nice.org.uk; accessed 15 December 2006). 2 The annual cost to the NHS of treating patients with smoking-related disease is around £1.5 billion. 3
The proportion of adults in the UK who smoked cigarettes fell substantially during the 1970s and the early 1980s, after which it declined gradually until the early 1990s. Since this time it has plateaued, and in 2003–4 26% of adults aged 16 or over smoked cigarettes, an identical rate to that in 2002/3. The gap between men and women smokers has narrowed, and in 2003–4 28% of men and 24% of women were cigarette smokers. In July 2004 the government set a new target to reduce the overall proportion of cigarette smokers in England to 21% or less by 2010 (www.statistics.gov.uk; accessed 15 December 2006).
Inhaled nicotine is strongly addictive and stopping smoking results in craving and withdrawal symptoms. However, smokers who quit before the age of about 35 years have a life expectancy only slightly less than those who have never smoked. Even cessation in middle age improves health and substantially reduces the excess risk of death, and quitting at any age provides both immediate and long-term health benefits. It is estimated that about 4 million smokers a year attempt to quit, but that only 3–6% of these (1–2% of all smokers) succeed (www.nice.org.uk; accessed 15 December 2006).
Smokers have a range of options when the decision has been made to attempt to quit, the most common of which is unaided cessation, so-called ‘cold turkey’. Other alternatives are bupropion, counselling with or without pharmacotherapy, hypnosis, acupuncture or use of over-the-counter nicotine replacement therapy (NRT).
GPs in the UK maintain a record of the smoking habits of all patients and are encouraged to offer advice and support to smokers to help them quit. Smokers can be referred to a local smoking cessation service where counselling will be offered and, if deemed appropriate, pharmacological support prescribed.
Scope of the ERG report
The principal research question is whether varenicline is clinically effective and costeffective compared with NRT or bupropion, an antidepressant, in supporting smoking cessation in adults who smoke tobacco products and have indicated a desire to quit smoking. Varenicline is a selective nicotinic receptor partial agonist that is indicated for smoking cessation in adults. The recommended dose is 1 mg of varenicline twice daily following a 1-week titration period. At the time of writing of the ERG report the cost of varenicline was £1.95 per day per patient.
Key outcomes presented within the sponsor submission include: survival, morbidity related to smoking, quit rates, adverse effects of treatment, health-related quality of life and cost-effectiveness. Clinical effectiveness outcomes are presented only for the intention to treat populations within the clinical trials; subgroup analyses are not presented. 4
Methods
The ERG report3 comprised a critical review of the evidence for the clinical effectiveness and cost-effectiveness of the technology based upon the manufacturer’s/sponsor’s submission to NICE as part of the STA process.
The sponsor commissioned an independent review group to undertake a meta-analysis and indirect comparison of controlled trials. Aside from the indirect comparison, the McMaster review makes comparisons of clinical effectiveness previously undertaken in three (publicly funded) Cochrane reviews, the latest versions of which are by Silagy et al. 5 (NRT), Hughes et al. 6 (bupropion) and Cahill et al. 7 (varenicline). As these reviews were all relatively recent we did not undertake new searches. We used the Cochrane Tobacco Addiction Group to identify studies that were inappropriately excluded from the review. The ERG reran the meta-analyses and undertook an additional indirect comparison to validate the manufacturer’s estimates of treatment effect.
A mathematical model to estimate the incremental cost-effectiveness of varenicline versus bupropion, NRT and placebo was presented by the sponsor; this model was made available to the ERG for scrutiny. The model was based upon an earlier smoking cessation model [the Health and Economic Consequences of Smoking (HECOS) model] previously reported by Orme et al. 8 The model uses the state transition methodology to simulate the experiences of individuals following an initial attempt to quit smoking. The model includes five morbidities that are related to smoking: COPD, lung cancer, coronary heart disease (CHD) events, asthma and stroke. These morbidities were included in the model as they were reported by the sponsor to account for the greatest mortality, morbidity and cost associated with smoking. The ERG critically appraised the sponsor’s model and undertook a detailed assessment of its internal and external consistency.
Results
Summary of submitted clinical evidence
The sponsor submission reported the methods and results of four clinical studies of varenicline. The first two studies were double-blind controlled trials of 12 weeks of varenicline versus sustained-release bupropion and placebo. These studies suggested that varenicline results in a statistically significant improvement in the odds of quitting at 12 weeks. The odds ratio (OR) for the quit rate during the last 4 weeks of the study was 1.90–1.93 (p < 0.001) for varenicline versus bupropion, and 3.85 (p < 0.001) for varenicline versus placebo. In terms of sustained abstinence (weeks 9–52), the OR for varenicline versus bupropion was significantly different in one study (OR = 1.77, p = 0.004), but not in another (OR = 1.46, p = 0.057). When compared against placebo, the OR for the sustained quit rate for varenicline versus placebo was 2.66–3.09; this improvement was statistically significant in both studies (p < 0.01). The third study was a placebo-controlled maintenance trial that examined whether a further 12 weeks of varenicline treatment would maintain the rate of abstinence among those successfully treated on one 12-week course of varenicline. At week 24, patients who received varenicline had an OR of 2.48 of maintaining abstinence compared with patients who received placebo; this improvement was statistically significant (p < 0.001). For weeks 13–52, the improvement remained significant (OR = 1.34, p < 0.02). The fourth study was an open-label study that compared 12 weeks of varenicline therapy with 10 weeks of NRT transdermal patch. The results of this study were held as commercial-in-confidence.
The sponsor submission also detailed a large meta-analysis of varenicline versus NRT, bupropion and placebo. This analysis suggested that varenicline was superior to placebo and bupropion at 1 year and also at approximately 3 months. Based on indirect comparisons, varenicline was reported to be superior to NRT when compared with placebo controls or to all controls at 1 year and at 3 months.
Summary of submitted cost-effectiveness evidence
The submission reports the methods and results of a state transition model (the Benefits of Smoking Cessation on Outcomes or BENESCO model) to estimate the incremental cost-effectiveness of varenicline compared with bupropion, NRT and placebo. The model suggests that varenicline dominates (i.e. is more effective and less expensive than) bupropion, NRT and placebo. Treatment efficacy for each of the interventions is based on the results of a pooled analysis of 1-year quit rates sourced from the clinical trials of varenicline. Beyond this point the model assumes that short-term efficacy translates into long-term health gains and associated cost savings. This assumption of sustained benefit is subject to a substantial degree of uncertainty. Shorter time horizons may be less uncertain, but may underestimate the benefits of varenicline. Longer time horizons provide more favourable cost-effectiveness estimates for varenicline yet are subject to a much greater degree of uncertainty. Assuming a willingness-to-pay threshold range of £20,000–30,000 per quality-adjusted life-year gained, the probabilistic sensitivity analysis suggests that the probability that varenicline produces the greatest amount of net benefit is estimated to be 0.70.
Commentary on the robustness of submitted evidence
Strengths
The manufacturers have recruited a team of researchers from McMaster University (Hamilton, Ontario) to produce and publish a systematic review, which they have used as the basis for their analysis.
The structural assumptions included in the submission model appear to be intuitively sensible, and the costs and consequences of most important smoking-related morbidities (lung cancer, COPD, asthma, CHD and stroke) are included in the analysis.
Weaknesses
The manufacturer’s use of indirect comparisons is inappropriate because they had access to a direct comparison (the commercial-in-confidence randomised control trial). The indirect comparison was also flawed because it was based on a meta-analysis that inappropriately included and excluded studies, the effect of which would have been to exaggerate the effect size of varenicline.
The model assumes only a single quit attempt using a single smoking cessation intervention (varenicline, bupropion, NRT or placebo). In reality, smokers may attempt to quit more than once using several smoking cessation technologies. The costs and health outcomes of repeated quit attempts are not considered within the evaluation.
The model extrapolates lifetime outcomes for subjects attempting to quit smoking (up to 81 years of extrapolated costs and consequences) based on a pooled analysis of 1-year efficacy outcomes from clinical trials.
The model uses a large number of parameter values derived from US studies that may not reflect the smoking/abstinence behaviour of the population of England and Wales.
Methods for identifying and selecting costs and health utilities associated with morbidities are not reported or justified within the sponsor submission.
The presence of multiple computational errors should be borne in mind when considering cost-effectiveness results reported within the sponsor submission. Most notable was a structural error that violated a key condition of the Markov approach; consequently, the probability of being in any health state at any point in time does not consistently sum to 1 over the duration of the model time horizon.
The sensitivity analysis presented within the submission is very narrow and underestimates the true uncertainty surrounding the incremental cost-effectiveness of varenicline. In particular, the probabilistic sensitivity analysis was restricted to a limited number of parameters and is inherently flawed. The true uncertainty surrounding the incremental cost-effectiveness of varenicline has not been appropriately addressed within the submission.
The external validity of the model has not been demonstrated by the sponsor.
Conclusions
Varenicline is likely to be clinically effective and cost-effective if one assumes, as the clinical trials and the manufacturer’s model do, that each user makes a single quit attempt. The key area of uncertainty concerns the long-term experience of subjects who have remained abstinent from smoking beyond 12 months. The health economic model makes an assumption of sustained benefit for the remaining 81 years of the time horizon. The validity of the assumption of sustained benefit between treatment groups is unclear.
Summary of NICE guidance issued as a result of the STA
At the time of writing the guidance document issued by NICE in July 20079 states that:
-
Varenicline is recommended within its licensed indications as an option for smokers who have expressed a desire to quit smoking.
-
Varenicline should normally be prescribed only as part of a programme of behavioral support.
Disclaimers
The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the Department of Health.
Key references
- Peto R, Lopez AD, Boreham J, Thun M, Heath C, Doll R. Mortality from smoking worldwide. Br Med Bull 1996;52:12-21.
- National Institute for Health and Clinical Excellence (NICE) . Guide to the Single Technology (STA) Process n.d. www.nice.org.uk/page.aspx?o=STAprocessguide (accessed 19 September 2006).
- Pfizer UK Ltd . Varenicline 2009.
- Hind D, Tappenden P, Peters J, Kenjegalieva K. Varenicline for smoking cessation: a single technology appraisal. Report to the National Institute for Health and Clinical Excellence; 2007.
- Silagy C, Lancaster T, Stead L, Mant D, Fowler G. Nicotine replacement therapy for smoking cessation. Cochrane Database Syst Rev 2004;3.
- Hughes JR, Stead LF, Lancaster T. Antidepressants for smoking cessation. Cochrane Database Syst Rev 2007;1.
- Cahill K, Stead LF, Lancaster T. Nicotine receptor partial agonists for smoking cessation. Cochrane Database Syst Rev 2007.
- Orme ME, Hogue SL, Kennedy LM, Paine AC, Godfrey C. Development of the health and economic consequences of smoking interactive model. Tob Control 2001:10-61.
- National Institute for Health and Clinical Excellence . Varenicline for Smoking Cessation 2007.