What is the clinical effectiveness and cost effectiveness of cytisine compared with varenicline for smoking cessation: a systematic review and economic evaluation?

Measurement of abstinence

Clinical trials of interventions for smoking cessation may use a range of outcome measurements to evaluate abstinence. The Russell Standard21 is a set of criteria widely used to define smoking abstinence. These guidelines were developed in response to results from smoking cessation trial data historically being reported in a number of different ways. The Russell Standard outlines criteria important to the measurement and reporting of outcome data in such trials. A full description of these criteria are found in West et al. 21 Regarding the measurement of abstinence, the criteria recommend a duration of 6 months or 12 months, either from a designated quit date or allowing for a predefined grace period. Shorter periods of abstinence are reported to be insufficient in their ability to accurately predict long-term cessation. Regarding the definition of abstinence, historically a number of methods of measuring abstinence have been used. These include continuous abstinence, defined as abstinence between quit day and follow-up; prolonged abstinence, defined as sustained abstinence after an initial grace period, or to a period of sustained abstinence between two follow-ups; point prevalence abstinence, defined as the prevalence of abstinence during a time window immediately preceding follow-up and repeated point prevalence abstinence, defined as point prevalence abstinence measured at two or more follow-ups between which smoking is allowed. 22 Abstinence is often biochemically verified by measurement of carbon monoxide (CO). However, CO is eliminated from the body in around 24 hours;15 therefore, abstinence cannot be verified for longer periods than this. The Russell Standard recommends that abstinence should be defined as ‘a self-report of smoking not more than five cigarettes from the start of the abstinence period, supported by a negative biochemical test at the final follow-up’. 21

Current service provision

Stop smoking clinics in the UK typically include the option to attend specialist one-to-one sessions with a trained stop smoking advisor, group sessions or drop-in sessions. 23 Clinics generally involve some form of assessment of current smoking behaviour and willingness to quit, including CO monitoring, prescription of some form of pharmacotherapy if desired (NRT, bupropion hydrochloride or varenicline) and behavioural support focused on managing withdrawal symptoms and preventing relapse (including preparing to quit, setting a quit date and making plans for situations where the client may be tempted to smoke). 24

Success rates for the NHS smoking cessation treatments

Eight hundred thousand smokers each year attempt cessation through the NHS stop smoking services. 25 In any given quit attempt 0.5% of smokers attempt cessation using varenicline with specialist individual behavioural support through NHS stop smoking clinics, 0.2% use varenicline with specialist group behavioural support, 0.1% use varenicline with specialist drop-in behavioural support and 2.8% obtain a prescription for varenicline in NHS settings (e.g. primary care, hospital). 23 The estimated 52-week continuous abstinence rates for NHS specialist individual behavioural support clinics are 15% when combined with NRT monotherapy, 20% with NRT combination therapy, 17% with bupropion hydrochloride and 24% with varenicline. The estimated 52-week continuous abstinence rates for NHS specialist group behavioural support clinics are 20% when combined with NRT monotherapy, 26% with NRT combination therapy, 23% with hydrochloride and 31% with varenicline. The estimated 52-week continuous abstinence rates for NHS specialist drop-in behavioural support clinics are 11% when combined with NRT monotherapy, 15% with NRT combination therapy, 13% with bupropion hydrochloride and 19% with varenicline. The estimated 52-week continuous abstinence rates for brief interventions in NHS settings (e.g. primary care, hospital) are 7% for NRT monotherapy, 10% for NRT combination therapy, 8% for bupropion hydrochloride and 12% for varenicline. 23

A recent Cochrane review of nicotinic receptor partial agonists as aids to smoking cessation showed a modest efficacy for cytisine over placebo in helping people to stop smoking, although the study reports low absolute quit rates for these trials. 15 The authors report a twofold increase in quit rates for varenicline over placebo. These analyses, comparing each drug with placebo, found no difference in their efficacy. The authors highlight that trials have now been conducted in real-world settings, for example in smokers with underlying diseases or medical conditions who might under ordinary circumstances be excluded from clinical trials, and report that the findings remain stable in these populations. A recent review of the efficacy and safety of cytisine found it to be an effective treatment for smoking cessation. 26 The review highlights the low cost of cytisine and suggests that licensing of this drug may therefore be warranted because of its potential public health benefit. No head-to-head trials between varenicline and cytisine were identified in either review and, to date, no indirect comparisons of the two drugs have been conducted in the absence of such trials.

Identification of studies

A good-quality recent Cochrane review evaluating the clinical effectiveness and safety profile of both cytisine and varenicline was identified. 15 This Cochrane review will be referred to subsequently as Cahill et al. 15 The current review aimed to use the data from Cahill et al. 15 to inform clinical effectiveness and cost-effectiveness analyses. An update of this search was conducted, which aimed to identify any recent studies evaluating the clinical effectiveness of varenicline or cytisine. The aim of the current review was to compare the clinical effectiveness and cost-effectiveness of varenicline and cytisine, but in the likely absence of head-to-head trials between cytisine and varenicline, any comparators were considered that would enable an indirect comparison, for example placebo, NRT and bupropion.

The search was conducted in January 2013 and the search strategy from Cahill et al. 15 was rerun for trials and systematic reviews in the period December 2011 to January 2013. Although dianicline was included in the Cahill et al. 15 searches, development of the drug has been discontinued and, therefore, will not be included in the comparisons for this report. This term was therefore excluded from the search. Additionally, a search was run for the terms Champix or Chantix (brand names for varenicline) with no date restrictions, in order to identify earlier trials using brand rather than generic names, as these terms were not included in Cahill et al. 15

The search was also rerun with a cost-effectiveness filter with no date restriction for cost-effectiveness literature. The purpose of the cost-effectiveness search was to obtain data to inform the model and no systematic review of this literature was conducted. Cost-effectiveness methods and analyses are reported in Chapter 4.

Searches were conducted by an information specialist (AC). Examples of each of the search strategies in MEDLINE are provided in Appendix 1.

The following electronic databases were searched for published and unpublished research evidence, from December 2011 to January 2013 for the efficacy searches and from database inception for the cost-effectiveness searches:

• MEDLINE (Ovid) 1950–

• EMBASE (Ovid) 1980–

• Cumulative Index to Nursing and Allied Health Literature (CINAHL; EBSCOhost) 1982–

• The Cochrane Library including the Cochrane Systematic Reviews Database, Cochrane Controlled Trials Register, Database of Abstracts of Reviews of Effects (DARE), Health Technology Assessment (HTA) database and NHS Economic Evaluation Database (NHS EED) 1991–

• Biological Abstracts (via Web of Science) 1969–

• Science Citation Index (via Web of Science) 1900–

• Social Science Citation Index (via Web of Science) 1956–

• EconLit 1961–

• Conference Proceedings Citation Index–Science (CPCI–S) (via Web of Science) 1990–

• UK Clinical Trials Research Network (UKCRN) and the National Research Register archive (NRR)

• Current Controlled Trials 1898–

• ClinicalTrials.gov 1998–.

All citations were imported into Reference Manager (Thomson ResearchSoft, San Francisco, CA, USA) software and duplicates deleted.

Inclusion and exclusion criteria

• Study design: RCTs.

• Intervention or comparators: cytisine, in any formulation; varenicline, in any formulation. In the likely absence of data from head-to-head studies of cytisine compared with varenicline, any other comparators (e.g. placebo, NRT, bupropion) were considered that would allow an indirect comparison.

• Population: smokers.

• The primary outcome was abstinence at a minimum 6 months’ follow-up.

Based on the above inclusion/exclusion criteria, study selection was conducted by two reviewers (JL and EEH). In the first instance titles and abstracts were examined for inclusion. Both reviewers independently screened all retrieved citations. Discrepancies between reviewers were discussed and any remaining disagreement resulted in retrieval of the full paper for further consideration. The full manuscripts of citations judged to be potentially relevant were retrieved and further assessed for inclusion. Any remaining discrepancies between reviewers at full-paper stage were discussed and, if no agreement could be reached, were resolved by referring to the review’s clinical experts. A table of studies excluded at full-paper stage with reasons for exclusion is presented in Appendix 3.

Data extraction

Data were extracted without blinding to either authors or journal. Data from studies included in the Cahill review15 were extracted directly from the review by one reviewer (JL or EEH). Data from studies included following the updated search were extracted by one reviewer (JL) and checked by a second (EEH). Any data for doses not reported in Cahill et al. 15 were extracted directly from the original papers. Efficacy data for newly included studies were calculated using the same method reported in Cahill et al. 15 – based on the numbers of people randomised to an intervention and excluding any deaths or untraceable moves in accordance with the Russell Standard. 31 Drop-outs or those patients lost to follow-up are treated as continuing smokers. It is beyond the scope of this short report to conduct a systematic review of adverse events for varenicline and cytisine, taking into account long-term observational studies. Therefore, this assessment will adopt the same approach to adverse events as the Cahill et al. 15 review, extracting this information from RCTs retrieved through an update of their efficacy search. Adverse events data were extracted on the basis of number of participants who had taken at least one dose of treatment. Data from the strictest reported measurement of smoking cessation were extracted for use in the analyses, i.e. 7-day PPA or CAR. Where studies reported both CAR and 7-day PPA, CAR data were extracted in preference to 7-day PPA22 and only data from studies measuring CAR were used in the network meta-analysis for efficacy. Data from both types of studies were used for adverse events analyses. Efficacy data from studies that reported only PPA were extracted with the purpose of conducting a sensitivity analysis for significant differences in results by method of outcome measurement. Adverse events data were extracted for the four most common adverse events, as identified in the Cahill review, and these were abnormal dreams, nausea, headache and insomnia. Data for SAEs were also extracted.

Critical appraisal strategy

Critical appraisals of the quality of studies, retrieved by the updated search, followed the same format as reported by Cahill,15 using the Cochrane risk of bias tool (Higgins et al). 32 Studies were critically appraised by JL or EEH and checked by the second reviewer. Discrepancies were resolved by discussion between reviewers. Quality assessments aimed to evaluate the risk of bias in the current evidence base for the clinical effectiveness of both varenicline and cytisine.

Methods of data synthesis

The continuous abstinence data and adverse events data including abnormal dreams, headache, insomnia, nausea and SAEs were synthesised using a network meta-analysis. The analysis combines evidence across studies in which there is at least one treatment in a study that is common to at least one other study.

A random (treatment) effects model was used to allow for heterogeneity in treatment effects between studies. The model assumed a fixed (i.e. unconstrained) baseline effect in each study so that treatment effects were estimated within study and combined across studies. All analyses were implemented in WinBUGS (MRC Biostatistics Unit, Cambridge, UK). 33

The continuous abstinence, abnormal dreams, headache, insomnia, nausea and SAEs data were modelled using a complementary log-log link function to allow for variation in duration of follow-up between studies (see Appendix 4). This assumes that the times to event follow an exponential distribution and, hence, that the treatment effect is constant over time. Although these are strong assumptions, they are expected to be better than assuming there is no effect of duration of follow-up.

Results of the network meta-analyses are reported in terms of the hazard ratios (HRs) and 95% credible intervals (CrIs) relative to the baseline intervention (i.e. placebo). The posterior medians of the between-study standard deviations (SDs) together with their 95% CrIs are also presented.

Convergence of the models to their posterior distributions was assessed using the Gelman–Rubin convergence statistic. 34 Convergence occurred after 50,000 iterations for all outcome measures except for SAEs, which converged after 60,000 iterations. There was some suggestion of moderate autocorrelation between successive iterations of the Markov chains; to compensate for this the Markov chains were thinned every five iterations for continuous abstinence, nausea and SAEs, and every 10 iterations for abnormal dreams, headache and insomnia. Parameter estimates were based on 10,000 iterations of the Markov chains for continuous abstinence and nausea; 5000 iterations of the Markov chains for abnormal dreams, headache and insomnia and 8000 iterations of the Markov chains for SAEs to ensure that the Monte Carlo error was < 5% of the posterior SD. Although fewer samples would have been sufficient for estimating parameters for continuous abstinence, 10,000 samples were taken for the purpose of the expected value of sample information (EVSI).

The total residual deviance was used to assess formally whether or not the statistical model provided a reasonable representation of the sample data. The total residual deviance is the mean of the deviance under the current model minus the deviance for the saturated model, so that each data point should contribute about one to the deviance. 35

To enable the estimation of intervention-specific CARs, as required for the economic model, a separate random-effects meta-analysis was conducted on the placebo intervention arms. Absolute estimates of CARs were generated for each intervention by projecting the estimates of treatment effect (i.e. the log-hazard ratio) from the network meta-analysis onto the baseline CAR.

Updated search

The updated bibliographic search for clinical effectiveness retrieved 476 papers. Figure 1 shows the results of this search. For the clinical effectiveness search, 32 full papers were retrieved after screening of titles and abstracts. After the reading of these full papers, a further 29 papers were excluded (reasons given in Appendix 3). Three papers were included from the updated search. 37–39 All three papers measured smoking cessation rates by PPA. The study reported in Cahill et al. 15 as Pfizer 201140 was identified in the updated search as now being a published paper. 41 There were no differences in reported data in the published paper. Data for efficacy and adverse events were extracted from all newly included studies.

FIGURE 1.

Study flow chart [adapted from Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 2009;6:e100009736].

Assessment of clinical effectiveness

Results

Continuous abstinence

A network meta-analysis was used to compare the hazard of having continuous abstinence when treating with nicotine patch, cytisine 1.5 mg six times daily, varenicline 0.3 mg q.d., varenicline 1.0 mg q.d., varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d. and bupropion hydrochloride 150 mg b.i.d. compared with placebo. A total of 16 studies comparing pairs, triplets or quintuplets of interventions provided information at various study durations. 16,17,43,46–55,57,58,60

Figure 2 presents the network of evidence. A summary of all the trials (data) included in the network meta-analysis for continuous abstinence is provided (see Appendix 5, Table 38).

FIGURE 2.

Network diagram of different interventions for continuous abstinence. The nodes represent the interventions. Lines between nodes indicate when interventions have been compared. Different thicknesses of the lines represent the number of times that each pair of interventions has been compared.

The network meta-analysis model fitted the data well, with a total residual deviance close to the total number of data points included in the analysis. The total residual deviance was 40.99, which compared favourably with the 39 data points being analysed.

There was evidence of mild heterogeneity in treatment effects between studies (between-study heterogeneity SD 0.20, 95% CrI 0.02 to 0.45). All interventions apart from varenicline 0.3 mg q.d. and varenicline 1.0 mg q.d. were associated with a statistically significant effect on having continuous abstinence at a conventional 5% significance level relative to placebo. Cytisine 1.5 mg produced the greatest effect (HR 4.27, 95% CrI 2.05 to 10.05) relative to placebo (Table 6; see also Appendix 5, Figure 13). Cytisine 1.5 mg was the intervention with the highest probability of being the most effective intervention (p = 0.87) (Table 7).

TABLE 6 - Continuous abstinence: estimates of treatment-specific effect relevant to placebo and estimates of between-study SD from the posterior distribution

See Table 1 for regime details.

Intervention	Hazard ratio (95% CrI)
Nicotine patch	1.89 (1.06 to 3.49)
Cytisine 1.5 mga	4.27 (2.05 to 10.05)
Varenicline 0.3 mg q.d.	1.58 (0.65 to 3.53)
Varenicline 1.0 mg q.d.	1.08 (0.40 to 2.63)
Varenicline 0.5 mg b.i.d.	2.16 (1.54 to 3.38)
Varenicline 1.0 mg b.i.d.	2.58 (2.16 to 3.15)
Bupropion hydrochloride 150 mg b.i.d.	1.59 (1.10 to 2.32)
Between-study SD	0.20 (0.02 to 0.45)

TABLE 7 - Probability of treatment rankings [p(j = b)]

See Table 1 for regime details.

Rank (b)	Treatment (j)
	Placebo	Nicotine patch	Cytisine 1.5 mga	Varenicline 0.3 mg q.d.	Varenicline 1.0 mg q.d.	Varenicline 0.5 mg b.i.d.	Varenicline 1.0 mg b.i.d.	Bupropion hydrochloride 150 mg b.i.d.
1	0.00	0.02	0.87	0.02	0.00	0.03	0.06	0.00
2	0.00	0.09	0.07	0.08	0.02	0.14	0.61	0.00
3	0.00	0.17	0.04	0.09	0.03	0.37	0.27	0.03
4	0.00	0.30	0.02	0.14	0.05	0.28	0.05	0.16
5	0.00	0.22	0.01	0.18	0.08	0.12	0.01	0.38
6	0.09	0.14	0.00	0.24	0.15	0.05	0.00	0.33
7	0.42	0.06	0.00	0.15	0.26	0.01	0.00	0.10
8	0.48	0.01	0.00	0.09	0.41	0.00	0.00	0.00

Since repeated 7-day PPA may be used as a proxy for continuous abstinence,22 a sensitivity analysis was conducted including studies used measurement such as continuous abstinence or repeated 7-day PPA. Both the estimates and 95% CrIs of all treatment effects were similar to the results from the analysis including only studies that used continuous abstinence (see Appendix 5). However, the goodness of the model fit suggested that some studies, in particular those by Steinberg et al. ,56 Oncken et al. ,52 Heydari et al. 39 and de Dios et al. ,38 may come from a different model. The measurement used by Steinberg et al. ,56 Heydari et al. 39 and de Dios et al. 38 was the repeated 7-day PPA. Hence, the treatment effects were estimated using studies having continuous abstinence as the measurement and these were used in the economic model.

Abnormal dreams

A network meta-analysis was used to compare the hazard of having abnormal dreams when treating with nicotine patch, varenicline 0.3 mg q.d., varenicline 1.0 mg q.d., varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d., bupropion hydrochloride 150 mg b.i.d. and varenicline 1.0 mg b.i.d. for 52 weeks or placebo. A total of 14 studies comparing pairs or triplets of interventions provided information at various study durations. 16,37,39,41,46,48,51–55,57,58,61 No data were available for cytisine for abnormal dreams.

Figure 3 presents the network of evidence. A summary of all the trials (data) included in the network meta-analysis for abnormal dreams is provided (see Appendix 5, Table 39).

FIGURE 3.

Network diagram of different interventions for abnormal dreams. The nodes represent the interventions. Lines between nodes indicate when interventions have been compared. Different thicknesses of the lines represent the number of times that each pair of interventions has been compared.

The network meta-analysis model fitted the data reasonably well, with a total residual deviance close to the total number of data points included in the analysis. The total residual deviance was 37.98, which was slightly less than the 39 data points being analysed.

There was evidence of mild heterogeneity in treatment effects between studies (between-study heterogeneity SD 0.24, 95% CrI 0.02 to 0.74). Varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d. and varenicline 1.0 mg b.i.d. for 52 weeks were associated with statistically significant effects of having abnormal dreams at a conventional 5% level relative to placebo. Varenicline 1.0 mg b.i.d. for 52 weeks produced the greatest effect (HR 3.64, 95% CrI 1.44 to 10.01) relative to placebo (Table 8; see also Appendix 5, Figure 14). Varenicline 1.0 mg b.i.d. for 52 weeks was the intervention with the highest probability of being the most likely intervention to be associated with abnormal dreams (p = 0.54) (Table 9).

TABLE 8 - Abnormal dreams: estimates of treatment-specific effect relevant to placebo and estimates of between-study SD from the posterior distribution

Intervention	Hazard ratio (95% CrI)
Nicotine patch	2.01 (0.82 to 4.43)
Varenicline 0.3 mg q.d.	1.21 (0.46 to 2.99)
Varenicline 1.0 mg q.d.	1.75 (0.71 to 4.10)
Varenicline 0.5 mg b.i.d.	2.56 (1.23 to 5.91)
Varenicline 1.0 mg b.i.d.	3.29 (2.40 to 4.77)
Bupropion hydrochloride 150 mg b.i.d.	1.58 (0.96 to 2.70)
Varenicline 1.0 mg b.i.d. for 52 weeks	3.64 (1.44 to 10.01)
Between-study SD	0.24 (0.02 to 0.74)

TABLE 9 - Abnormal dreams: probability of treatment rankings [p(j = b)]

Rank (b)	Treatment (j)
	Placebo	Nicotine patch	Varenicline 0.3 mg q.d.	Varenicline 1.0 mg q.d.	Varenicline 0.5 mg b.i.d.	Varenicline 1.0 mg b.i.d.	Bupropion hydrochloride 150 mg b.i.d.	Varenicline 1.0 mg b.i.d. for 52 weeks
1	0.00	0.03	0.00	0.03	0.12	0.28	0.00	0.54
2	0.00	0.08	0.01	0.05	0.19	0.51	0.01	0.16
3	0.00	0.18	0.03	0.11	0.32	0.18	0.04	0.13
4	0.00	0.28	0.06	0.19	0.22	0.03	0.13	0.08
5	0.01	0.20	0.11	0.23	0.09	0.00	0.31	0.05
6	0.08	0.12	0.18	0.20	0.04	0.00	0.35	0.02
7	0.32	0.08	0.29	0.13	0.02	0.00	0.15	0.01
8	0.58	0.03	0.31	0.06	0.00	0.00	0.01	0.00

Headache

A network meta-analysis was used to compare the hazard of experiencing headaches when treating with nicotine patch, cytisine 1.5 mg, varenicline 0.3 mg q.d., varenicline 1.0 mg q.d., varenicline 0.25 mg b.i.d., varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d. and bupropion hydrochloride 150 mg b.i.d. compared with placebo. A total of 17 studies comparing pairs, triplets, quadruplets or quintuplets of interventions provided information at various study durations. 16,17,37,41,43,46–55,58,61

Figure 4 presents the network of evidence. A summary of all the trials (data) included in the network meta-analysis for headache is presented in Appendix 5, Table 40.

FIGURE 4.

Network diagram of different interventions for hazard of experiencing headaches. The nodes represent the interventions. Lines between nodes indicate when interventions have been compared. Different thicknesses of the lines represent the number of times that each pair of interventions has been compared.

The network meta-analysis model did not fit the data very well. The total residual deviance was 49.95, which was higher than would be expected given the 42 data points being analysed. In particular, the model did not fit the Smith et al. 55 and Nakamura et al. 49 studies particularly well. For Smith et al. 55 the model predicted six events in the placebo arm, which is twice as much as the number of events reported. For Nakamura et al. ,49 the model predicted nine events in the placebo arm, which is more than double the number of events reported in the study, which was only four. There was no obvious explanation in terms of the characteristics of the two studies, although we did not attempt a meta-regression because of limited available data.

There was evidence of mild heterogeneity in treatment effects between studies (between-study heterogeneity SD 0.19, 95% CrI 0.01 to 0.50). Varenicline 1.0 mg b.i.d. was the only intervention associated with a statistically significant increase in the hazard of experiencing headaches at a conventional 5% significance level relative to placebo (HR 1.23, 95% CrI 1.01 to 1.55). Varenicline 0.25 mg b.i.d. produced the greatest effect (HR 1.58, 95% CrI 0.78 to 3.47) relative to placebo (Table 10; see also Appendix 5, Figure 15), although the treatment effect was not statistically significant at a conventional 5% significance level. Varenicline 0.25 mg b.i.d. was the intervention with the highest probability of being associated with headaches (p = 0.46) (Table 11).

TABLE 10 - Headache: estimates of treatment-specific effect relevant to placebo and estimates of between-study SD from the posterior distribution

See Table 1 for regime details.

Intervention	Hazard ratio (95% CrI)
Nicotine patch	0.59 (0.30 to 1.16)
Cytisine 1.5 mga	0.85 (0.31 to 2.54)
Varenicline 0.3 mg q.d.	1.06 (0.61 to 1.92)
Varenicline 1.0 mg q.d.	1.08 (0.61 to 1.90)
Varenicline 0.25 mg b.i.d.	1.58 (0.78 to 3.47)
Varenicline 0.5 mg b.i.d.	1.48 (0.97 to 2.50)
Varenicline 1.0 mg b.i.d.	1.23 (1.01 to 1.55)
Bupropion hydrochloride 150 mg b.i.d.	1.03 (0.76 to 1.48)
Between-study SD	0.19 (0.01 to 0.50)

TABLE 11 - Headache: probability of treatment rankings [p(j = b)]

Rank (b)	Treatment (j)
	Placebo	Nicotine patch	Cytisine 1.5 mg	Varenicline 0.3 mg q.d.	Varenicline 1.0 mg q.d.	Varenicline 0.25 mg b.i.d.	Varenicline 0.5 mg b.i.d.	Varenicline 1.0 mg b.i.d.	Bupropion hydrochloride 150 mg b.i.d.
1	0.00	0.00	0.10	0.06	0.05	0.46	0.30	0.02	0.01
2	0.00	0.00	0.06	0.09	0.10	0.21	0.37	0.12	0.03
3	0.01	0.00	0.06	0.11	0.12	0.10	0.16	0.36	0.07
4	0.09	0.01	0.06	0.14	0.14	0.06	0.08	0.29	0.14
5	0.21	0.02	0.06	0.13	0.14	0.05	0.04	0.14	0.21
6	0.30	0.02	0.06	0.14	0.13	0.04	0.02	0.05	0.23
7	0.27	0.05	0.08	0.16	0.17	0.04	0.01	0.01	0.20
8	0.10	0.24	0.26	0.14	0.12	0.03	0.01	0.00	0.10
9	0.01	0.66	0.25	0.03	0.03	0.01	0.00	0.00	0.01

Nicotine patch and cytisine 1.5 mg were less likely than placebo to be associated with headache, although the effects were not statistically significant at a conventional 5% significance level. The HR for nicotine patch compared with placebo was 0.59 (95% CrI 0.30 to 1.16) and for cytisine 1.5 mg compared with placebo was 0.85 (95% CrI 0.31 to 2.54).

Insomnia

A network meta-analysis was used to compare the hazard of experiencing insomnia when treating with nicotine patch, varenicline 0.3 mg q.d., varenicline 1.0 mg q.d., varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d., bupropion hydrochloride 150 mg b.i.d. and varenicline 1.0 mg b.i.d. for 52 weeks. A total of 16 studies comparing pairs, triplets or quintuplets of interventions provided information at various study durations. 16,41,46–48,50–55,57–61

Figure 5 presents the network of evidence. A summary of all the trials (data) included in the network meta-analysis for insomnia is presented in Appendix 5, Table 41.

FIGURE 5.

Network diagram of different interventions for insomnia. The nodes represent the interventions. Lines between nodes indicate when interventions have been compared. Different thicknesses of the lines represent the number of times that each pair of interventions has been compared.

The network meta-analysis model fitted the data reasonably well, with a total residual deviance close to the total number of data points included in the analysis. The total residual deviance was 36.91, which was slightly less than the 38 data points being analysed.

There was evidence of mild heterogeneity in treatment effects between studies (between-study heterogeneity SD 0.15, 95% CrI 0.00 to 0.40). Varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d., bupropion hydrochloride 150 mg b.i.d. and varenicline 1.0 mg b.i.d. for 52 weeks were associated with a statistically significant increase in the hazard of experiencing insomnia at a conventional 5% significance level relative to placebo. Bupropion hydrochloride 150 mg b.i.d. produced the greatest effect (HR 2.27 95% CrI 1.70 to 3.05) relative to placebo (Table 12; see also Appendix 5, Figure 16). Bupropion hydrochloride 150 mg b.i.d. was the intervention with the highest probability of being associated with insomnia (p = 0.45) (Table 13).

TABLE 12 - Insomnia: estimates of treatment-specific effect relevant to placebo and estimates of between-study SD from the posterior distribution

Intervention	Hazard ratio (95% CrI)
Nicotine patch	1.36 (0.79 to 2.20)
Varenicline 0.3 mg q.d.	0.84 (0.48 to 1.47)
Varenicline 1.0 mg q.d.	1.22 (0.72 to 2.02)
Varenicline 0.5 mg b.i.d.	1.76 (1.09 to 3.03)
Varenicline 1.0 mg b.i.d.	1.68 (1.40 to 2.07)
Bupropion hydrochloride 150 mg b.i.d.	2.27 (1.70 to 3.05)
Varenicline 1.0 mg b.i.d. for 52 weeks	2.16 (1.03 to 4.79)
Between-study SD	0.15 (0.00 to 0.40)

TABLE 13 - Insomnia: probability of treatment rankings [p(j = b)]

Rank (b)	Treatment (j)
	Placebo	Nicotine patch	Varenicline 0.3 mg q.d.	Varenicline 1.0 mg q.d.	Varenicline 0.5 mg b.i.d.	Varenicline 1.0 mg b.i.d.	Bupropion hydrochloride 150 mg b.i.d.	Varenicline 1.0 mg b.i.d. for 52 weeks
1	0.00	0.01	0.00	0.00	0.10	0.00	0.45	0.43
2	0.00	0.04	0.00	0.02	0.20	0.09	0.44	0.21
3	0.00	0.08	0.00	0.04	0.29	0.37	0.08	0.12
4	0.00	0.14	0.01	0.09	0.21	0.43	0.02	0.10
5	0.03	0.37	0.04	0.24	0.14	0.10	0.00	0.08
6	0.24	0.21	0.09	0.36	0.05	0.01	0.00	0.03
7	0.53	0.09	0.17	0.18	0.01	0.00	0.00	0.01
8	0.19	0.05	0.68	0.07	0.00	0.00	0.00	0.01

Varenicline 0.3 mg q.d. was less likely than placebo to be associated with insomnia, although the effect was not statistically significant at a conventional 5% significance level (HR 0.84, 95% CrI 0.48 to 1.47).

Nausea

A network meta-analysis was used to compare the hazard of experiencing nausea when treating with nicotine patch, cytisine 1.5 mg, varenicline 0.3 mg q.d., varenicline 1.0 mg q.d., varenicline 0.25 mg b.i.d., varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d., bupropion hydrochloride 150 mg b.i.d. and varenicline 1.0 mg b.i.d. for 52 weeks. A total of 22 studies comparing pairs, triplets, quadruplets or quintuplets of interventions provided information at various study durations. 16,17,37,39,41,43,46–61

Figure 6 presents the network of evidence. A summary of all the trials (data) included in the network meta-analysis for nausea is presented in Appendix 5, Table 42.

FIGURE 6.

Network diagram of different interventions for hazard of experiencing nausea. The nodes represent the interventions. Lines between nodes indicate when interventions have been compared. Different thicknesses of the lines represent the number of times that each pair of interventions has been compared.

The network meta-analysis model fitted the data reasonably well, with a total residual deviance close to the total number of data points included in the analysis. The total residual deviance was 55.20, which compared favourably with the 53 non-zero data points being analysed.

There was evidence of mild heterogeneity in treatment effects between studies (between-study heterogeneity SD 0.08, 95% CrI 0.00 to 0.29). Varenicline 1.0 mg q.d., varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d., bupropion hydrochloride 150 mg b.i.d. and varenicline 1.0 mg b.i.d. for 52 weeks were more likely than placebo to be associated with nausea. Varenicline 1.0 mg q.d., varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d. and varenicline 1.0 mg b.i.d. for 52 weeks were associated with a statistically significant increase in the hazard of experiencing nausea at a conventional 5% significance level relative to placebo. Varenicline 1.0 mg b.i.d. for 52 weeks produced the greatest effect (HR 6.20, 95% CrI 3.30 to 13.61) relative to placebo (Table 14; see also Appendix 5, Figure 17). Varenicline 1.0 mg b.i.d. for 52 weeks was the intervention with the highest probability of being associated with nausea (p = 0.95) (Table 15).

TABLE 14 - Nausea: estimates of treatment-specific effect relevant to placebo and estimates of between-study SD from the posterior distribution

See Table 1 for regime details.

Intervention	Hazard ratio (95% CrI)
Nicotine patch	0.73 (0.44 to 1.10)
Cytisine 1.5 mga	0.78 (0.36 to 1.74)
Varenicline 0.3 mg q.d.	0.92 (0.55 to 1.52)
Varenicline 1.0 mg q.d.	2.26 (1.50 to 3.44)
Varenicline 0.25 mg b.i.d.	0.98 (0.48 to 1.82)
Varenicline 0.5 mg b.i.d.	1.48 (1.03 to 2.10)
Varenicline 1.0 mg b.i.d.	3.63 (3.10 to 4.27)
Bupropion hydrochloride 150 mg b.i.d.	1.12 (0.83 to 1.48)
Varenicline 1.0 mg b.i.d. for 52 weeks	6.20 (3.30 to 13.61)
Between-study SD	0.08 (0.00 to 0.29)

TABLE 15 - Nausea: probability of treatment rankings [p(j = b)]

Rank (b)	Treatment (j)
	Placebo	Nicotine patch	Cytisine 1.5 mg	Varenicline 0.3 mg q.d.	Varenicline 1.0 mg q.d.	Varenicline 0.25 mg b.i.d.	Varenicline 0.5 mg b.i.d.	Varenicline 1.0 mg b.i.d.	Bupropion hydrochloride 150 mg b.i.d.	Varenicline 1.0 mg b.i.d. for 52 weeks
1	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.05	0.00	0.95
2	0.00	0.00	0.00	0.00	0.01	0.00	0.00	0.94	0.00	0.05
3	0.00	0.00	0.01	0.00	0.92	0.01	0.04	0.01	0.00	0.00
4	0.00	0.00	0.05	0.04	0.06	0.09	0.70	0.00	0.06	0.00
5	0.08	0.01	0.07	0.11	0.01	0.20	0.18	0.00	0.34	0.00
6	0.25	0.02	0.07	0.15	0.00	0.12	0.05	0.00	0.32	0.00
7	0.38	0.06	0.08	0.16	0.00	0.13	0.02	0.00	0.17	0.00
8	0.22	0.16	0.15	0.21	0.00	0.16	0.01	0.00	0.08	0.00
9	0.06	0.34	0.22	0.20	0.00	0.16	0.00	0.00	0.03	0.00
10	0.00	0.40	0.35	0.12	0.00	0.12	0.00	0.00	0.01	0.00

Nicotine patch, cytisine 1.5 mg, varenicline 0.3 mg q.d. and varenicline 0.25 mg b.i.d. were less likely than placebo to be associated with nausea, although the effects were not statistically significant at a conventional 5% significance level.

Serious adverse events

A network meta-analysis was used to compare the hazard of experiencing SAEs when treating with nicotine patch, cytisine 1.5 mg, varenicline 0.3 mg q.d., varenicline 1.0 mg q.d., varenicline 0.25 mg b.i.d., varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d., bupropion hydrochloride 150 mg b.i.d. and varenicline 1.0 mg b.i.d. for 52 weeks or placebo. A total of 18 studies comparing pairs, triplets, quadruplets or quintuplets of interventions provided information at various study durations. 16,17,41,46–58,60,61

Figure 7 presents the network of evidence. A summary of all the trials (data) included in the network meta-analysis for SAEs is presented in Appendix 5, Table 43.

FIGURE 7.

Network diagram of different interventions for risk of experiencing SAEs. The nodes represent the interventions. Lines between nodes indicate when interventions have been compared. Different thicknesses of the lines represent the number of times that each pair of interventions has been compared.

The network meta-analysis model fitted the data reasonably well, with a total residual deviance close to the total number of data points included in the analysis. The total residual deviance was 46.67, which compared favourably with the 43 data points being analysed.

There was evidence of mild heterogeneity in treatment effects between studies (between-study heterogeneity SD 0.25, 95% CrI 0.01 to 0.86). Nicotine patch, cytisine 1.5 mg, varenicline 0.25 mg b.i.d., varenicline 0.5 mg b.i.d., varenicline 1.0 mg b.i.d., bupropion hydrochloride 150 mg b.i.d. and varenicline 1.0 mg b.i.d. for 52 weeks were associated with a higher risk of experiencing SAEs relative to placebo.

Varenicline 0.3 mg q.d. and varenicline 1.0 mg q.d. weeks were associated with a lower risk of experiencing SAEs relative to placebo, although none of the treatment effects was statistically significant at a conventional 5% significance level.

Nicotine patch produced the greatest effect (HR 5.33, 95% CrI 0.98 to 45.21) relative to placebo (Table 16; see also Appendix 5, Figure 18). Nicotine patch was the intervention with the highest probability of being associated with SAEs (p = 0.62) (Table 17).

TABLE 16 - SAEs: estimates of treatment-specific effect relevant to placebo and estimates of between-study SD from the posterior distribution

See Table 1 for regime details.

Intervention	Hazard ratio (95% CrI)
Nicotine patch	5.3 (0.98 to 45.21)
Cytisine 1.5 mga	1.27 (0.24 to 8.58)
Varenicline 0.3 mg q.d.	0.049 (0.00 to 1.34)
Varenicline 1.0 mg q.d.	0.078 (0.00 to 1.32)
Varenicline 0.25 mg b.i.d.	2.06 (0.46 to 7.83)
Varenicline 0.5 mg b.i.d.	1.02 (0.31 to 3.46)
Varenicline 1.0 mg b.i.d.	1.10 (0.74 to 1.65)
Bupropion hydrochloride 150 mg b.i.d.	1.29 (0.61 to 2.78)
Varenicline 1.0 mg b.i.d. for 52 weeks	2.61 (0.69 to 15.16)
Between-study SD	0.25 (0.01 to 0.86)

TABLE 17 - SAEs: probability of treatment rankings [p(j = b)]

See Table 1 for regime details.

Rank (b)	Treatment (j)
	Placebo	Nicotine patch	Cytisine 1.5 mga	Varenicline 0.3 mg q.d.	Varenicline 1.0 mg q.d.	Varenicline 0.25 mg b.i.d.	Varenicline 0.5 mg b.i.d.	Varenicline 1.0 mg b.i.d.	Bupropion hydrochloride 150 mg b.i.d.	Varenicline 1.0 mg b.i.d. for 52 weeks
1	0.00	0.62	0.06	0.00	0.00	0.10	0.01	0.00	0.01	0.20
2	0.00	0.19	0.14	0.00	0.00	0.23	0.04	0.00	0.05	0.33
3	0.01	0.10	0.14	0.01	0.01	0.26	0.08	0.04	0.14	0.20
4	0.05	0.04	0.12	0.01	0.01	0.15	0.14	0.14	0.22	0.11
5	0.15	0.02	0.09	0.01	0.01	0.07	0.13	0.25	0.22	0.06
6	0.29	0.01	0.06	0.01	0.01	0.06	0.11	0.28	0.13	0.03
7	0.33	0.01	0.09	0.01	0.01	0.06	0.16	0.19	0.11	0.03
8	0.15	0.01	0.23	0.04	0.03	0.06	0.26	0.09	0.10	0.03
9	0.01	0.00	0.05	0.36	0.48	0.01	0.06	0.01	0.01	0.00
10	0.00	0.00	0.01	0.55	0.44	0.00	0.01	0.00	0.00	0.00

Strengths and limitations

A strength of the network meta-analysis is that it enabled a comprehensive comparison of all interventions of interest taking into account all available information. Parameters were estimated using a Bayesian framework which allowed for uncertainty in the estimate of the between-study SD and the ability to make probabilistic statements about the rankings of the interventions.

The continuous abstinence and adverse events data were modelled using a complementary log-log link function to allow for variation in the duration of follow-up between studies. This model assumes that the times to event follow an exponential distribution and, hence, that the treatment effect is constant over time. While these are strong assumptions, they are expected to be better than assuming there is no effect of duration of follow-up on the observed event rates.

In practice, the exponential models fitted the data for each outcome measure reasonably well except for the headache data. There was no obvious reason for this in terms of study characteristics, although there were insufficient data to perform a meta-regression.

A sensitivity analysis of the continuous abstinence data, assuming that the repeated 7-day PPA measurement is equivalent to the continuous abstinence measurement, made negligible difference to the results and inferences. The results based on the analysis of the continuous abstinence data only were incorporated into the economic model because the goodness-of-fit of the model suggested that data from some of the studies using repeated 7-day PPA measurement may not belong to the same model.

The estimation of intervention-specific continuous abstinence rates as required for the economic model was calculated by projecting the estimates of treatment effect (i.e. the log-hazard ratio) from the network meta-analysis onto the absolute risk of continuous abstinence for the placebo group. The absolute risk of continuous abstinence for the placebo group was estimated from a separate random-effects meta-analysis of studies in which the control arm was the placebo intervention. The predictive distribution of the absolute risk in a new study was used to represent uncertainty to reflect heterogeneity between studies.

Methods

The conceptual model

The BENESCO model is a state transition model designed to capture important long-term outcomes of smoking cessation treatments. The BENESCO model has been used in numerous previous evaluations. 63–73

The model uses an annual cycle length and assumes that all smokers die at age 100 years, if death has not been simulated at an earlier age. A hypothetical cohort of 10,000 smokers enters the model, with each smoker assumed to make a single quit attempt, assisted by either varenicline or cytisine. The distribution of the cohort in terms of sex, age (three age categories are used – 18–34 years, 35–64 years and 65–100 years) and chronic smoking-related diseases [lung cancer, chronic obstructive pulmonary disease (COPD), coronary heart disease (CHD) and stroke] is assumed to be representative of smokers in England and Wales.

At the start of the model, every cohort member begins in the smoker state. At the end of the first year, a proportion of smokers successfully cease smoking and become quitters; this proportion is determined by the efficacy of the cessation aid treatment received. The model assumes that no further attempts to quit are made and that those who fail to quit remain smokers until death. However, there is a possibility that quitters may relapse and start smoking again in future years. Potential to relapse to smoking is incorporated into the model as a decreasing function of time since cessation and is independent of cessation treatment (varenicline or cytisine). For the four model cycles following cessation, cohort members are assigned recent quitter status and risk of relapse is highest. After four cycles without relapse, recent quitters attain long-run quitter status. The annual relapse rate is lower for long-run quitters than for recent quitters in the next five cycles and lower still in subsequent cycles, with this underlying relapse rate continuing for the duration of the model.

At the end of each year, the cohort is distributed into different smoking states (smoker, quitter, relapsed smoker) according to their current smoking state and relapse rates. Figure 8 details the possible transitions between smoking states.

FIGURE 8.

Transitions between smoking states.

Within these broad smoking states, cohort members are distributed between the following disease states: no current morbidity, lung cancer, COPD, CHD, stroke and asthma exacerbation. These health states were selected by the authors of the BENESCO model to correspond to the diseases accounting for the greatest morbidity, mortality and cost attributable to smoking. 65 The health states are mutually exclusive and death is an absorbing state. The probability of transition between disease states at the end of each cycle is dependent on current disease state, smoking status, age and sex, as these factors have been shown to be independent determinants of risk.

The model has categorised four of the health states as either acute (CHD and stroke) or chronic (COPD and lung cancer) conditions. Transitions within acute and chronic conditions are not allowed and, therefore, it is not possible for a cohort member to experience a CHD event following a stroke. Transitions from acute disease states to chronic disease states are possible, but not from chronic conditions to acute conditions. Asthma exacerbations were transient in nature and assumed to resolve within 1 year, and could only occur from the no current morbidity health state. Figure 9 illustrates possible transitions between health states in the model.

FIGURE 9.

Transitions between health states.

Each health state is associated with utility and cost values as detailed later. Therefore, cohort members accumulate costs and health outcomes each cycle until death. Adverse events are not considered within the BENESCO model framework.

Future costs and benefits were discounted at a rate of 3.5% per annum, and the perspective is that of the UK NHS for costs and health effects on the individual for outcomes, in line with National Institute for Health and Care Excellence (NICE) guidance. 74 Attention now turns to parameter values and distributions used in the probabilistic sensitivity analysis (PSA). As detailed in the protocol, many model inputs are derived from a previous manufacturer’s single technology appraisal (STA) report. 65 As this is a slight limitation, it is unlikely to affect the key conclusions from this report regarding the relative cost-effectiveness of varenicline and cytisine.

Results

Mean costs and mean treatment effects associated with each treatment

The results of the PSA are presented as the primary results of interest, as, unlike deterministic estimates, they take into account the distributions of input parameters and interaction between parameters and, therefore, are the more accurate estimates. Table 31 shows the primary results of the PSA analysis: per smoker total discounted costs, life-years (LYs) and quality-adjusted life-years (QALYs) for the two treatments. Cytisine is expected to be less costly and more effective than varenicline and, so, can be said to dominate varenicline based on the expected values.

TABLE 31 - Mean per smoker discounted total and incremental costs, LYs and QALYs from the economic analysis

Treatment	Costs		LYs		QALYs
	Total	Incremental	Total	Incremental	Total	Incremental
Cytisine	£4973	–£251	17.53	0.03	14.38	0.03
Varenicline	£5225	–	17.50	–	14.35	–

Figure 10 presents the cost-effectiveness acceptability curve114 for the two treatments. At any threshold of willingness to pay, up to £100,000 per QALY gained, cytisine was the optimal intervention in over 90% of the simulations within the PSA. This reflects the higher costs associated with varenicline treatment. As the willingness to pay increases, the probability that cytisine is preferable falls and the likelihood that varenicline is optimal rises. Given that cytisine was estimated to be the more effective treatment in 90% of simulations, the value for cytisine on the cost-effectiveness acceptability curve will asymptote at 90%.

FIGURE 10.

Cost-effectiveness acceptability curves for cytisine and varenicline.

Expected value of sample information analyses

Similarly to EVPPI, a novel method was undertaken that allowed the computational time required to be markedly reduced. Within this method the posterior expectation for the INB conditional on each new simulated trial data were approximated using approximate Bayesian computation. The method produced results comparable to those present by Ades et al. 113 in the pivotal methodological paper. A manuscript is current being prepared for submission to a peer-reviewed journal.

Within the EVSI calculation it was assumed that 3 million people would benefit from the increased information regarding the relative efficacies of cytisine and varenicline. The EVSI for a year-long trial that directly compares varenicline 1.0 mg b.i.d. with cytisine 1.5 mg was estimated using trial sample sizes ranging from 50 to 20,000 participants per arm, and at willingness-to-pay values of £20,000 and £30,000. The net value of the RCT was estimated assuming that to enrol a person in a RCT was £1000, as previously used by Stevenson et al. 120 This paper stated that ‘although in reality there will be fixed costs and some form of economies of scale to be exploited, this value appears a reasonable approximation to the costs of successfully funded bids in the United Kingdom’.

A graphical depiction of the results of the EVSI analyses are shown in Figure 11 (when a threshold of £20,000 per QALY is used) and Figure 12 (when a value of £30,000 per QALY is used).

FIGURE 11.

Results from the EVSI analyses assuming a willingness to pay of £20,000 per QALY.

FIGURE 12.

Results from the EVSI analyses assuming a willingness to pay of £30,000 per QALY.

In both analyses, conducting a RCT of varenicline compared with cytisine with 1000 smokers per arm appeared optimal, although the results were comparable to trials of 500 or 2000 smokers per arm. Sensitivity analyses (not shown) were conducted reducing the cost per person in the RCT to £500 to acknowledge the relatively small duration of the trial. Assuming a willingness to pay of £20,000 per QALY, the conclusions did not alter; at £30,000 per QALY a trial of 2000 in each arm was estimated to be slightly preferable to 1000 in each arm, although the results were very similar for these trial sizes.

Discussion on modelling aspects

Probabilistic sensitivity analysis outputs from the economic model suggest that cytisine for smoking cessation will produce greater mean LYs and QALYs, and lower mean lifetime costs than varenicline, which was previously considered to be the most cost-effective smoking cessation treatment strategy. At a willingness-to-pay threshold of £20,000 for an additional QALY, the probability that cytisine treatment is preferable to varenicline treatment is 0.95, and this probability does not fall below 0.9. Despite this, the value of further information on the relative effectiveness of the two strategies is high because of the very large numbers of smokers treated.

A key driver of the dominance of cytisine treatment over varenicline treatment in the economic analysis is the relative effectiveness of cytisine compared with varenicline, as shown in the univariate sensitivity analysis. In summary, the treatment which generates the greatest number of quitters will have the best long-term health outcomes as efficacious treatment also has the impact of reducing costs associated with longer-term conditions associated with smoking. If treatment costs were equal for varenicline and cytisine, the probability that cytisine is the optimal choice is 0.9 (at any willingness-to-pay value), reflecting the 0.9 probability that cytisine has the greater 1-year continuous cessation probability.

It was not possible to validate the economic model outputs against results in the STA report, as the number in the simulated cohort in the latter was not reported and, therefore, per smoker values are unknown. 89 Other previous applications of the BENESCO model have used non-UK populations and parameter inputs, making comparison of total LYs and QALYs difficult. 62–64,66–72,89 However, results across these studies and here have been similar, in that the intervention with the greatest clinical effectiveness (short-term cessation probability) has consistently had the greatest cost–utility.

The key limitation of the model structure used is the imposed assumption of no underlying quit rate, among failed quitters or relapsed smokers, which is likely to favour treatments with higher effectiveness. Other UK studies have modelled the cost–utility of competing smoking cessation strategies and incorporated an underlying quit rate. 92,95,121 In each of these studies, unlike here, the most efficacious strategy had the highest treatment cost, but like here, the strategy with greatest short-term clinical effectiveness was the optimal strategy, at a willingness-to-pay threshold of £20,000 per QALY gained. In these models the annual probabilities of relapse to smoking, smoking-related disease incidence and death have been assumed to be constant92,95,121 compared with the decreases related to time since cessation in the present model. Assuming a sharp fall in probabilities linked to unfavourable health outcomes, rather than a decline over time, is less realistic, but further biases results towards those with higher clinical effectiveness if the full benefits of smoking cessation are assumed instantly obtainable.

It is difficult to incorporate both an underlying quit rate and transition probabilities that vary with time since quit into a state transition model structure, without incorporating numerous tunnel states. Individual person-level models may be a better avenue for accurately quantifying the cost–utility of smoking cessation strategies in future. At least two such models have been built to date. 122,123 Given the resources provided for this project it was agreed in the protocol that construction of a more accurate model than the BENESCO model was not feasible.

The transition probabilities and some parameter inputs in the model were taken from the manufacturer’s submission for the NICE varenicline STA,65 and it is not known whether or not these data are the best currently available. From the results of the deterministic sensitivity analysis, model outputs are robust to parameter inputs other than relative effectiveness of the two treatments. Uncertainty around the probabilities of transition to disease states has not been explored, but if the relative risks of smoking-related disease incidence and mortality can be assumed to decrease after smoking cessation, cytisine for smoking cessation will represent a better use of the health-care budget than varenicline using average values given current information.

Clinical effectiveness findings

The systematic review of clinical effectiveness included 23 studies, comprising a total of 10,610 participants. 16,17,37–39,41,46–61 The review was an update of a previous Cochrane review by Cahill et al. 15 and the updated search added three new trials to the previous review. All of these trials were of varenicline. No new trials of cytisine were identified, so that only two high-quality RCTs of cytisine with outcome data after a minimum follow-up of 6 months have been conducted to date. 17,43

A network meta-analysis was used to allow a comprehensive synthesis and comparison between smoking cessation treatments including cytisine, varenicline, nicotine patch and bupropion hydrochloride. A random (treatment)-effects model was used incorporating a log-log link function to allow for variation in the duration of follow-up between studies.

Results showed that cytisine 1.5 mg produced the greatest effect on CAR relative to placebo (HR 4.21, 95% CrI 2.11 to 9.84). Cytisine 1.5 mg was the intervention with the highest probability of being the most effective intervention (probability = 0.86). As point prevalence abstinence is often used in smoking cessation trials as a proxy for continuous abstinence, a sensitivity analysis was conducted including studies using both continuous abstinence rates and 7-day point prevalence abstinence. The results of this analysis were similar to those using only CAR data. However, the goodness of fit of the model suggested that the data from some of the trials using point prevalence as an outcome measure may arise from a different model. Consequently, only treatment effects estimated from data obtained from the CAR studies were included in the economic model.

Previous recent systematic reviews have reported both cytisine and varenicline to be clinically effective aids to smoking cessation. Cahill et al. 15 reported both varenicline and cytisine to be clinically effective treatments for smoking cessation. Cytisine was reported to have modest clinical efficacy, although the authors noted low absolute quit rates. Cytisine has been licensed as a treatment for smoking cessation in a number of Eastern European countries for several decades. Despite this, only two trials of good quality with a minimum of 6 months’ follow-up that have evaluated the clinical efficacy of cytisine compared with placebo met the inclusion criteria for both the Cahill review15 and the current review. The Hajek review26 of cytisine identified more trials; however, these did not fit the inclusion criteria for the Cahill review15 or this current review. The most common reason for this was that the length of follow-up was too short. However, their analysis of including only the high-quality trials still showed cytisine to be a clinically effective smoking cessation aid. For varenicline, the Cahill review15 reported that participants treated with the standard dose of varenicline had a twofold increased chance of quitting than placebo. The authors note that varenicline has been shown to be clinically effective in trials in real-world settings and for participants who may ordinarily be excluded from clinical trials. These participants include patients awaiting surgery, or patients hospitalised with medical or psychiatric conditions. The trials identified in the updated search support this assertion, with varenicline showing clinical efficacy for pre-operative smokers37 and for light smokers in a Latino community setting. 38

Adverse events findings

Data for the four most common adverse events (abnormal dreams, headache, insomnia and nausea), as identified in the Cahill review,15 and SAEs were analysed. Standard-dose varenicline treatment was associated with significantly higher rates of headaches, insomnia and nausea than placebo; there was no significant difference in the rates of abnormal dreams. There were no significant differences in rates of headaches or nausea between cytisine and placebo; data were not identified for abnormal dreams or insomnia. However, these results must be interpreted with caution, as they are a result of data from RCTs only. Most of the trials reported in this review applied criteria that excluded individuals with underlying medical conditions such as a history of depression or cardiovascular illness. Such individuals may be more likely to develop SAEs than those with no history of these medical conditions. In addition, the follow-up period of many trials may not be sufficiently long to capture all relevant adverse events. Etter124 notes with caution the toxicity of the seeds of C. laborinum, from which cytisine is derived. The lethal dose in humans is not currently known; however, there is no current evidence to suggest that poisoning can occur from use of cytisine used for smoking cessation. A lack of systematic reviews of adverse events that include cohort studies means that, as is emphasised by Cahill et al.,15 conclusions regarding the safety profile of both varenicline and cytisine are currently limited.

Nevertheless, the current review and network meta-analysis found that there were no differences in SAEs, and that differences in mild, transient adverse events were not clinically significant. Overall, the safety evidence in the current review was weak and a full safety review was not undertaken. A full safety review of cohort studies for both varenicline and cytisine is needed. For example, a large cohort study found no evidence of an increased risk of self-harm, depression and suicidal thoughts relative to NRT or bupropion hydrochloride. 125

A number of systematic reviews and meta-analyses report on specific adverse events of varenicline. Previous reviews have reported mixed results when considering the association of varenicline with adverse events (e.g. Singh et al. ,29 found an increased risk of cardiovascular events, while Prochaska et al. 126 found no such link). Leung et al. 27 found an association between varenicline and an increased risk of gastrointestinal events. Tonstad et al. 28 found no significant association between varenicline and serious neuropsychiatric events; however, an association has been suggested for those taking varenicline who are currently experiencing depression. The FDA’s recently added warning to the product label of Chantix highlighted a small increased risk of certain cardiovascular events in patients with pre-existing cardiac conditions. This warning, coupled with its existing warnings, supports Cahill et al. ’s conclusion that a link between varenicline and certain adverse events cannot be ruled out. Concerns about the safety of varenicline have been raised, resulting in a series of warnings from the FDA. These have arisen through post-marketing reports of an increased risk of suicidal behaviour, serious cardiac events and gastrointestinal complaints. A full and detailed account of the current available evidence is presented in the review by Cahill et al. 15

Previous reviews have reported slightly more frequent adverse events among those taking cytisine than those taking placebo, which include weight gain, nausea and digestive problems, tachycardia and changes in blood pressure. 124 A review by Hajek et al. 26 found that many of the cytisine trials provided minimal support to participants, for example the drug was distributed by post. They highlight that more intensive support during attempted smoking cessation increases quit rates, and that this may be more relevant for participants taking cytisine, as its dosing regimen is complex. As both studies of cytisine identified in this review used the same standard dosing schedule, it has not been possible to identify advantages or disadvantages of different doses, in terms of both efficacy and adherence. In the event that cytisine were to be licensed in the UK, practitioners should consider the potential risk of adverse events when making treatment decisions for individual patients.

Cost-effectiveness findings

Probabilistic sensitivity analyses showed cytisine to produce greater expected mean LYs and QALYs, and lower mean expected lifetime costs than varenicline, and is therefore expected to dominate varenicline. The economic analysis is driven by the relative clinical effectiveness of cytisine and varenicline. The treatment that generates the greatest number of quitters will have the best long-term health outcomes, as smoking cessation produces reduced costs associated with longer-term conditions associated with smoking. Based on the currently available data there is a greater probability that cytisine is more efficacious than varenicline. However, this conclusion is uncertain, and owing to the very large numbers of smokers treated (around 800,000 receiving NHS treatment for smoking cessation each year), the value of further information on the relative effectiveness of the two treatments is high.

The EVPI was calculated as £12 per smoker assuming a willingness to pay of £20,000 per QALY gained and £21 per smoker assuming a willingness to pay of £20,000 per QALY gained. Although these are small EVPI values per person, the number of people affected by the decision is large – with a likely population of 3 million affected. The current economic analysis suggests EVPI values of £35M and £63M at a willingness-to-pay levels per QALY of £20,000 and £30,000 respectively. With a more conservative value of 1 million smokers affected, the values are in the region of £15M. As the EVPI is greater than the potential cost of a head-to-head trial, the second stage of the value of information analysis was necessary, i.e. calculating the EVPPI. The EVPPI of the HR of smoking cessation of cytisine compared with varenicline remained high (in excess of £33M) and, therefore, formal EVSI analyses were undertaken. This indicated that a direct head-to-head trial of cytisine and varenicline, with 1000 patients in each arm appeared an appropriate use of resources.

Recommendations for future research

It is recommended that a head-to-head trial of varenicline and cytisine is undertaken, with 1000 patients in each arm being an appropriate number.

Strengths and limitations of the review

A strength of this review was the quality of the trials included. No high-risk trials were included, with most trials judged to be at low risk of bias. Strict inclusion criteria of the trials meant that many trials excluded participants who may be more at risk of adverse events, for example those with underlying medical conditions. For this reason, the adverse events analyses may not give a comprehensive picture of adverse events associated with each treatment.

Use of a network meta-analysis allowed a comprehensive comparison of all interventions of interest, including a number of different dosing schedules. Only two studies matching the inclusion criteria that evaluated cytisine were identified,17,43 compared with 21 studies of varenicline. 16,37–39,41,46–61 The varenicline data include studies of its clinical effectiveness in a number of real-world settings, which allows their results to be generalised to wider populations. Cytisine has yet to be studied in subpopulations.

A strength of the economic modelling is that EVSI analyses were undertaken to quantify whether or not a RCT is justified and, if so, an appropriate number of smokers to recruit. The model constructed was based on the BENESCO model and the limitations of this model, primarily no underlying quit rate, are applicable here. Furthermore, the transition probabilities and some parameter inputs were taken from a manufacturer’s submission to NICE65 and it is not known whether or not these data are the best available. However, such limitations are unlikely to significantly bias the comparison of varenicline and cytisine.

It was not possible to obtain a cost for cytisine direct from the manufacturer and costs may vary between countries. The cost of cytisine may increase if the manufacturer is required to incur costs associated with fulfilling UK licensing requirements, although this does not change the fundamental conclusion that a head-to-head trial of cytisine compared with varenicline is needed.

A potential limitation is that the review and economic evaluation contained only two trials that examined cytisine. In addition, the safety evidence in the current review was weak and a full safety review was not undertaken. The dearth of robust evidence concerning cytisine further highlights the importance of a high-quality head-to-head trial.

Cahill search

1. (‘cytisine’ or ‘tabex’ or ‘varenicline’ or ‘nicotine receptor partial agonist’).tw.

Champix or Chantix search

1. (Champix or Chantix).tw.

Randomised controlled trials filter

1. randomized controlled trial.pt.

2. controlled clinical trial.pt.

3. randomized controlled trials/

4. random allocation/

5. double blind method/

6. single blind method/

7. clinical trial.pt.

8. exp Clinical Trial/

9. (clin$ adj25 trial$).ti,ab.

10. ((singl$ or doubl$ or trebl$ or tripl$) adj25 (blind$ or mask$)).ti,ab.

11. placebos/

12. placebos.ti,ab.

13. random.ti,ab.

14. research design/

15. or/1-14

Systematic reviews filter

1. Meta analysis/

2. Meta analys$.tw.

3. Metaanaly$.tw.

4. exp Literature review/

5. (systematic adj (review or overview)).tw.

6. or/1-5

7. Commentary.pt.

8. Letter.pt.

9. Editorial.pt.

10. Animals/

11. or/7-10

12. 6 not 11