A systematic review and cost-effectiveness analysis of specialist services and adrenaline auto-injectors in anaphylaxis

Research questions

This section addresses the four research questions:

1. In adults, young people and children who receive emergency treatment for suspected anaphylaxis, which people are at high risk of anaphylactic episodes? For which people would further anaphylactic episodes have significant impact? Which people can be identified as needing special consideration?

2. What are the effects of history-taking, including signs and symptoms, and physical examination in identifying the possible cause?

3. What are the effects of providing adrenaline auto-injectors, including by whom?

4. After assessment, when should referral take place?

Identification of studies

The evidence reviews used to develop the guideline recommendations were underpinned by systematic literature searches, following the methods described in ‘The guidelines manual’ (2009). 1 The aim of the systematic searches was to comprehensively identify the published evidence to answer the review questions developed by the Guideline Development Group (GDG) and Short Clinical Guidelines Technical Team.

The search strategies for the review questions were developed by the information specialist with advice from the systematic review team. Structured questions were developed using the PICO (population, intervention, comparison, outcome) model and translated into search strategies using subject heading and free-text terms. The strategies were run across a number of databases, with no date restrictions applied to the searches.

The NHS Economic Evaluation Database (NHS EED) was searched for economic evaluations. A search filter for economic evaluations was used on bibliographic databases. There were no date restrictions applied to the searches.

The searches were undertaken between 17 January and 17 March 2011.

Scoping searches

Scoping searches were undertaken in January 2011 using the following websites and databases (listed in alphabetical order) shown in Table 1; browsing or simple search strategies were used. The search results were used to provide information for scope development and project planning.

Main searches

The following sources were searched for the topics presented in the sections below:

• Cochrane Database of Systematic Reviews (CDSR) (Wiley)

• Cochrane Central Register of Controlled Trials (CENTRAL) (Wiley)

• Database of Abstracts of Reviews of Effects (DARE) [Centre for Reviews and Dissemination (CRD)]

• Health Technology Assessment database (HTA) (CRD)

• NHS Economic Evaluation Database (NHS EED) (CRD)

• Science Citation Index (SCI) (Web of Science)

• Cumulative Index to Nursing and Allied Health Literature (CINAHL) (EBSCOhost)

• EMBASE (OvidSP)

• MEDLINE (OvidSP)

• MEDLINE In-Process & Other Non-Indexed Citations and MEDLINE Daily Update (OvidSP).

Identified references were downloaded in EndNote X4 software (Thomas Reuters, CA, USA) for further assessment and handling.

Inclusion and exclusion criteria

Data abstraction strategy

Included studies were summarised using evidence tables for prognostic studies (see appendix K3 of the NICE guidelines manual). 1 These tables can be found in Chapter 3 (see Results). Extraction of one reviewer was checked by another. Furthermore, Grading of Recommendations Assessment, Development and Evaluation (GRADE) summary of findings tables2 were prepared. Any disagreement was discussed with a third reviewer.

Critical appraisal strategy

Quality and strength of evidence of included studies was assessed using the methodology checklist for prognostic studies (see appendix J of the NICE guidelines manual). 1 These tables can be found in Appendix 2. In addition, quality was assessed using the GRADE methodology. 2

Methods of data synthesis

Not applicable.

Quantity and quality of research available

The searches of electronic searches yielded in 11,058 references. After screening of titles and abstracts, 10,951 references were excluded. The remaining 107 references were obtained and the full texts were screened. Five studies were included, with another 60 studies highlighted as possibly being relevant for the background and/or the cost-effectiveness analysis (CEA). A flow chart of the screening process is presented in Figure 1.

FIGURE 1.

Flow chart of study identification. T/A, title and abstract.

Assessment of clinical effectiveness

All five included studies were prospective observational studies reporting on risk of recurrence. 3–7 The studies, conducted in five countries (Australia, Germany, Italy, Spain and the USA), included 1725 patients overall.

The risk of bias using the NICE methodology checklist for prognostic studies1 was rated as low for three studies3,4,7 or medium (two studies)5,6 (Table 2). Two of the studies were published only as abstracts, which limited the amount of methodological details reported in these studies. 4,6 Overall, problems included unclear definition of recurrence (three studies),4,5,7 unclear patient selection (one study),3 insufficient details on role of funding source (one study)5 and missing details on included patients (one study). 6 The quality assessment is presented in Appendix 2.

Using the GRADE methodology,2 quality of evidence extracted from the included studies was rated as ‘very low’. It should be noted that using the GRADE approach quality of evidence from observational studies is initially rated as ‘low’. During further assessment, certain areas can lead to upgrading or downgrading of the quality. Application of the GRADE methodology to the included studies is shown below (see Table 4). Each row of the table reports on outcomes that are addressed by included studies, and highlights problems with any of the included studies in relation to each outcome. Footnotes identify specific threats to validity identified. As can be seen in the table, the main reasons for downgrading of included evidence were missing details on blinding, as well as size of studies, i.e. number of included participants. Readers should note that the different systems of identifying bias (NICE methodology checklist for prognostic studies vs GRADE) yield slightly different conclusions on the levels of threat to validity and therefore the quality of studies is described differently (see Tables 2 and 4).

All included studies reported the number of patients with recurrent anaphylactic episodes. Risk of recurrence was estimated to be between 30% and 42.8%. Overall, 497 of 1386 patients (35.9%) had a recurrent anaphylactic episode (see Table 4). One study suggested the rate of a third event to be 5.2%, with a higher risk of recurrence for women [relative risk (RR) 2.14, 95% confidence interval (CI) 1.17 to 3.9]. 4 In children of < 12 years, an overall recurrence of 27% was reported with food being the most frequent allergen (71%). 5 One larger study (432 patients) reported serious recurrences in 45 patients (10.4%) of whom 18 (40%) received adrenaline. 7 This study also presented findings on mortality and reported no deaths. 7 Another study presented results for sex, age, and race (see Table 4 for details). 4

Characteristics and findings of the included studies are presented below. Table 2 shows characteristics of the five included studies. The findings of these studies are presented in Tables 3 and 4.

No studies that fulfilled inclusion criteria for objectives 2–4 were identified.

Summary

Overall, five prospective observational studies reporting on risk of recurrence were included. 3–7 No studies were found for the questions on history-taking, adrenaline auto-injectors, and referral.

The included studies reported a recurrent anaphylactic episode for 497 of 1386 patients (35.9%), indicating that recurrent episodes are relatively common for anaphylactic patients (see Table 4). Findings of single studies suggested that women have a higher risk of recurrence. Around one-quarter (27%) of recurrences in children of < 12 years are caused by food.

Limitations and implications for future research

Although a comprehensive search was undertaken to identify relevant studies (see Quantity and quality of research available), only five studies were included (see Assessment of clinical effectiveness). All of these studies are observational studies with low or medium risk of bias assessing the risk of recurrence. The studies were relatively small (1725 patients) and assessed the risk of recurrence in various patient groups. This should be taken into account when formulating recommendation based on these studies.

No studies addressing any of the other clinical research questions in terms of history-taking, physical examination, provision of adrenaline auto-injectors or referral to specialist allergy clinics for those with anaphylaxis were identified.

Lack of good data to inform the effectiveness of anaphylaxis interventions means that RCTs or at least well-designed observational studies of the components of care in SSs should be conducted. Ideally, these should report findings based on large numbers of participants, if possible divided into relevant subgroups.

Krasnick et al. 1996

This study10 was designed to determine the efficacy of a specialist treatment in a University Allergy-Immunology Division using oral corticosteroids, antihistamines, and sympathomimetics for patients with idiopathic anaphylaxis. A total of 225 patients, diagnosed with idiopathic anaphylaxis and treated in one university hospital from 1971 to 1990, were retrospectively reviewed. The costs of both emergency care [physician fees, medications (intravenous corticosteroids, subcutaneous adrenaline and intramuscular diphenhydramine), pulse oximetry and cardiac monitoring] and hospitalisation (general medical floor hospital admission and intensive care unit admission with and without need of intubation and mechanical ventilation) were estimated on the basis of costs of services at Northwestern Memorial Hospital, Chicago, IL, USA, during the year 1995 (no details on unit costs were reported). Optimal discriminant analyses (ODAs) were used to determine whether or not the treatment protocol made a significant decrease in hospital costs for four subgroups of patients with idiopathic anaphylaxis. Significant decreases in emergency room visits occurred for three of the four subgroups of patients with idiopathic anaphylaxis. Significant decreases in the number of hospitalisations (p< 0.022) and intensive care unit admissions (p< 0.009) occurred for the patients with idiopathic anaphylaxis with generalised symptoms (two subgroups). Overall, there were 165 emergency room evaluations, 17 hospitalisations and 18 intensive care unit admissions (five admissions requiring intubation) before patients received the specialist treatment at a cost of US$225,000. There were 51 emergency room visits, three hospitalisations, and no intensive care unit admissions after patients received the SS at an estimated cost of US$40,260, producing a saving of US$184,740.

Shaker 2007

This study9 was designed to evaluate the cost-effectiveness of the prophylactic self-injectable adrenaline in mild childhood venom anaphylaxis from a societal perspective, although the only cost data included in the model were the market costs of an AI (US$50 per year). A Markov model evaluated two scenarios: one using an AI and another not using an AI for the treatment of venom anaphylaxis. The base case in each scenario was represented by a 6-year-old child. The year ‘2007’ was used as the baseline cost year and a discount rate of 3% was used for future costs and years. Literature sources were used to estimate mortality but the model assumed that all deaths would be prevented by the AI, regardless of time between trigger and death or success in use. One-way sensitivity analysis was performed of the following parameters: age, fatality rates of anaphylaxis and duration of use of AI after prescription.

The main findings were as follows: the incremental cost of prophylactic AI for mild childhood venom anaphylaxis was US$469,459 per year of life saved and US$6,882,470 per death prevented when evaluated at a 40-year time horizon. The sensitivity analysis revealed that the use of AI might become cost-effective at US$97,146 per life-year saved only if the annual fatality rate exceeded 2 per 100,000 persons at risk. The conclusion of this study was that the use of prophylactic AI to prevent fatalities in children with mild venom anaphylaxis is not cost-effective if the annual venom-associated fatality rate is < 2 per 100,000 persons at risk. The source of financial support of this study was not reported.

Desai and Carroll 2009

This study8 compared the costs and consequences of using an established device (probably the EpiPen) compared with a novel device (Intelliject) for treatment of a uniphasic anaphylactic reaction. The decision tree model evaluated the two scenarios from a health-payer perspective, but no information was provided on the baseline cost year, length of the time horizon and a discount rate used. The consequences included recovering without visiting the emergency department (ED), ED use and hospitalisations. The costs included in the model were costs of device use, ED use and hospitalisations. Data were obtained from literature, an online query tool for health care cost (HCUPnet) and clinical study data of the company that developed the new AI (Intelliject Inc., Richmond, VA, USA). One-way sensitivity analyses were conducted for patients' probabilities of carrying the device, using it correctly and of recovery and death after using the device incorrectly. The base-case results per 100 patients indicate that the new device would lead to more patients recovering without visiting the ED (57 vs 35), similar rates of ED use without hospitalisation (7) and fewer hospitalisations (2 vs 4). The results also indicated higher device costs (US$15,837 vs US$6291) and the same ED use costs (US$9375), but lower costs for hospitalisations (US$15,303 vs US$30,606), leading to lower total costs of the new device (US$40,515 vs US$46,272) (no statistical analyses on outcomes and costs were reported). Sensitivity analyses indicated that the new device would have lower total costs and lead to better consequences under most tested assumptions. The authors stated that the assumed price premium (not reported) of the new device provided lower total costs, and a higher recovery rate, as well as fewer hospitalisations.

Summary

None of these studies is useful in directly addressing the questions regarding SSs. However, the study by Krasnick et al. 10 does provide useful data in terms of the time to remission in idiopathic anaphylaxis and this is used in the de novo CEA described below. The study by Shaker9 does address the question regarding AI but the model is too simplistic, assuming that protection is guaranteed. Also, the population is those who have had a ‘mild’ reaction, which is not directly comparable with our definition of anaphylaxis, which is life-threatening. The study by Desai and Carroll 20098 was unfortunately too poorly reported to be useful.

Research questions

The analysis aimed to inform the following two questions:

1. What is the cost-effectiveness of referral to specialist allergy clinics for the diagnosis of anaphylaxis and for the prevention of future episodes and the reduction in morbidity and mortality from future episodes?

2. What is the cost-effectiveness of adrenaline auto-injectors for the treatment of anaphylaxis including the cost implications of training in the use of the auto-injectors?

Population

The population of interest is all patients with anaphylaxis (irrespective of the cause) who needed emergency treatment.

However, as the title ‘… suspected anaphylaxis’ suggests, there is a problem with diagnosis,11 which includes the definition of anaphylaxis. For example, Stewart and Ewan12 use the term ‘severe’ anaphylaxis and associate it with loss of consciousness or fainting. On this basis, they count 9 out of 55,000 emergency admissions. They then included 15 others to make 24 with ‘generalized reactions involving hypotension and/or respiratory difficulty’. The rate of referral to SSs was [through the general practitioner (GP)] 4 out of 24. In a study by El-Shanawany et al. 13 in Wales, the 77 cases identified in 6 months implied a rate out of a population of about 500,000 of 30.8 per 100,000 people-years. This was much higher than the 6.7 in the UK previously estimated by Sheikh et al. 14 However, a more recent study in the UK by Gonzalez-Perez et al. 15 produced an estimate of 34.38. The El-Shanawany study13 also revealed that the rate of referral to SSs was zero. Erlewyn-Lajeunesse et al. 11 selected cases of asthma, and urticarial and allergic reaction, as well as anaphylaxis according to physician diagnosis (in the absence of a gold standard) to test diagnostic criteria. This could imply that the suspected population is composed essentially of those suffering an allergic reaction albeit less severe as well as those with asthma. However, this Guideline definition rules this out by including: ‘… rapidly developing life-threatening airway, breathing and/or circulation problems …’ (http://www.nice.org.uk/nicemedia/live/12346/52120/52120.pdf, p. 1).

This fits with the definition used by Brown et al. 16 and, therefore, implies that, in the absence of a known trigger, other conditions that cause such life-threatening problems might be included in the population and might thus be referred to SSs. Indeed, in the absence of further information on the nature of those patients seen in a SS, an increase in referral, as is being considered, might actually increase the prevalence of patients not suffering from anaphylaxis.

However, in the latest UK guidelines for emergency treatment17 there is a recommendation that all of those who are suffering from anaphylaxis should be referred to a SS and there is no mention of any difficulty in diagnosing anaphylaxis. Indeed, the suggestion is that the diagnosis of anaphylaxis has been made in the vast majority of cases by discharge, other possible diagnoses having been ruled out. It is on this basis that the comparison is between SSs and standard care (SC), given definite diagnosis of anaphylaxis.

Comparators

The following combinations were considered in the model:

1. SC, no AI: SC plus no prescription of AIs where SC is defined as the absence of referral to a SS. It is not defined any further but is expected to consist of no more than GP consultation. AIs come in the form of either EpiPen or Anapen (Lincoln Medical Ltd, Salisbury, UK) [British National Formulary (BNF) no. 6118] and in several doses, recommended as 500, 300 and 150 g for adults, children aged 6–12 years and children aged < 6 years, respectively. 17 There is little variation in cost and so the cost of AI was based on the current cost of EpiPen of £26.45 (note that this value includes a price reduction that, at time of writing, had not been reflected in BNF 6118). Based on expert opinion, it was assumed that AIs should be replaced every 12 months, that adults require two AIs at any one time, and that children require four (because it is common practice to keep two at school and two at home).

2. SC plus AI: Injectors are recommended in the latest guidelines by the Resuscitation Council UK, to be prescribed for all patients with ‘… life-threatening features’ (p. 162). 17

3. SS no AI: All patients with suspected anaphylaxis are referred to a SS in accordance with the same guidelines: ‘All those who are suspected of having had an anaphylactic reaction should be referred to a specialist in allergy’ (p. 158). 17 The same guideline goes on to state: ‘All patients presenting with anaphylaxis should be referred to an allergy clinic to identify the cause, and thereby reduce the risk of future reactions and prepare the patient to manage future episodes themselves’ (p. 166). 17

4. SS plus AI: All patients both attend a SS and are prescribed AIs.

Framework

Given the lack of CEA evidence, a cost–utility analysis19 was undertaken with costs and quality-adjusted life-years (QALYs) considered over patients' lifetimes from a UK NHS perspective in accordance with NICE methods guidance. 20 Costs were in 2011 GB pounds (£) and an annual discount rate of 3.5% was used. Despite these treatments being for short-term use, a lifetime horizon is most appropriate to capture the full impact of treatment.

Model structure

A Markov model21 was constructed with mutually exclusive health states. The model simulated the course of events in a hypothetical cohort of persons with anaphylaxis who had been treated in an emergency care setting in the UK, aged ≥ 6 years. The model initially divides the cohort according to their relative incidence (referred to as ‘trigger probability’), into the four main causes of anaphylaxis: drugs (including medication, biologics, vaccines and anaesthetics), insects (stings), food and idiopathic origin11 (see Trigger probability). In the model, as time progresses, persons move from one state to another state according to a set of transition probabilities (see sections on model parameters below: Rate of recurrence, Mortality rate, Idiopathic treatment and Venom immunotherapy). The cycle length of the model was set to 3 months.

A cycle length of 3 months was chosen for convenience in modelling rates of recurrence as probability of a single recurrence event, as it can be shown that the longer the period the greater the error. Intuitively, this can be understood by considering that the longer the period then the greater the probability of more than one event occurring. For example, using the probability density function of the Poisson distribution, the probability of one event in 3 months with an annual rate of 0.28 (that of idiopathic cause, which is the highest of all causes used in the model) is 0.065. Although actually more than one event could occur in this time, the probability of two events is only 0.002 and that of more events is extremely small at only about 0.00005. Given the large amount of uncertainty in all parameter estimates, it was believed to be acceptably close and all other rates (for food, drug and insect causes) are no larger than about 0.12, which produces even less of an error. A shorter cycle length could have been used but there would still have been an error, although smaller, and this would have only increased model calculation time.

The health states are ‘death’, ‘at risk’ (of recurrence), ‘recurrence’ and, for idiopathic cause only, ‘remission’ (Figure 2). All members of the cohort begin in the ‘at-risk’ state and move in the next 3 months to the ‘recurrence’ state, with a probability according to the rate of recurrence (see explanation above), except if the cause was not known (i.e. idiopathic cause), where recurrence could occur only if remission had not.

FIGURE 2.

Diagram showing the health states and transitions between them; transitions can occur from any live state to the death state.

Those in the ‘at-risk’ or ‘remission’ states (idiopathic and insect only) were assumed to have general population age and sex-specific mortality. 22 Those in the ‘recurrence’ state had this mortality plus an additional probability. First, they were divided into those who used an AI or not, according to a probability for correct use (see Mortality). For both SS and SC plus AI, this probability was greater than zero, as all patients were assumed to be prescribed two injectors, each of which has a 6-month life. It was then assumed that all would continue be supplied and thus incur the cost until death, unless there was remission. In the ‘no AI comparators’ the probability was zero.

Under SC, unless there was remission (idiopathic and insect only), the recurrence rate was assumed to be constant. This was based on a lack of evidence to the contrary presented in any of the guidelines or the systematic review. The effectiveness of SSs, therefore, was partly mediated by a change in recurrence rate, which depends on trigger and is explained below.

Parameterisation

All parameter values were estimated using the best evidence available and according to best practice. 20,26 Unfortunately, the systematic review revealed only few and generally poor-quality studies on rates of recurrence by trigger and none comparing the effectiveness of SSs versus SC or the effectiveness of AIs (by any measure, e.g. reduction in rate of recurrence), which is confirmed by other recent reviews. 17,27–29 All other parameter estimates were chosen in order to be as UK relevant as possible, based on evidence that was either directly cited by recent UK or international guidelines or found by citation searching from these sources. This method was chosen in order to maximise the efficiency of obtaining high-quality relevant estimates.

In accordance with best practice and the principle that expert opinion proxies for the beliefs of the decision-maker, which, in effect, is NICE, expert opinion from the GDG was sought for all parameters. This was done either to provide an estimate in the absence of evidence or to provide an estimate based where possible on the presentation of some evidence. Practically, it involved asking during a GDG meeting for consensus as to the ‘most likely’, ‘lowest’ and ‘highest’ values of parameters. In order to facilitate this, where possible data from the literature were presented and in these cases, the source is given as ‘expert opinion and based on [data]’.

Because the latest NICE guidance20 demands probabilistic sensitivity analysis (PSA),30 parameters to estimate distributions were also estimated. Where the source was deemed to be good enough the sampling distributions of the probabilities (beta for binomial and Dirichlet for multinomial) were used. 31 In most other cases, a triangular distribution was used, based on expert opinion elicited as the lowest, most likely and highest values. In order to make the expected value the same as the most likely, all triangular distributions were symmetrical. The table containing the estimates and summarising the sources is split into several tables between sections in order to facilitate explanation, although it is also presented in full in Appendix 5.

Population characteristics

For the population in the model the following two (Table 6, parameters 1–2) assumptions have been made: 50% of the patients in the model are male and the starting age is 30 years. Although there is a little variation between studies as to what age defines someone as a child, we assume that it is < 17 years.

Rate of recurrence

For the model, the annual rate of recurrence of anaphylaxis caused by drugs after referral to SSs was based on expert opinions (Table 7, parameter 3). This rate will probably be very low based on the idea that it is very unlikely that the same drugs which caused the first anaphylactic reaction will be prescribed for the same patient again.

Parameter 4, the annual rate of recurrence of anaphylaxis due to food in SSs, was based on the data of two longitudinal prospective observational studies on the effectiveness of a management programme providing advice on nut avoidance and emergency medication in the UK. These two studies reported only three recurrences out of over 13,000 observation months, which is equivalent to a rate of about 0.003 per patient-year in adults and/or children who were diagnosed for peanut or tree nut. 33,34 However, these studies were not controlled trials. Furthermore, nut allergy patients are only a subgroup of all anaphylactic patients who will be referred after emergency treatment to specialist allergy care. Therefore, based on expert opinion, a more conservative estimate of 0.01 was chosen, although the minimum of 0 allowed for the possibility of very effective treatment.

Under SC, the most likely values for the annual rate of recurrence of anaphylaxis due to food or drugs and idiopathic (see Table 7, parameters 5 and 6) in current practice were based on the findings of a prospective study of 432 patients who were referred to a community-based specialist practice in Australia. 7 This was the only study from the systematic review that reported rates of recurrence by cause and the results had to be read off a graph (figure 1, p. 1037). The rate of anaphylaxis due to food was calculated by a combination of figures on incidence of anaphylaxis due to food and exercise-induced anaphylaxis (as these were not separated in the report) (Table 8).

The average across all foods was calculated by dividing the total annual number (sum across all r per year in the table) by the total number at risk (summing across all n in the table). The annual rate of recurrence of anaphylaxis due to insect sting (see Table 7, parameter 8) was based on the findings of the most recent (2010; 343 with anaphylaxis) UK study (Gonzalez-Perez et al. 15), as figures for the Australian population are not likely to resemble those for the UK population because the risk of experiencing an insect bite or sting is much higher in Australia than in the UK. Gonzalez-Perez et al. 15 reported a range from about 0.05 to 0.1 for any cause and so, given expert opinion, the higher rate was chosen as the most likely.

Based on expert opinion, 0.05 was chosen as the lowest value for all causes and the highest value followed from making the distributions symmetrical.

Trigger probability

As stated in literature, it is difficult to calculate the exact incidence rates of anaphylaxis as a result of difficulties with coding, diagnosis and reporting (Sampson et al. 35) and actual rates remain unclear.

In the model, the figures for probability of anaphylaxis due to insect sting and idiopathic anaphylaxis (Table 9, parameters 9 and 10) were estimated based on a 1-year study analysing The Health Improvement Network (THIN) database on 2.3 million patients (age 10–79 years) who had been enrolled with a GP in the UK for at least 1 year (Gonzalez-Perez et al. 15).

In the model, the probabilities that anaphylaxis was due to drug were specified for adults and children (see Table 9, parameter 11) using the figures of a retrospective study on emergency calls for allergic reactions within greater Manchester, also in a 1-year period, by the North West Ambulance Service in the UK (Capps et al. 36).

As can be seen, all probabilities were converted from multi- to binomial (essentially from marginal to conditional), which produces exactly the same result as if they had been treated as multinomial. This was done for ease of use in the model software (TreeAge Software, Inc., Williamstown, MA, USA, 2009). This means that the probability of idiopathic anaphylaxis is calculated first from r/n (103/343). Then, the probability of insect given not idiopathic is calculated given that idiopathic is ruled out from 46/240.

Next, the probability of drug given being not idiopathic or insect is calculated from 19/87 or 236/303, depending on whether child or adult. The probability of food given being not idiopathic or insect or drug is then simply 1 – probability of drug.

Table 9 gives a description of the model inputs, which imply the following marginal probabilities (r/all anaphylaxis = r/343): idiopathic 30.03%; insect 13.41%; food 44.21% (children) and 12.51% (adults); and drug 12.35% (children) and 44.05% (adults).

Mortality

Details of mortality from anaphylaxis are shown in Table 10.

The number of deaths due to anaphylaxis in the UK was estimated from the findings reported by the working group of the Resuscitation Council (Soar et al. 17). This was based partly on a set of studies using a register of deaths due to anaphylaxis compiled by Pumphrey,37 Pumphrey and Gowland,38 and Pumphrey and Roberts. 39 The number of anaphylaxis cases was estimated by figures for the period of 2009–10 from the Department of Health Hospital Episode Statistics (HES)32 (www.hesonline.nhs.uk). As already stated in the section on incidence rates, both of the reported figures are likely to be underestimates, as it is difficult to diagnose and correctly code anaphylaxis, so the mortality rate will probably not vary much from this. These figures imply an annual probability of dying of 20/3517 = 0.005687, i.e. about 0.5%.

Effect of adrenaline injectors on mortality

In order to estimate the effect of the AIs, it is necessary to ‘subtract’ out the effect of the injectors in order to estimate the probability of death with no AI. Put another way, the estimate of mortality shown above is lower than the mortality rate due to anaphylaxis in the presence of both the use of emergency services (referred to as ‘ambulance’) and AIs. Therefore, to estimate the effect of AI, we first need to estimate an ‘underlying’ rate plus ambulance effect only. Note that all of the calculations to estimate the probability of dying given no AI were performed in TreeAge from the parameters for death given emergency services and current AI use and parameters for time to death and ambulance response times shown below (Table 11).

Having calculated the probability with no AI, the effect of AIs can be applied, either with SC or with SSs. As will be explained below, the parameter in the model that estimates the effect of SC or SSs is the proportion of correct use, which would be expected to be higher with SSs than with SC.

In the absence of direct evidence as to how many deaths have actually been prevented by AIs, there are several steps in the calculation, which implies the need to use several parameters and, thus, the need to make some assumptions. However, it will be attempted to make these explicit and justified where possible. Also, as with all parameters, they were all subject to sensitivity analysis. Before the exposition, in order to improve clarity, the result of the calculations is first summarised by intervention (presence of AIs or ambulance service) in Table 11.

First, it was assumed that the effect of ambulance or AI depended on the time between exposure to trigger and death. Of course, with idiopathic this would be impossible, as there is no trigger. Indeed, the register by Pumphrey,37 and summarised by Soar et al. ,17 does contain these data for food, drug [oral and injected (although only ‘oral’ used, as ‘injected’ most likely to be administered in a health-care setting)] and insect. However, the total number of observations (111) is small. Therefore, time to death was estimated, making the assumption that the average across these three groups would apply to any cause including idiopathic. In practice, all of these times were times to first cardiac arrest, but, given that all individuals died, it is assumed that, in order to prevent death, adrenaline must be administered before this point. It was also assumed that the time to death observed in those who died was similar to that in those avoided by either the emergency services (referred to as ‘ambulance’) or AI.

Therefore, first, the proportions dying in each of the categories reported by Soar et al. 17 (2.1–4.5, 4.6–9.9, 10–20 and > 20 minutes) was estimated, as shown in Table 12.

‘Drug’ only included oral and not injected, on the basis that injected would have been administered by a health-care professional with little need for AI. These values, which were inputs in the model, imply the following proportions in each of the time categories shown in Table 13.

Using the probabilities of each trigger (excluding idiopathic) from the same sources as used above allows calculation of the probabilities of time to death for any trigger.

For example, about 62% of cases of patients with anaphylaxis from any cause would still be alive for up to 20 minutes, which means that death might be prevented by the arrival of an ambulance within that time. Therefore, to calculate the deaths that could be prevented by AI, one needs to first estimate the effect of the ambulance service. For example, if 100% of response times were < 4.5 minutes then there would be no need for AI but also there would be no deaths, which, of course, is not the case.

Therefore, to estimate the response times, the data from an audit of ambulance services were used;40 the proportions of responses in each of the reported categories (< 8, 8–18 and > 18 minutes) were estimated for each of the emergency categories, A (essentially life-threatening) and B, shown in Table 14.

The category ‘< 8 minutes’ is not reported for ‘B’ and so it was assumed to be zero. This is unlikely to be a problem, as the proportion of calls to anaphylaxis in category B is likely to be very small. Indeed the figures used were from Capps et al.,36 where there were < 10% in ‘B’ (referred to as ‘amber’ in that study). ‘Purple’ and ‘red’ were assumed to be equivalent to ‘A’. The category ‘< 8 minutes’ was assumed to correspond to 4.6–9.9 minutes, assuming that response time would never be < 4.6 minutes. The categories ‘8–18 minutes’, ‘10–20 minutes’, ‘> 20 minutes’ and ‘> 18 minutes’ were assumed to be equivalent. These r and n values, used as inputs in the model, imply the proportions shown in Table 15.

The proportions for any category are calculated by taking the average, weighted by the total numbers in each of the categories.

This, therefore, permitted the estimation of the proportion of all deaths that would not be saved by ambulance and thus could be saved only by correct and timely use of AI. For example, all of those with a time to death of < 4.6 minutes would not be prevented, whereas the proportion who would still die in the ‘10–20 minutes’ category would be only those for whom the ambulance response time was in the > 18-minute category. The formula is:

where Propnot_amb is the proportion of deaths that would occur as a result of anaphylaxis, which are not prevented by ambulance, which depends on the response time distribution so that:

where Propdie is the proportion who die in each time period, shown in Table 13, and Propresp is the proportion who respond within that time period, shown in Table 15. It can be seen that Propnot (2.1–4.5 minutes) = Propdie (2.1–4.5 minutes), because it is assumed that the ambulance never arrives that early. It can also be seen that a factor of 0.5 is used for some proportions; these are where the response time period is the same as the time period for death. Multiplying by 0.5 implies that only 50% of response times are less than time to die. This is an assumption given the lack of more precise data within each period.

From the data in the tables:

This means that about 13% of anaphylaxis deaths are not prevented by ambulance.

Now to calculate the effect of AIs it was assumed that all AIs, if used successfully, would be used within the ‘4.6–9.9 minutes’ category. This implies that all of those deaths not prevented by ambulance in < 4.6 minutes would still not be prevented. However, this does not imply that all deaths in the time window of 4.6 minutes or longer would be prevented, as this applies to only those who actually use the injector correctly; there is another parameter, which is the proportion who do this, which might be < 100%. Indeed, in the Capps et al. study,36 only about 44% (53/119) of those who eventually were given adrenaline (by ambulance or injector) received an adrenaline by AI. This means that the proportion of deaths not saved by either ambulance or AI can be estimated:

i.e. the proportion of deaths prevented by AI in each time period is the probability of correct use, Pcorrect (53/119) multiplied by the proportion that would not have been prevented by ambulance with a correction factor of 0.5 for the period 4.6–9.9 minutes only. Therefore, it can be calculated that:

i.e. about 9% of deaths are prevented by both ambulance and AI use. This is therefore the proportion of deaths from anaphylaxis (without any intervention) that would remain in the event of current ambulance service provision and current AI use. Therefore, to calculate the overall (no intervention) mortality rate, Pdeath, use:

where ‘Nintervention’ is the number of deaths with current service and AI use, which is 20 (see Table 10); ‘Pintervention’ is the proportion of deaths not saved, which was calculated to be 0.087852; and ‘Nno intervention’ is the number of deaths that would have occurred and ‘Nanaphylaxis’ is the number of cases of anaphylaxis, which is 3517 (see above).

Substituting (7) into (6) gives:

i.e. the probability of dying from anaphylaxis without any treatment would be about 6%, which would result in 3517 × 0.064729 = about 228 deaths per year.

Therefore, we can now fulfil the aim of this section and calculate the probability of dying with ambulance and no AI, which is:

‘PdeathnoAI’ is the probability of death used in the model for no injector use. This means that ‘Pdeath AI’ is the probability of death with correct AI use (recall that those deaths at < 4.6 minutes would not be prevented even with correct use), which can be calculated by assuming that the proportion given AI is 100%:

This is because the only deaths not prevented by 100% correct AI use are those that occur within 4.5 minutes. This means that, whereas current AI use (44%) saves about nine deaths per year, if AI use was 100% correct, there would be only about one death per year, saving an extra eight lives per year.

In the model, ‘Pdeath’ is calculated by using ‘Pcorrect’ from Capps et al.,36 53/116 (about 44%) (Table 16).

This is not the value used to estimate the probability of correct use in the model, i.e. during the cohort simulation, as Capps et al. 36 also presented separate values for children (< 15 years) and adults (shown with the value for SS in Table 17).

The probability of using an AI given SC (see Table 17, parameters 20 and 21) was based on the figures of use of AIs before arrival of the North West Ambulance Service (Capps et al. 36) and the total number of patients who received adrenaline. These figures are much lower than the 514 patients (adults and children) who eventually presented with symptoms that might be consistent with anaphylaxis, i.e. this implies that not all patients who had the symptoms of an anaphylactic reaction needed/received an adrenaline injection for treatment.

It is expected that the compliance of patients who received education (in SSs) will increase (Table 18, parameter 22). Compliance with AIs is mainly dependent on the knowledge of how to use it in a correct way, as well as the will to ensure that it is easily accessible and to use it when necessary. However, no estimate could be found of the effect of SS on compliance. Therefore, in the base case, 90% correct use was assumed, although recall that this means that those with a very short time to die (‘< 4.6 minutes’ category from Soar et al. 17) will still die (see Table 18). This makes the estimate more conservative.

Idiopathic treatment

Estimates to calculate probability of remission came from an observational study by Krasnick et al. ,10 which was used because it was the only study that could be found that included any time to event data to enable the probability of remission to be estimated. Data on years of follow-up and years in remission were provided, from which time to remission could be calculated by subtraction. Table 19 shows the data extracted for frequent and infrequent recurrence categories.

Only those and all of those experiencing frequent episodes received treatment with prednisolone. As this implies specialist provision, the probability of remission with SSs is the sum of that with frequent episodes (plus treatment) and infrequent episodes (no treatment), whereas the probability of remission with SC is only that of the infrequent episodes.

Because data on rate of recurrence were not available separately for those experiencing frequent or infrequent episodes, it was assumed that the same (average) rate (see parameter 6, Table 7) applied to both. Thus, when remission occurs, the average rate would decrease, as for those in remission the rate is zero. Therefore, the advantage of SSs over SC can be explained in the following way. The probability of recurrence given SSs is the sum across both the frequent, some of whom go into remission due to treatment, and the infrequent, some of whom go into remission spontaneously. However, the probability of recurrence given SC is the sum across the frequent, none of whom go into remission, and the infrequent, some of whom go into remission spontaneously.

From the data in Table 19 the median of time to remission was calculated, which was then used to inform the probability of remission (per cycle length, i.e. 6 months) in the model where, according to the definition of the median, the probability of remission per cycle (median time) = 0.5 and a constant rate (exponential model) assumed. The median was estimated by assuming that censoring (no remission at follow-up) indicated remission. This is a conservative estimate of time to remission. However, excluding the censored data produced a lower estimate and so the estimates of 4 and 1.5 for frequent are probably not too low. These estimates were used to form the most likely with assumptions as to the low and high (Table 20).

It was assumed that the rate of recurrence among those who did not go into remission would remain the same, which is probably an underestimate as the median time to remission is longer in those with frequent recurrence. Remission is still allowed to occur with SC, although only in those with infrequent recurrence, but also with no rise in the remaining rate so that there should be little bias towards either SC or SSs.

The proportion of frequent anaphylaxis (0.5) was also taken from the study by Krasnick et al. 10, which uses the same definition of frequent as the guideline, shown in Table 21.

Venom immunotherapy

Venom immunotherapy is indicated for patients who have a history of severe systemic reaction to a sting. 41 The effectiveness of VIT (Table 22, parameter 26) is estimated to be 85%; this is based on several studies that report a range of effectiveness of 75–95% (Krishna and Huissoon). 42 There is also a potential risk of anaphylaxis with VIT and, thus, increased cost and reduced utility but these are assumed to be negligible, especially given that the therapy is administered in a clinic where there is access to adrenaline and other emergency care (based on Cox 2011). 25 As VIT is time-consuming in terms of both frequency of treatments and total duration of therapy, and there is also the possibility of adverse reactions caused by VIT, we presume that not all patients will continue immunotherapy for 3 years (parameter 27, see Table 22). This figure is based on the finding of Goldberg et al. ,43 who reported a dropout rate of 40% in a study evaluating the attitudes of patients in Israel with insect venom allergy regarding after-sting behaviour and proper administration of adrenaline. We assumed that in the UK, 10 years later, the dropout rate of VIT would be much lower (about 20%) as a result of better care and fewer adverse events. Also, because of knowledge of these problems and the fact that, depending on the results of skin and anti-immunoglobulin E (IgE) testing, not everyone is eligible (as low as 65% according to Cox et al. )25 it was conservatively estimated that uptake would be about 60% (parameter 28; see Table 22).

Health valuation estimation

For the calculation of QALYs of the NICE reference case20 we needed an estimate of utility values (usually between 0 and 1), ideally obtained using the EuroQoL (EQ-5D index) instrument.

Utility (with no adjustment for anaphylaxis) was estimated as a function of age from a large recent EQ-5D US population study. 44 Decrements were then applied to each state except for that of ‘remission’.

For the estimation of the utility decrement due to being at risk of recurrence of anaphylaxis, the study by Voordouw et al. 45 was used. This case–control study45 using a postal survey was designed to evaluate the household costs associated with food allergy and also reported EQ-5D index data of 125 patients. The utility decrement was estimated as 0.08 (based on the difference between the values reported of 0.887 for cases and 0.803 for control subjects; p < 0.05) (Table 23, parameter 29).

We presumed that the impact of anaphylaxis will be very short but profound. The estimation of mean duration of having recurrence of anaphylaxis (parameter 30; see Table 23) was based on the finding that the mean loss of about 9 whole quality-adjusted life-days for severe allergic reaction due to penicillin is equivalent to utility decrement of the whole of the age-dependent utility for 1–9 days, reported in another CEA. 46 Unfortunately, this value was no obtained using the EQ-5D instrument, but appeared to be based on an assumption. Indeed, the mean length of hospital stay reported in the HES32 is only about 1 day, but this is likely to be an underestimate of the duration of the effect on well-being of recurrence. Therefore, a value half-way between these extremes was chosen, which is the expected value of a uniform distribution bounded by 1 and 9.

Finally, there was expert opinion that the reassurance provided by attending an SS through, for example, diagnosis of trigger and learning how to avoid triggers, as well as the provision of AI, should reduce the utility decrement due to the condition (parameter 31; see Table 23). Therefore, in the absence of any evidence as to the extent of this effect, ranges of 0–0.5 for a factor to be multiplied by a utility increment equal to the decrement due to anaphylaxis, were chosen for each of SSs and AI (parameter 32; see Table 23). This means that, at best (factor = 0.5 for SS + 0.5 for AI = 1), the combination could completely remove the decrement and, at worst, have no effect (factor = 0).

Resource use and unit cost estimation

Table 24 gives a description of the unit costs (in £) and the resource-use data used in the model.

The mean cost and standard error of inpatient care was estimated from the individual Primary Care Trust data for the period of 2009–10 from the Department of Health Hospital Episode Statistics (www.hesonline.nhs.uk)32 (see Table 24, parameter 33).

The average costs of AIs were based on the costs reported in the BNF 6118 (parameter 34). The lifespan of an AI was assumed to be 6 months and two prescribed or replaced at a time, based on expert opinion. Only those who were prescribed an AI incurred that cost and this was assumed to be for the rest of their lives. The costs of treatment of patients in SS were based on the NHS reference costs. 47 All individuals with anaphylaxis, regardless of trigger, incurred the cost of two appointments (one initial and one follow-up) in the first 3-month cycle of the model. These were based on the ‘multiprofessional’ categories: for children, Paediatric Clinical Immunology and Allergy (Service code 255) and, for adults, Clinical Immunology and Allergy (Service code 316) (see Table 24, parameter 35). Expert opinion was to include the cost of training in the use of the auto-injectors, i.e. there was no additional training cost.

Only those with an insect trigger and who underwent VIT incurred those additional costs (see Table 24, parameters 36–40). Model estimates on current practice of VIT in the UK are based on an audit that evaluated the adherence to international guidelines48 and on expert opinion (Dr Pamela Ewan, Allergy Department, Addenbrookes Hospital, 9 June 2011, personal communication). Most of the VITs in the UK were given by injection of a purified extract (Pharmalgen, ALK-Abelló UK, Reading, UK), using an induction scheme of weekly injection for about 10 weeks and continuation for about 3 years, with about a 6-weekly interval during maintenance. 48 In the model, a mean duration time of VIT of 3 years with a range of 2–4 years is used48 (see Table 24, parameter 36). The average costs for bee and wasp extracts used for VIT were based on the costs reported in BNF 6118 (parameter 38–39). The number of weeks between VIT maintenance doses was based on expert opinion and on the American guideline for allergen immunotherapy25 (see Table 24, parameter 40). The duration of the build-up phase, based on expert opinion, was up to 10 weeks and, given that the next dose would not occur for at least another 4 weeks, implied that the cost in the first 3-month cycle was only that of initial treatment (£70). The cost thereafter is therefore calculated as number of maintenance doses multiplied by cost of maintenance dose (£60), where mean number of maintenance doses is duration of maintenance divided by number of weeks between doses.

Only those with idiopathic anaphylaxis with frequent episodes incurred the additional cost of prednisolone (see Table 24, parameters 41–44). The recommendation from two international guidelines23,24 for prednisolone is 1–2 weeks every day, starting at 60–100 mg, until symptoms are under control and then decreasing over a period of about 2–3 months.

Those with food triggers also incurred additional regular follow-up costs in accordance with expert opinion that would be necessary to reinforce avoidance measures. According to expert opinion, the frequency would vary depending on the specific food trigger and age, with milk trigger in children having the highest frequency. However, an average of about once every 2 years over a lifetime was assumed in the base case (see Table 24, parameter 45). The cost of each follow-up was also taken from the NHS reference costs47 (see Table 24, parameter 46).

Discount rate

The discount rate for costs and benefits was 3.5% in accordance with NICE methods guidance. 16

General model assumptions

We assumed that 50% of the population consisted of males, which is based on the HES. 32 Furthermore, we assumed that there are only four main triggers of anaphylaxis: drug, food, insect/venom and idiopathic (no known cause). We expected that in SC there is either no referral to SSs or a GP referral only after anaphylaxis, and that SSs essentially consisted of SC plus referral to SSs on the basis that the patient would probably see his/her GP as well. We also assumed, based on expert opinion, that anyone with anaphylaxis gets only two sessions with SSs unless cause of anaphylaxis is insect or idiopathic. In all causes, patients receive benefit from recurrence rate reduction, utility increase and mortality rate reduction from SSs, and from only mortality rate reductions with AI. We assumed that historic recurrence and mortality rates are due to SC only, given the likely low rate of referral to SSs: in one study the referral rate was zero. 13 Finally, we expected the cost of recurrence to be due to hospital admission only, i.e. no further follow-up costs were included, which is conservative in terms of the chances of SSs being cost-effective.

Further assumptions are explained in each of the sections on model parameters below.

Time horizon

The time horizon was lifetime in accordance with NICE methods guidance. 20

Base-case results

An arbitrary age of 30 years was chosen for the base case, and Table 25 and Figure 3 show the results of the model run probabilistically (10,000 simulations).

FIGURE 3.

Base-case results (probabilistic).

This shows that SC with AI would not be cost-effective. SS with no AI would be cost-effective if the threshold [willingness to pay (WTP) for a QALY] was greater than about £740, and SS with AI would be cost-effective if the threshold was > £1800 per QALY. Given a threshold of £20,000 this would make SS with AI cost-effective.

In order to show the effect of the uncertainty a cost-effectiveness acceptability curve (CEAC) was plotted.

Figure 4 shows that, at a threshold above about £2000 per QALY, SS with AI is most likely to be cost-effective, and, below this, SS without AI would be most likely.

FIGURE 4.

Cost-effectiveness acceptability curve: base case.

Table 26 shows the results of the deterministic (parameters at expected values) analysis.

It can easily be seen that there is virtually no difference between the results, indicating that the expected cost and QALYs are close to a linear function of the parameter values. It is for this reason and that it is much quicker to run the TreeAge software deterministically that all one way or threshold sensitivity analyses were conducted deterministically.

Sensitivity analysis

Table 27 shows the results for the age of 5 years.

The results are essentially very similar to those for the age of 30 years, except that SC plus AI was extendedly dominated [incremental cost-effectiveness ratio (ICER) to move from SC no AI is greater than to move from SC plus AI to SS no AI].

Table 28 shows the effect of varying the time horizon instead of using a lifetime.

Table 28 shows that, as the time horizon decreases, SSs plus AI becomes less likely to be cost-effective. Indeed, threshold analysis (see next paragraph) reveals that, starting at the age of 30 years, for a range of time horizons from 1 to 3 years, assuming a WTP of £20,000 per QALY, SC plus AI would be cost-effective. This is true up to 2 years only for children.

Threshold analysis was conducted on all parameters. All probabilities were varied between 0 and 1 and, unless stated otherwise, a WTP of £20,000 was used.

No change to SS plus AI being cost-effective was observed for the following:

• Population age (0–90 years, base case: 30 years).

• Probability trigger was drug.

• Probability trigger was idiopathic.

• Probability trigger was insect.

• Rate of recurrence with drug caused anaphylaxis with SC (base case: 0.12).

• Rate of recurrence with drug caused anaphylaxis with SS (up to 0.12, base case: 0.001).

• All rates of recurrence due to some trigger (drug, food or insect) with SSs (up to 10 times base case for all in multiway sensitivity analysis).

• Cost of SSs (up to £10,000, base case: about £250 or about £400, depending on age)

• Frequency of follow-up for food trigger (up to once per month, base case once every 2 years).

• Proportion frequent idiopathic.

• Probability of remission, either frequent or infrequent.

• Cost per milligram of prednisolone (up to £1, base case: £0.02).

• Cost of VIT (initial or maintenance) (up to £200).

• Effectiveness of VIT (0–1, base case: 0.85).

• Probability of correct use with SSs (0–1, base case: 0.9).

• Probability of dying from anaphylaxis with no intervention.

• Utility improvement factor for SSs (0–0.5, base case: 0.25).

• Utility improvement factor for AI (0–0.5, base case: 0.25).

It was observed that there was a change from:

SS plus AI to SC plus AI above 0.35 for rate of recurrence in food-caused anaphylaxis (base case: 0.01)

SS plus AI to SS no AI above 0.03 for probability of dying with injector (correct use) (base case: 0.000252)

SS plus AI to SS no AI above £146 for cost of injector (base case: £26.45) (at start age of 30 years; less than this implies a higher threshold)

SS plus AI to SS no AI below 0.03 for utility improvement factor with AI (base case: 0.25)

SS plus AI to SC plus AI between time horizon of about one and, for adults, 3 years and, for children, 2 years (base case: lifetime)

SS plus AI to SS no AI below 0.77 for probability of correct use with a SS, no utility increment for AI use (base case: 0.9)

Therefore, in summary, that SS plus AI was cost-effective at a threshold of £20,000 per QALY was robust to all sensitivity analysis except mostly at relatively extreme values of a small number of parameters. The only exception was if it was assumed that there was no benefit to having AIs irrespective of whether or not they were used correctly: this analysis was performed given the lack of evidence to inform the utility increment for AIs (see Health valuation estimation). In this case, a threshold of 0.77 (77%) does not seem that implausible.