Are adverse effects incorporated in economic models? An initial review of current practice

Article history

The research reported in this issue of the journal was commissioned and funded by the HTA programme on behalf of NICE as project number 07/57/01. The protocol was agreed in May 2007. The assessment report began editorial review in August 2008 and was accepted for publication in May 2009. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HTA editors and publisher have tried to ensure the accuracy of the authors’ report and would like to thank the referees for their constructive comments on the draft document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

None

Copyright statement

© 2009 Queen’s Printer and Controller of HMSO. This monograph may be freely reproduced for the purposes of private research and study and may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NETSCC, Health Technology Assessment, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK. 2009Queen’s Printer and Controller of HMSO

Adverse effects of health-care interventions

Health-care interventions have the potential for unwanted harm as well as the hoped-for benefit. These unwanted harms are known as adverse effects. Occurrences of harm recorded during a clinical study of a health-care intervention, and which may or may not be caused by the intervention, are referred to as adverse events. All drugs are associated with potential adverse effects, more specifically referred to as adverse drug reactions, some of which can be anticipated from the preclinical and clinical pharmacology and others which are unexpected and are identified only after considerable patient exposure in clinical practice. Some adverse effects in the context of one indication may actually represent another therapeutic indication for the drug. Procedural interventions are not necessarily free of unwanted adverse effects. A less-invasive surgical procedure may be beneficial in the short term but may be associated with an increased rate of reintervention in the long term, for example stapled haemorrhoidopexy. 1 Psychosocial interventions, although often assumed to be benign, can also have unwanted adverse effects but these may not be investigated. 2 Diagnostic tests can also have unwanted adverse effects, either directly, such as through adverse reactions to contrast media,3 or through the negative consequences of false positives, which can result in unnecessary treatments, or iatrogenic effects, through raising concerns over health. 4 It is self-evident that the benefits of a treatment must not be outweighed by adverse effects, but how data are found and used to populate these two sides of the equation is complicated.

Research into adverse effects can be problematic in terms of their identification, quantification and valuation. In drug development, preclinical studies are conducted on homogeneous, inbred healthy animals and early investigations in humans are restricted to healthy individuals. Such studies are inadequate for the prediction of idiosyncratic adverse events that may occur in the context of the heterogeneous population of unique genetic and environmental factors. Randomised controlled trials (RCTs) designed primarily to investigate efficacy will, based on the early evidence, exclude patients at risk of adverse effects and therefore the data on adverse effects derived from such RCTs will never provide a complete picture of the adverse effect profile of a drug. 5 Additionally, the time horizon of many RCTs will not be long enough to capture rare but important adverse events. Similarly, RCTs of non-pharmacological interventions are likely to have selected populations, thereby limiting their generalisability. Larger observational studies, although not suffering from the failings of the RCTs, are limited by confounding factors, which prevent the drawing of unequivocal causal links between the intervention and an adverse event. Furthermore, even these large studies are limited by sample size when rare adverse effects are considered.

In addition to the difficulties in identifying adverse effects and deriving accurate estimates of their incidence, valuing them is also problematic. How does one weight the numerous minor adverse effects against the risk of a single serious event? How does the researcher value the reduction in risk of very rare events? These issues can also be problematic for other outcomes that are used to inform decision-making and they are part of the reason that many advocate the use of a single index score, which should, if appropriately measured, capture all relevant outcomes.

Adverse effects in health technology assessments

A recently published definition of health technology assessment (HTA) states that it is ‘. . . a multidisciplinary process that summarises information about the medical, social, economic and ethical issues related to the use of a health technology in a systematic, transparent, unbiased, robust manner. Its aim is to inform the formulation of safe, effective health policies that are patient focused and seek to achieve best value. Despite its policy goals HTA must be firmly rooted in research and the scientific method’ (European network for Health Technology Assessment: www.eunethta.net/HTA/). Across countries the practice of HTA varies; often it comprises a systematic review of the clinical effectiveness evidence and an economic evaluation. This form of report is increasingly being used by decision-makers to help make treatment recommendations. For example, technology assessment reports that include a decision model are a key part of the decision-making process used by the National Institute for Health and Clinical Excellence (NICE) in England and Wales. 6 For the purpose of this project we have focused on technology assessments that have included both a systematic review and a decision model.

The ultimate objective of the assessment of health-care technologies is to assist decision-makers in the difficult task of choosing between two (or more) mutually exclusive alternatives by comparing benefits obtained against the resources consumed. In the absence of perfect information, decision models are a helpful tool to provide evidence so that societal health gains can be maximised from scarce resources. There is a need to ensure that for all interventions being compared the relevant outcomes and resource use have been captured in the evaluation. All interventions will have multiple outcomes and outcomes will vary between interventions. Importantly, these outcomes should be incorporated using some standard index; this may be best achieved through the use of a generic preference score [utility, health-related quality of life (HRQoL)] that will allow interventions with more than one outcome to be easily compared. The use of a single index allows the multidimensional changes in health to be translated into a single score. However, although this is the ideal method for incorporating the outcomes of an intervention into a decision model, in practice finding/calculating utilities that capture all of the relevant outcomes is difficult. Thus, a utility may capture one aspect of an intervention adequately, for example the efficacy, but may reflect poorly other outcomes, such as adverse effects. In practice, outcomes are incorporated into models in a variety of ways: relative treatment effects, withdrawals and costs, as well as utilities. Because of this diversity of methods, transparent reporting of decision models is essential.

Decisions about adoption of treatment should consider both negative and positive effects. 7–9 It has been recommended that systematic reviews of adverse effects should be considered as important as the review of efficacy. However, the vast majority of systematic reviews focus on efficacy or clinical effectiveness without adequately addressing adverse effects. 10 Similarly, it is not clear that economic evaluations always consider, and incorporate, the appropriate adverse effects. Decision models provide us with an explicit framework that we can use to help inform decision-making. However, the output of any model is heavily dependent on the model inputs and any results can be considered robust only if all relevant inputs have been included. For many interventions this should include some consideration of adverse effects.

If there is a failure of technology assessments to adequately incorporate adverse effects, this could limit the results obtained or recommendations made. The impact of including adverse events in the economic model could potentially change the findings; interventions found to be cost-effective may be shown to be not cost-effective, or less cost-effective than comparable treatments, when adverse effects are considered properly.

To redress this potential overemphasis on efficacy within technology assessment, the consideration of adverse effects data needs to be encouraged. The Centre for Reviews and Dissemination (CRD)11 and the Cochrane Collaboration12 have recently published initial guidance on incorporating adverse effects in systematic reviews. However, the need to include the results of such systematic reviews into economic models has not been addressed directly. Although the methodological guidance on good practice for decision-analytic modelling in health care issued by NICE in 2004 and updated in 20086 does not specifically address how adverse effects data are incorporated into the model the guidance does acknowledge the importance of their inclusion.

There is a possibility that the importance of adverse effects in decision models is undervalued. It is unclear how they are considered and incorporated in economic models; it is possible that they do not appropriately contribute to the evidence provided to decision-makers. The initial step in developing the methodology regarding the incorporation of adverse effects in technology assessments and to produce further guidance should be to review current practice to establish the current status.

Objectives

There were two main objectives to this research. They were:

1. To identify what, if any, methodological research exists on the incorporation of adverse effects in economic models.

2. To review published technology appraisals to establish the current practice of researchers. Our review did not fulfil the intentions laid out in the protocol in that it did not address the question, ‘Are adverse effects incorporated adequately and appropriately in economic models?’. We decided that within this first stage project such subjective questions could not be addressed and thus we limited the review to the more objective questions of whether adverse effects were included and how they were included.

The aim of this research is to generate a sound body of information upon which to build recommendations for future research and/or best practice.

Introduction

Before embarking on a review of practice in relation to the incorporation of adverse effects in economic models it was important to investigate the relevant methodological research available to researchers. We therefore conducted a review of such research. Because of the difficulties of searching and screening for publications relating to methodological research the publications and information identified by this review should be taken as a reasonable, but not necessarily exhaustive, sample of the existing information.

Methods of review of methodological research

Results of review of methodological research

The electronic searches identified 736 references. Of these, 44 were considered to be potentially relevant and were ordered for screening of the full paper. Full paper screening identified five published articles that met the inclusion criteria for the review (Table 1). The list of excluded articles is given in Appendix 6. It should be noted that even these ‘included’ articles contained very little information or guidance of direct relevance to the incorporation of adverse effects in models. The full data extraction is given in Appendix 4.

All five publications were appraisals of existing guidelines or practice and aimed to provide guidance for modellers. In two publications14,15 the information relevant to adverse effects in models was derived from an appraisal of existing guidelines, whereas three publications14,16,17 aimed to develop a checklist or specific guidance for modellers (Phillips et al. 14 did both); one publication13 described a survey of the sources and quality of data used in economic models (HTAs between 1997 and 2003).

All five publications made some direct (if only passing) reference to adverse effects in models. Of the themes that could be taken as relevant to adverse effects in models the most common was the discussion of the inclusion of outcomes (Table 2). Three of the five articles addressed this directly and all three concurred that models should include all relevant outcomes. Two clarified that relevant outcomes meant those that differed between the interventions of interest; from this one can infer that when adverse effects differ in frequency or severity between treatments in a model, or at least if the resources used because of them differ, they should be included in the model. 14,16,17 All three explicitly stated that adverse effects should be considered as an outcome. 14,16,17

The other themes were the choice of the model parameters (Table 3) and the source and quality of the (adverse effects) data (Table 4).

Adverse effects were rarely addressed when discussing the choice of parameter. Even though the publication by Philips et al. 14 has been included here, it merely states that ‘the choice of outcomes in the model should be justified’. Tappenden et al. ,15 although advocating the inclusion of adverse effects, considered only those adverse effects that were expected to be avoided by the treatment of interest. They did not consider adverse effects incurred as a consequence of the treatment. Thus, overall, it may seem that adverse effects are not explicitly high on the list of priority outcomes to be considered in models. However, it is likely that adverse events may be considered as just one amongst many outcomes for a particular treatment and, as for any outcome, when they are important they will be a high priority.

Tappenden et al. 15 suggest that adverse effects are best included as part of HRQoL. This is because of the existence of multiple events/outcomes of varying severity and the need to capture and value these in an appropriate manner, i.e. through the use of a single index.

Three of the publications touched upon the source and quality of data (Table 4). Although two14,17 of the five articles advocate conducting a full systematic review of key parameters, neither publication gives any guidance as to what is a key parameter. The study by Cooper et al. 13 would suggest that, in practice, in the majority of cases adverse effects are not considered a key parameter. In 10% of reports adverse effects had not been included in the model. When adverse effects and complications had been included, in 31% (at best) of the reports the source of the data was unclear. Data from meta-analysis of RCTs with direct comparison between interventions of interest and using final outcomes were used in 14% of cases, and data from a single directly relevant RCT were used in 17% of cases. A further 2% of reports used data from a single RCT using a surrogate outcome, 14% used data from case–control or cohort studies and 12% used expert opinion.

It is noteworthy that in the most recent review of guidance on economic decision modelling14 a chapter on appropriate methods for the identification and quality assessment of secondary parameter estimates does not mention adverse effects.

Summary findings of review of methodological research

It is clear from the available guidance that all relevant outcomes should be included in the economic decision model and there appears to be a general if not clearly stated consensus that this includes adverse effects.

One might have expected adverse effects to feature in guidance on how to select parameters for the model. However, it is likely that adverse events may be considered as just another outcome for a particular treatment and, as such, when they are important they will be a high priority. The position taken by Tappenden et al. 15 that adverse effects are captured through HRQoL may be typical and it may well be that in most cases analysts attempt to capture adverse effects in this manner.

Guidance for decision modelling suggests that there should be a full systematic review for key parameters but there is no real indication that adverse effects should be considered as one of these key parameters. There is an implicit assumption that adverse effects are very important, but the lack of clear reference to adverse effects may well reflect uncertainty and lack of clarity regarding how they should be dealt with and considered in economic models.

Methods

Results

Summary of findings from the review

The review covered a broad range of HTAs in terms of the therapeutic area, type of intervention and type of decision model employed. In total, 85% of the reports included adverse effects in the clinical effectiveness review and 54% of the decision models included adverse effects in the model. Just under half (49%) included adverse effects in both the clinical review and the model.

The link between the adverse effects in the clinical review and the model was generally weak. Although 18 of the models used adverse effects data from the clinical review and 14 reviews did include a meta-analysis of adverse effects, only 3/80 (< 4%) used the results of a meta-analysis from the systematic review of clinical effectiveness and none of these was able to use only the data from the review without some further manipulation being required.

There was no apparent relationship between inclusion of adverse effects in the model and therapeutic area, type of intervention or year of report, nor type of model. Models with a 20-year or longer time horizon did include adverse effects more often than those with shorter time horizons. This could be a reflection of the more comprehensive nature of long-term models, but was not investigated further.

Of those models that did include adverse effects, 67% used a clinical adverse effects parameter, 79% used a cost of adverse effects parameter, 86% used one of these and 60% used both. It was beyond the remit of this review to determine whether these clinical and cost parameters were appropriate or adequate, or if all relevant adverse effects had been incorporated, or if the data used for their capture were reliable. These are all questions that require further research to be answered.

Most models (83%) used utilities but determining whether these utilities captured adverse effects was problematic. Only two models (2.5%) used solely utilities for adverse effects and were explicit in their beliefs that the utility captured relevant adverse effects. A total of 35 reports (81% of those models that included adverse effects and 44% of all reports) derived utilities from patients on treatment and might therefore be interpreted as capturing adverse effects. The issue of utility derivation is widely debated among health economists and it is not clear that there is consensus on who should value the health states, or which valuation technique should be used. 32 In an attempt to estimate the likelihood of adverse events being captured within the utility we used a simplifying assumption, namely that eliciting utilities from patients on treatment was the most likely method to have implicitly captured adverse effects. Further, only those models that had not explicitly captured adverse events through the use of cost or clinical data were investigated in any depth.

A total of 13 reports (30% of those models that included adverse effects and 16% of all reports) used withdrawals related to drug toxicity and therefore might be interpreted as using withdrawals to capture adverse effects, but this was explicitly stated in only three reports. However, the remaining 10 models also incorporated adverse effects explicitly through at least one other parameter.

Of the 37 models that our review classed as not having included adverse effects in the decision model, 18 gave a justification for this omission. Most commonly the justification was a lack of data, followed by the adverse effects having minimal impact on quality of life or cost.

Overall, 43 models included adverse effects and a further 18 gave a reason for not including adverse effects. Thus, 19/80 (24%) reports appeared to have no explicit consideration of adverse effects in the decision model.

Reporting of adverse effects

A key part of the present review was determining whether or not adverse effects had been included in the decision model. This proved to be more difficult than had been anticipated and raised important issues regarding the transparency of the reporting of models. In particular, the lack of explicit reporting with regards to which adverse effects had been considered in the model and how they had been captured and evaluated led to a number of difficulties. In many instances some interpretation and understanding of methodology was required to ascertain if and how adverse effects had been captured. For example, HTAs with poorly reported model structures failed to show when adverse effects had been captured through the withdrawal arm of a decision tree, or if one or more of the health states defined within the model structure included adverse events. Also common was a failure to mention adverse events anywhere in the text, presenting only a table of cost input parameters. Although it is legitimate to present adverse effects parameters in this way, and it is possible to unearth the relevant information from within the report, it is highly likely that many readers may miss this pertinent information and may well fail to understand how adverse effects were incorporated or, worse, may draw the erroneous conclusion that adverse effects were not included in the model.

It is widely accepted in the health economics community that more formal, transparent and replicable approaches to the identification and assessment of the quality of model inputs may reduce the ‘black box’ nature of decision models and lead to less scepticism regarding model outputs. 13 With specific reference to adverse effects, it is essential that reporting allows a reader to understand why adverse effects are important to the decision problem, how and where those adverse effects included were identified, and what methods were used to incorporate the relevant adverse effects into the model. When appropriate there must be clear justification for the non-inclusion of adverse events; legitimate decisions for not including adverse effects, such as adverse effects having a negligible impact on health outcomes, or no impact on costs and resources, should be explicitly reported.

The findings of the review of methodology papers (see Chapter 2, Summary findings of review of methodological research) show that, although there appears to be an implicit assumption within modelling guidance that adverse effects are very important, there appears to be a lack of clarity and guidance on how they should be dealt with and considered in modelling. This is likely to be due in part to the diversity of the decision problems and the wide range of important/unimportant outcomes that this diversity creates. Within this range of outcomes adverse effects may be considered as just one more outcome. The most relevant outcomes, which may or may not include adverse effects, are specific to each treatment pathway evaluated.

The transparency of the reports that were reviewed for this project varied greatly. However, in many cases the reporting was insufficient for the audience for whom the reports are intended; determining which outcomes had been deemed most relevant and therefore included was problematic. We acknowledge the fact that the level of detail that can be reported is often restricted by word limits, but in the instance of HTA reports the limitations are not so restrictive as to limit the transparency of reporting.

Different ways to capture adverse effects

There are a number of areas within a decision-analytic framework in which adverse effects might be incorporated or captured. These include the model structure, clinical events, utilities, costs and resources. How adverse effects are incorporated is heavily dependent on the intervention being evaluated, the impact of the adverse effect and the scope of the decision problem.

The inclusion of adverse effects through a clinical event may seem the most obvious method. In practice, 67% of models used a clinical probability. However, use of a clinical probability is not a guarantee that all of the relevant adverse effects have been captured. Some reports included a single adverse effect or adverse effects of one intervention, with no consideration for the adverse effects of the comparator interventions. Detailed analysis of these issues was beyond the scope of the present review, but further research into this important issue is warranted. Clear explanations by authors of why certain clinical parameters are included rather than others should be an important aspect in the reporting of decision models. Only a very small proportion of models used data directly from the accompanying systematic review. It would appear that further efforts need to be made to include relevant adverse effects outcomes in the systematic reviews of HTAs. When a systematic review of adverse effects is not possible or feasible, the clinical effectiveness review could include a summary of the adverse effects profiles of the interventions of interest; this could then be used to structure and populate the model either directly or by helping the appropriate utilities to be used.

It is justifiable that some models include only a cost/resource parameter to capture adverse effects. For example, some adverse effects may have no significant or measurable impact on quality of life or health benefit, but may lead to an increase to inpatient length of stay. If this is true it may be appropriate for a cost estimate of that stay to be incorporated into the model. Our review found that 79% of models incorporated a cost parameter but only 10% incorporated cost parameters without explicit inclusion of a clinical parameter. This may be justifiable, but without justification it may make little sense to the reader.

In the evaluation of pharmaceutical interventions, adverse effects (i.e. toxicity) may be incorporated into the model structure through withdrawals. This allows individuals who experience the adverse effect to follow an alternative pathway, which has relevant costs and benefits associated with it. Of the 16 models that incorporated withdrawals, 13 were evaluating pharmaceutical drug interventions and appeared to include adverse effects in this manner. However, the nature of withdrawals and whether or not the reports’ authors anticipated that they capture adverse effects was not explicitly reported in all of the HTAs.

A high proportion of the reports reviewed derived a utility outcome. This is not surprising given that this is recommended within the current NICE methods guide. 6 These guidelines reflect that it is important to be able to value outcomes, including adverse effects, in a consistent manner and that a single preference score is the most appropriate for policy decision-making purposes. 6 Although it is outside the scope of this report to debate the issues surrounding the use of generic valuation tools and whose values should be elicited, it is worth discussing the impact that these variations may have on the ability of the utility to capture adverse effects. It is likely that utilities elicited from patients on treatment may capture some, or all, of the adverse effects experienced by those patients. However, although a number of HTAs did appear to derive utilities from patients on treatment, few made specific claims that adverse effects had been captured through this methodology.

The methods by which the utility valuations were obtained varied. The argument for the use of a generic preference-based measure is to allow comparisons between health-care programmes. Whether for the same condition or when they involve different medical conditions and treatments32 there can be a need to address disparate outcomes in a consistent manner. This is the position that has been adopted by NICE. 6 However, an alternative is to use condition-specific descriptions that may be more sensitive to changes in the given condition and may better reflect the concerns of the patient. 33 This was not an issue that was explicitly addressed in this review, although the majority of reports, as would be expected given the NICE guidance, used, or mapped to, a generic measure. There is some evidence of generic measures being insensitive for certain conditions, such as respiratory disease, but there are a number of potential issues that need to be addressed when mapping non-preference-based measures onto preference-based measures and using values from the literature. 32

Given the variation in ways of describing health, the valuation techniques and respondents (patients, general public, clinical experts), the values that are likely to be found in the literature may vary greatly. This complexity highlights the need for explicit reporting, which in turn would enable the reader to make better judgements about whether it is realistic to expect some adverse effects of interventions, long term or short term, to be captured within the utility. How best to ensure that any adverse effects of interventions are captured within the utility needs further investigation and it is likely that more rigorous methods will need to be adhered to.

The link between the systematic review and decision model

The scope of the decision problem being addressed by the systematic review component and the decision model may differ. Often, the systematic review may focus only on the effectiveness (both positive and negative) of the intervention being evaluated. The scope of the decision modelling question may be much broader, aiming to evaluate the total net benefit of an intervention including any downstream effects that might be observed. This leads to a divergence in both the question being posed and the data required to provide an answer.

The results of the review show that a high number of models (95%) considered one, or more, of the effectiveness outcomes that were evaluated in the systematic review and in the majority of instances the data from the review were used in some capacity. However, the links between the review and the modelling components are not as strong for adverse effects. A high proportion (85%) of the reviews evaluated adverse effects, some from a broad focus and some from a narrow focus. However, just over half of the models incorporated those same outcomes into the model, with fewer again utilising the data obtained by the review. This is not necessarily a negative finding; in some cases it may reflect the slightly different focus of the two components of the reports.

The source of the adverse effects parameter was rarely the results of the systematic review. Our review did not investigate in detail the other sources of adverse effects data, although it is clear that non-systematically derived literature-based data were the most commonly used. In their study, Cooper et al. 13 found that, at best, 14% of adverse effects outcome data were sourced from the best quality sources, i.e. a meta-analysis of randomised controlled trials.

Issues with evaluations of diagnostic/screening interventions

Economic evaluations of diagnostic tests are intrinsically more difficult than assessments of therapeutic interventions, mainly because of the uncertainty surrounding the relationship between the diagnostic test and the health outcomes finally achieved. 34 Decision modelling of diagnostic technologies typically encompasses the outcomes from future treatments and management as well as the impact of the actual test. It is not uncommon for such models to include adverse effects of treatments without including those of the test of interest. This may be because the impact of the adverse effects of the test are minimal compared with those of the future treatments. This may be entirely appropriate, but it needs to be stated explicitly. In general, diagnostic reports appear to separate into two groups: those that link the test to an intermediate outcome, for example cost per case detected, and those that link the test to a final outcome, for example cost per quality-adjusted life-year. As the aim of an HTA report is to inform national policy, one might expect a wider perspective that included final outcomes in the form of utilities. However, this may not always be possible because of data limitations. Additionally, models that explicitly include adverse effects may tend to be those in economic evaluations of the more invasive diagnostic/screening technologies. Intuitively one can imagine that a test such as a coronary angiogram, which involves the injection of a dye into the blood, may be more likely to have adverse events associated with it than a test that involves a dipstick. Information on these issues was not data extracted, although it was observed that in some of those reports in which the test seemed more invasive there appeared to be some discussion around the impact of false positives on quality of life.

False-positive results from diagnostic technologies can have adverse effects. An HTA report published in 200035 found little evidence to support the impact of false positives on quality of life, although it is clear that both false positives and false negatives have the potential to impact on the outcomes of the decision model and to affect both the costs and benefits. The report found that decision analysis is likely to be valuable in demonstrating how false results (positive/negative) may be incorporated in screening decisions. Explicitly incorporating values associated with false results may show how they affect decisions about the appropriateness of screening. However, the report found limited empirical evidence to support its findings.

Recommendations for practice

• The main recommendation is for much clearer and explicit reporting of adverse effects in decision models. As a minimum, separate sections on adverse effects should be included in the clinical effectiveness and modelling chapters of every technology assessment report.

• Efforts should be made to ensure that all components of technology assessments explicitly consider those outcomes, including adverse effects, that are relevant to the decision problem. Similarly, there should be explicit recognition in future guidelines that ‘all relevant outcomes’ should include some consideration of adverse events.

• Whenever the inclusion of adverse effects is not relevant a justification should be explicitly provided by the authors. By doing this, the readers will be made aware that adverse effects were considered at some stage of the process.

• Improved links between the outcomes of the model and the data inputs presented in the systematic review and model description may aid the reader’s understanding and support the decision-maker.

• Even when a systematic review of adverse effects data is not feasible, summaries of adverse effects data that can be used to address the decision problem should be presented in the clinical effectiveness review.

Recommendations for research

This report has presented an overview of the current situation regarding the consideration of adverse events in HTA models. It is clear that there are a number of limitations to, and issues outside the scope of, this project that still need to be addressed through further research. Our suggestions for further research include:

• A detailed review and critique of the methods used to identify and incorporate adverse effects in economic models.

• A detailed assessment of how to judge the relevance and appropriateness of the adverse events included. This may involve an in-depth analysis of a subset of reports involving interaction with the report authors.

• Although it is unlikely that any single standard methodological approach could be appropriate for all decision problems, some investigation into whether some methods are more appropriate for certain types of decision problems or clinical areas may be warranted.

• Further investigation into the methodology of mapping disease-specific outcome measures to generic outcome measures.

Contribution of authors

Ms Dawn Craig assisted in the preparation of the protocol and contributed to the identification, extraction and interpretation of data and the preparation of the final report. Mr Steven Duffy conducted the literature searches and contributed relevant sections to the final report. Mr Tiago Fonseca contributed to the extraction and interpretation of data and the preparation of the final report. Dr Catriona McDaid contributed to the identification, extraction and interpretation of data and the preparation of the final report. Mr Christian Stock contributed to the identification, extraction and interpretation of data and the preparation of the final report. Dr Nerys Woolacott prepared the protocol and provided input at all stages of the project.

Disclaimers

The views expressed in this publication are those of the authors and not necessarily those of the HTA programme or the Department of Health.