What evidence is there for a relationship between organisational features and patient outcomes in congenital heart disease services? A rapid review

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its proceeding programmes as project number 13/05/12. The contractual start date was in March 2014. The final report began editorial review in April 2014 and was accepted for publication in May 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

none

Copyright statement

© Queen’s Printer and Controller of HMSO 2014. This work was produced by Turner et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

Rapid review methods

Owing to the need to complete this review within a very short timeframe (12 weeks including a 3-week protocol development stage) rapid review methods were used to ensure the efficient identification and synthesis of the most relevant evidence.

Rapid review methods are still in their relative infancy, in comparison with the more established systematic review. Harker and Kleijnen3 examined a number of rapid reviews in order to develop understanding and definition of what a rapid review was. Rapid reviews are undertaken over a short time frame with a streamlined methodology. This streamlined methodology is a necessary compromise from a standard systematic review. Although Harker and Kleijnen3 found considerable variation in the methodologies adopted by rapid reviews, acknowledging that there is not a ‘one size fits all’ methodology, they advise ‘clear and transparent description and discussion of methodology utilised and acknowledge any limitations’. This advice has informed our choice of methods and report writing.

Our review did not attempt to identify all relevant evidence or to search exhaustively for all evidence that meets the inclusion criteria; the search approach aimed to identify the key evidence of most relevance to the review question.

The scope to both search for and review related evidence, reflecting the multiple dimensions of the topic, was considerable and, thus, was considered prohibitive within the given time frame. The rapid review therefore focused on the most relevant evidence from CHD services for children and adults. The rapid review was based on a proposed conceptual framework included in the study protocol. This allowed us to:

• define the scope of the search strategy

• define inclusion and exclusion criteria to specify what types of studies were to be included in the final report

• construct summary tables of all included studies to present key information and findings

• synthesise the evidence from the included studies.

Protocol development

The protocol for the review was developed iteratively between the School of Health and Related Research (ScHARR), NHS England and the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) programme. In addition, comments were sought from key stakeholders, who were part of the NHS England Clinical Advisory Panel for the CHD review. The protocol development started on 7 January 2014 and was published on the NHS England website on 10 February 2014. 4

Use of the conceptual framework

There is an extensive health services research evidence base documenting associations between a range of organisational factors, particularly factors related to location, nature and size of specialist facilities and outcomes, in both elective and emergency service provision. There is also a major field of research that has explored, both quantitatively and qualitatively, the impact of different aspects of service organisation and delivery which influence patient safety and may reduce the risk of adverse outcomes for patients. In order to make the relationship between this wider evidence base and the, relatively limited, scope of this commissioned rapid review more explicit, a logic model (or conceptual framework) was developed for the study protocol and this is included in Appendix 1. This figure shows the relationship between the specific inclusion criteria for this review and the much wider context of factors of known relevance which were considered for inclusion in the review if there were relevant data within the included studies. This approach was chosen based on the need to both limit the scope of the review to the most relevant evidence, while not ignoring the very wide range of organisational, cultural and patient-related factors already known to be important predictors of outcome. The conceptual model was used to inform (1) the literature search, (2) development of inclusion and exclusion criteria, (3) data extraction and (4) evidence synthesis.

Literature searching

A range of search methods, as outlined below, were used in order to identify evidence to answer the rapid review research questions in a timely fashion:

• Stage 1 – search of health and medical databases.

• Stage 2 – citation searching.

• Stage 3 – call for evidence from topic experts.

• Stage 4 – scrutiny of reference lists of published reviews/key evidence.

• Stage 5 – scrutiny of reference lists of included papers.

The search process was undertaken with reference to the protocol, in particular to the conceptualisation of the different subareas within which to identify relevant evidence (Figure 1).

FIGURE 1.

Conceptualisation of the evidence base.

A systematic search of medical and health-related databases [MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), The Cochrane Library and Web of Science] was undertaken for the years 2009–14 together with citation searching, reference list checking and recommendations from stakeholders to identify evidence for 2003–14. The rationale for limiting the review to 2003–14 was that this was in line with the dates used by Ewart2 and would limit the body of evidence to a manageable but meaningful number of studies.

Inclusion/exclusion criteria

The inclusion or exclusion of studies in the review was according to the criteria in Table 1.

Assessment according to inclusion and exclusion criteria

References identified from stages 1 and 2 were downloaded into Reference Manager, version 12, to be sifted for inclusion in the review. All potential titles were examined for inclusion by one reviewer. Any titles that did not meet the inclusion criteria were excluded. Following the examination at the title level, any remaining references were scrutinised at the abstract level. For any references where possible inclusion was unclear, a second reviewer independently examined the corresponding full text.

Ten per cent of the titles and abstracts of these citations identified by the searches were checked by a second reviewer (and a check for consistency undertaken).

For stages 3, 4 and 5 references were checked following the same three-stage process as for stages 1 and 2 (title, abstract and full text).

Assessment for inclusion of conference abstracts identified from all stages of the search was undertaken by one reviewer and checked by a second. Both reviewers assessed each conference abstract based on three criteria, namely whether or not:

• the abstract fulfilled the inclusion criteria, in terms of the explanatory variables and outcomes

• the evidence in the abstract was included within an already included paper

• there were sufficient data in the abstract to be able to use the data in a meaningful manner to address the aims of the review.

Data extraction, including development of the data extraction tool

The aim of the data extraction process was to focus on the most critical information for evidence synthesis rather than exhaustively extracting and critiquing all available information within individual papers. Owing to the rapid nature of the review, data extraction was undertaken by five reviewers.

A standardised data extraction form was developed using the following process. The initial draft of the data extraction tool was designed as a comprehensive way to capture all relevant information from the studies on a broad range of factors related to CHD services that may affect patient outcomes following interventions. Four members of the ScHARR review team tested this initial draft on three studies. 6–8

It became apparent that these studies, which focused on the relationship between volume and mortality, considered complexity of the underlying cardiac condition and other patient-level factors in their analysis, but did not include details of relevant organisational factors such as staffing and proximity of related services. Similarly, mortality was the only outcome considered in these studies and other relevant outcomes such as morbidity, complications, length of stay (LOS) and readmissions were not included.

The data extraction tool was therefore revised in the light of this initial data extraction. The revision also included reference to data tables included in other reviews in this area: Ewart2 and Bazzani and Marcin. 8 The final layout was determined to explicitly include the following key details, in addition to the information included as standard on a data extraction form:

• where data were obtained from a database, whether contribution to the database was voluntary (to indicate potential bias in reporting) and whether the purpose of the database was administrative or clinical (to highlight the potential limitations of the details available)

• whether volume was considered as a continuous or categorical variable and, if categorical, what were the thresholds determined by the study for the different categories

• the covariates used in the analysis

• in the quantitative assessment of the relationship between volume/proximity and mortality, a breakdown of the crude association and the adjusted association (for case mix ± other covariates).

• where an association was identified, what was the nature of this relationship (linear or non-linear)?

A sample data extraction form is available in Appendix 3.

Quality assessment

Rather than using a standard checklist approach, the focus was on an assessment of the overall usefulness of the included evidence in answering the research questions. The assessment of usefulness was based on a number of factors which included:

• whether the study adjusted for severity of condition

• whether the study adjusted for age

• whether the study was multicentre

• whether the study included more than one intervention/condition

• whether the contribution to the database used to collect the data was voluntary and whether data were collected comprehensively or collectively.

Assessment of the limitations of included studies was also undertaken using the limitations reported by study authors in the included studies.

Synthesis

Data were extracted and tabulated. This tabulation was used to inform the narrative synthesis in the results section. A meta-analysis was not considered given that the review was a rapid review and there was considerable heterogeneity in the design, methods and setting of the included studies making the clinical value of such a formal statistical analysis open to debate.

Results of the literature search

The full papers and conference abstracts identified as a result of the literature search are described in the modified Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) diagram in Figure 2.

FIGURE 2.

Modified PRISMA diagram. a, This includes seven papers originally included in Ewart. 2

To summarise Figure 2, 39 full journal articles and four conference abstracts met the inclusion criteria. Four additional abstracts met the inclusion criteria; however, the evidence included in them was already included in a full paper. Upon scrutiny, the information included in the abstracts was insufficient for full data extraction and could not be used in a meaningful manner to address the aims of the review. Therefore, a decision was made to extract as much data as possible from these abstracts and include this information for reference in the report appendix but to not include this evidence in the analysis. The tables can be found in Appendix 3.

Second screening of retrieved references

In order to check the screening consistency of the single reviewer a second reviewer screened approximately 10% of the references (n = 300). Reviewer 2 tagged 5 out of 292 (1%) references excluded by reviewer 1 as potential includes and tagged 1 out of 8 (12.5%) references included by reviewer 1 as probable excludes. This gave a kappa statistic of 0.77, generally acknowledged as good agreement. The three additional potential includes identified by reviewer 2 were tenuous includes (two review articles potentially relevant as background, and an article for which only a title was available), whereas the one article tagged as ‘include’ by reviewer 1 and ‘exclude’ by reviewer 2 was subsequently checked for inclusion at the full-text stage. Therefore, it was unlikely that any relevant primary studies were overlooked in the 10% sample checked and this result can be extrapolated to the remainder of the screening process.

List of studies included in the review

Box 1 lists the studies that met the criteria for inclusion for review.

List of conference abstracts included in the review

Table 2 lists the conference abstracts that met the criteria for inclusion in the review.

Characteristics of included studies

Thirty-nine full-text papers were included in the review. The characteristics of these papers are summarised in Table 3.

No UK studies were identified and 36 out of 39 studies (92%) included only paediatric patients. The majority of studies (90%) were conducted in the USA and most were multicentre (92%). We have classified included studies in to three broad groups: those where the primary objective was to explore the relationship between volume of service and mortality outcome for a range of CHD conditions (18/39); those where the focus was on the relationship between volume and mortality outcome for specific single conditions or procedures (14/39); and those where the focus was on the impact of a variable other than volume or where non-mortality outcomes only were reported (7/39). For studies involving specific conditions or procedures these were mainly complex conditions, such as hypoplastic left heart syndrome (HLHS), pulmonary atresia and/or procedures including the Norwood procedure, arterial switch operation (ASO), transposition of great arteries (TGA) and Blalock–Taussig shunt procedure (BTSP) (10/14); heart transplant (2/14); ventricular septal defect (VSD) repair cases only (1/14); and ventricular assist devices (VADs) only (1/14).

Two studies included a paediatric CHD population as a subgroup in studies that examined a range of cardiothoracic procedures38,28 and one a range of common paediatric operations. 12 For these studies only the findings related to the CHD population are reported here. Three procedure-based studies for heart transplant10,16 and VAD32 included patients with conditions other than CHD.

The majority of studies used routine data sets (35/39) and, among these, voluntary clinical or mixed clinical and administrative data sources predominated (21/39), with 13 studies utilising involuntary administrative data. Descriptions of these data sets are provided in Appendix 4. Five studies used study-specific data including one using data from a clinical trial. 40

Half of the studies included children of all ages (age range 0–20 years), 14 out of 39 included only newborns and infants and three studies included adults.

Mortality was the primary outcome measure used, with two studies reporting only morbidity outcomes. The use of routine data is reflected in the types of study design used. There were no primary clinical trials with retrospective observational designs being the predominant feature. There was one before-and-after study assessing the impact of a cardiac paediatric intensive care unit (cPICU). 19

Study populations and settings

Table 4 provides a summary of the dates, inclusion dates, study settings and sample sizes. Where reported, numbers of centres and centre volumes are included. In-hospital mortality is death during the admission for the procedure.

Most of the included studies were conducted after 2009 (29/39, 74%), with 14 studies conducted before 2009. The latter comprised the seven studies included in the Ewart review2 and an additional seven studies identified as a consequence of our broader search strategy and inclusion criteria to include adult studies and those concerned with non-mortality outcomes or the impact of factors other than volume. Fifteen studies (38%) covered time periods of greater than 5 years. Just over half (8/14) of the studies for specific conditions or procedures, in which case numbers will be smaller, utilised data from more than 5 years compared with 28% of studies where all conditions were included. Unsurprisingly, there is a marked difference in sample sizes between studies including all CHD conditions compared with those including highly selected populations based on single conditions or procedures and single-centre studies. Where reported, there are also differences in the centre volumes with studies on specific conditions or procedures having lower-volume thresholds. Among these 14 studies, nine included centres with 20 or fewer cases per year. For studies including all CHD cases, 10 out of 25 had centres with ≤ 200 cases per year and five of these had < 100 cases per year, including two studies with very low-volume centres with < 10 cases per year. 41,28

The primary end point for measuring mortality outcome was within the post-operative period, with 31 out of 37 (84%) studies reporting in-hospital mortality. Seven studies measured mortality at 30 days and four studies measured mortality up to 1 year.

Study analyses: adjustment for confounders and risk

The CHD population is highly complex and varied in terms of both the range of conditions it encompasses and the associated severity and risk of mortality for different conditions. Three CHD risk scores that take account of surgical complexity and associated risk of mortality have been developed for risk adjustment in CHD: Society of Thoracic Surgeons–European Association for Cardio Thoracic Surgery (STS-EACTS), Risk Adjusted classification for Congenital Heart Surgery (RACHS-1) and the Aristotle Complexity score. A detailed description of each score is provided in Appendix 4. Other risk scores do exist for CHD, but have not been used in the studies that have been included in the review. Outcome is also dependent on a range of patient, demographic and service factors that need to be taken into account in study analyses. We extracted details of all covariates used in the analyses of each included study and identified 67 different types of covariate (excluding subgroups within types). Thirty-one (79%) of the studies included a covariate that accounted in some way for the patient’s condition. Of these, 18 used a risk score for surgical complexity, eight a condition descriptor, three a procedure descriptor and two an International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnostic code. Of other covariates, the most commonly used were age (18/39), comorbidity (14/39), sex (13/39) and ethnicity (9/39). Some studies of highly selected groups of patients did not always adjust for common covariates such as complexity (where a single condition was the subject) or age (where the study population was all neonates).

A detailed summary of the 32 covariate types reported in at least 2 of the 39 included studies is provided in Appendix 4.

Overview of main findings

We have summarised the main findings of each included study in terms of whether or not a measurable effect of volume on mortality outcome was reported. Effect is defined as an inverse relationship between volume and mortality, that is increasing volume results in decreasing mortality (or, conversely, low volume is associated with higher mortality). Where survival is reported, the effect relationship is increasing survival with increasing volume and vice versa. Kazui et al. 28 reported an inverse relationship between volume and mortality with higher mortality in low-volume centres, and Sakata et al. 38 found no relationship between volume and morbidity for the CHD subgroup. Both reported wide variation in mortality rates across all volumes and both concluded that risk-adjusted measures are needed to explore this relationship more robustly.

Summary of the evidence about the relationship between volume and outcomes

The evidence reviewed did not include any UK-based studies and is predominantly based on outcomes in paediatric patients. Overall, we have found that although the evidence does demonstrate a relationship between volume and outcome in the majority of studies this relationship is not consistent. Instead there is a mixed picture with both effect and no effect being reported. Studies on single conditions or procedures were more likely to identify an effect of volume on mortality but, given that the focus of these studies was populations of patients with complex conditions and associated surgical procedures that require highly specialised care and expertise, this in itself is unsurprising. The findings from these studies were not unequivocal as even within these highly selected groups there was considerable variation in effect depending on procedure type and individual centre performance. What these studies do indicate is the potential value of centralising or regionalising highly specialised services for very rare and complex cases. However, it cannot be assumed that comparable effects can be achieved for a much broader range of conditions and, therefore, used to define CHD centre volume. It is possible that surgeon volume may be as important as centre volume for these complex cases.

The findings from studies that did consider broader CHD populations were more equivocal. In some studies where an effect was identified, the effect was weak or only demonstrable for specific subgroups of patients. There was no clear indication that the evidence for the volume and mortality relationship was substantially stronger than the evidence for a no effect relationship in these broader groups. The findings further highlight the complex relationship between volume and outcome and the range of other factors which also have an effect. Some of these, such as condition severity, are well established but the effect of association of processes, systems and individual clinical effects on outcome remain unknown.

We also searched for evidence from studies on adult CHD, but this yielded only three papers. One of these studies was concerned exclusively with cardiac transplantation for a range of conditions, not just CHD, so is of limited value other than to provide more general evidence of the potential value of centralising specialist services. The main focus of the other two studies was the effect of surgeon type and both found that adult CHD patients had better outcomes when operated on by paediatric surgeons in specialist children’s centres. Karamlou26 found that outcome was associated with surgeon volume and Kim et al. 29 found a similar association with adult procedure volume indicating the influence of expertise on outcome.

The previous systematic review conducted by Ewart2 included studies published up until 2009. We have included studies considered by that review in this rapid review together with related studies published from 2009 to March 2014. The review by Ewart2 included seven studies and concluded that, while the evidence did suggest there is a relationship between volume and outcome, it is likely that volume is a surrogate marker that encompasses other processes and system factors, the effects of which are unknown. The additional evidence included in this review primarily adds further to our understanding of the complexity of the relationship between volume and outcome. While there is now a larger number of studies reporting a relationship between volume and outcome, these studies also increase the evidence that this is unlikely to be a simple, independent and purely directly causal relationship. The effect of volume on outcome relative to the effect of other, as yet undetermined, health system factors remains a complex and unresolved research question.

Summary of the evidence about the relationship between proximity and outcomes and volume and non-mortality outcomes

We also attempted to identify studies that explored factors related to influencing outcomes in CHD other than the relationship between volume and mortality. This yielded only a small number of relevant papers. Two studies found a benefit in terms of reducing mortality and morbidity in patients cared for in specialist ICUs. One study identified lower mortality for patients requiring ECMO who were cared for in high-volume ECMO units. Two studies on distance to specialist cardiac care found no relationship to mortality. Similarly, we found only two studies in which the primary objective considered the effect of volume on complications. However, a small number of the studies that examined the volume–mortality relationship also measured morbidity as a secondary outcome. Such a small number of relevant studies does not provide a robust evidence base on related factors but collectively they do highlight that the overriding emphasis of research studies on CHD services has been dominated by measurement of the relationship between volume and mortality and mainly short-term, in-hospital mortality. Care is the product of a complex set of processes, of which volume of activity in any given centre or unit is only one contributor. There appears to be relatively little evidence from studies that attempt to measure the effect of related processes on outcome. The consequences of care, and hence outcomes, are also greater than may be captured by data on short-term mortality. Long-term mortality is also important, as are a range of other important short- and long-term outcomes for survivors including morbidity (for example, complications) physical and neurological functioning and quality of life, and service consequences such as LOS and costs, that seem to have received scant attention. As a consequence, the available evidence base that can inform CHD service design is seriously limited and does not reflect the complex features and relationships that contribute to service provision.

What are the issues that have emerged from the evidence?

We have not conducted a systematic review but in assessing a broader topic range and more current literature we have identified some key themes.

1. There are a range of factors which influence mortality in CHD, and centre volume is only one of them. In our data extraction we recorded variables within studies that were also identified as associated with mortality. This process revealed a wide range of patient, demographic and service factors that also have an impact on outcome. The most influential risk factor for mortality by far is the severity of the condition and the associated surgical complexity needed to treat that condition. Where an effect of volume on mortality was measured, in general, this tended to be greater in high-risk patients, as illustrated by the studies on complex single conditions. This is further supported by some of the studies that included broader CHD populations. It is reasonable to assume that complex high-risk surgery requires high-level surgical expertise. A small number of studies have attempted to try to disentangle the effects of individual surgeon performance on outcome but with mixed results. This requires further exploration as this complex relationship of what has an effect – a high volume of complex procedures in a centre or a high volume of complex procedures by an individual surgeon – is still unclear. Furthermore, there is some evidence27 that it cannot be assumed that a high level of technical competence in one complex procedure translates across a range of conditions.

2. Medicine moves forward and clinical advances, training, increasing expertise and changes in service provision mean outcomes for CHD have also changed over time. Five studies that analysed data over long time periods (≈ 10 years) measured changes in mortality over time and found that, irrespective of other factors including volume, mortality decreased despite increasing complexity8,18,30,41,44 illustrating ongoing clinical improvement. What this also means is that the relevance of findings from historical studies or more recent studies that have used historical data will not reflect current care and clinical improvements, so relevance to contemporary services needs to be considered. This observation also has implications for future research. The most recent study by Welke et al. 42 attempted to establish the case volume thresholds needed to detect changes in mortality and concluded that some individual procedures occurred too infrequently or mortality rates were too low to reliably use mortality as a measure of between-centre performance. If clinical advances continue to improve survival, this principle will need to be borne in mind.

3. Although aggregated data may show a difference in mortality rates between low- and high-volume centres, such aggregation may mask between-centre variation. The studies by Gray et al. ,21 Pasquali et al. ,34 Karamlou et al. 27 and McHugh et al. 30 all identified variation between centres, with some low- or medium-volume centres performing equally as well as those with high volume. These studies acknowledged that there are likely to be other centre effects such as training, management protocols, expertise, teaching hospitals, availability of services, composition of care teams and quality programmes that influence outcome. As a result it is unclear whether it is volume or these other effects that are influencing outcome.

4. The evidence base available to guide UK decisions on service design and configuration for CHD is dominated by retrospective and uncontrolled studies conducted within the USA. A noteworthy absence is the lack of any relevant large, well-designed UK multicentre studies. The extent to which the reported findings are generalisable and relevant to the UK setting is therefore limited. In the USA, services are organised very differently to the UK. Key differences include geography and, therefore, distances to specialist care; multiple providers of health care, which means variation in organisation of services, for example numbers of units within different counties and states; and complex health service financing models. Many of the studies have analysed centres with very low volumes of cases – for very rare complex cases the volume of cases may be less than five a year and for broader CHD services some studies have included centres treating fewer than 20 cases a year.

5. Elsewhere and in line with other specialist services there has been a move to centralisation or regionalisation of CHD services, particularly in Europe. 51,52 In the UK, CHD services for children are already regionalised, so evidence on the relationship of very low-volume centres on mortality has little relevance to decision-making about services which are already highly centralised. However, CHD services for adults are less centralised, so decision-making relating to service provision may be informed by evidence relating volume and outcomes.

6. It is axiomatic that, with this centralisation, there is also a corresponding increase in volume as more cases are concentrated in fewer centres but centres will also be characterised by the range of factors associated with service provision discussed previously. It remains unclear whether the impact of volume on outcome is largely a consequence of higher-volume units organising and providing a complex service and high-quality service with all the right components that would be expected to reduce risk, or an independent factor directly related to the advantages of dealing with a larger number of cases. For example, staff may have more experience of specific procedures and potential complications. It is the individual and combined effects of these complex factors on clinical outcomes for patients that remain to be unpicked. Without this better understanding the appropriate interpretation of the observed volume–outcome relationship remains unclear. There is also a lack of evidence about the effects of service factors such as proximity to specialist services and the impact of care on outcomes other than mortality.

7. Despite the growing number of studies on the relationship between volume and outcome, few studies have suggested what the optimum size of a CHD centre in terms of volume should be. Fewer than half of the included studies analysed volume as a continuous variable (14/35 relevant studies), which would provide the most robust evidence from which to consider volume thresholds. Analyses conducted with volume as a categorical variable carry several limitations in informing decisions about volume thresholds in terms of both decisions about within study thresholds and the questionable robustness of the findings. This is particularly the case when comparisons have been made between very high- and very low-volume centres only. Dinh and Maroulas18 suggested that the inverse relationship between volume and outcome detected in their modelling study on 10 years of data was sufficiently robust to allow calculation of volume thresholds. However, these authors did not go as far as identifying what this should be. Hirsch et al. 23 suggested that a reasonable threshold for referral of children requiring the Norwood procedure is centres doing at least 20 procedures a year and 10 procedures a year for ASO. Bazzani and Marcin8 constructed scatterplots of volume against mortality and found no obvious threshold for centre volume. The review by Ewart2 considered the data presented by Welke et al. 6 and suggested a possible threshold of 200–250 cases per year. Welke et al. 6 clearly expressed the view that volume is likely to be a surrogate for the processes and characteristics of care systems that produce outcomes and that centre-specific quality measures would be more informative than volume thresholds. Pasquali et al. 34 and Vinocur41 concurred with this view and suggested that service design decisions should be guided by a range of individual centre performance measures and not volume. There are consistent and clear messages within the literature we have reviewed about the danger of viewing volume in isolation. Furthermore, included studies also caution concerning the likely, but as yet poorly understood, interaction of volume with the numerous other clinical and structural dimensions that contribute to delivering high-quality services and, hence, good outcomes. Finally, questions still remain concerning what volume should be the item of consideration: is it whole-service volume, complex procedure volume or individual surgeon volume that should direct decisions?

Methodological limitations of the included studies

Contributions of authors

Ms Janette Turner (Senior Research Fellow) contributed to proposal writing, undertook data extraction and led on the evidence synthesis.

Dr Louise Preston (Research Associate) contributed to the proposal writing, designed and ran the literature search, contributed to report writing and managed the project.

Dr Andrew Booth (Reader) undertook citation searches, contributed to the proposal writing, assessed evidence for inclusion in the review, proofread the final report, constructed summary tables, assessed the methodological limitations of the included studies and was the chief methodologist on the review.

Mr Colin O’Keeffe (Research Fellow) undertook data extraction and contributed to evidence synthesis through the production of summary tables and other key tables.

Mrs Fiona Campbell (Research Fellow) undertook data extraction and contributed to the construction of summary tables.

Dr Amrita Jesurasa (Honorary Clinical Lecturer in Public Health) undertook data extraction and report writing.

Dr Katy Cooper (Research Fellow) undertook the double-sifting and contributed to the proposal writing and construction of summary tables.

Professor Elizabeth Goyder (Professor of Public Health) was the senior lead on the project, contributed to the proposal writing and undertook the sifting of conference abstracts and construction of summary tables.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health.

Publications

There are currently no publications associated with this rapid review.

Appendix a: conceptual framework

(PDF download)

Appendix b: proposed search strategy (based on Ewart2)

(PDF download)

Appendix c: references

(PDF download)

Appendix a: stage 1 – database search strategy

Appendix b: stage 2 – citation searching

Citation searches were conducted on Google Scholar (14 February 2014) for any references citing any of the following eight studies included in the Ewart review:

1. Bazzani and Marcin8

2. Chang et al. 7

3. Checchia et al. 15

4. Hirsch et al. 23

5. Tsang et al. 53

6. Welke et al. 44

7. Welke et al. 6

8. Welke et al. 43

One hundred and eight-four individual citations (from an initial combined set of 366) remained following de-duplication and removal of non-English references.

Appendix c: stage 3 – evidence suggested by stakeholders and reasons for inclusion/exclusion

Appendix d: stage 4 – references of reviews and other reports used as a source of evidence

Eggli 2010. 94

Ewart 2009. 2

Moons et al. 2010. 65

Queensland Government 2006. 58

Tsang and Utley 2009. 53

Appendix e: list of full-text excludes and reasons for exclusion

Appendix a: list of criteria included on data extraction form

• Ref ID study (author, year, country).

• Aim of study.

• Data source/type of data/study design.

• Dates of study.

• Sample size.

• Population characteristics.

• Unit characteristics.

• Procedures included.

• Definition of volume/proximity.

• Type of risk adjustment (none, administrative data, clinical data, clinical data with robust prediction model).

• Covariates used.

• Relation of volume/proximity to mortality.

  • Crude.

  • Adjusted (case mix ± other).

  • Age adjusted.

  • Non-linear versus linear relationship.

• Relation of other characteristics to mortality (covariates used).

• Other outcomes.

• Comments.

• Headline/key messages.

Appendix b: study groupings

Appendix c: Study descriptive tables

Appendix d: data tables

Appendix e: conference abstracts descriptive table

Appendix f: conference abstracts data table

Appendix a: data source description table

Appendix b: risk adjustment for congenital heart surgery (based on Jacobs et al. 2012116)

Complexity stratification tools have seen increasing popularity in the analysis of outcomes associated with congenital and paediatric cardiac surgery, reflecting the fact that so many different distinct types of operations are performed. Since 2002, complexity stratification has been used extensively by the STS-CHD database and the EACTS Congenital Heart Surgery Database.

Appendix c: table of covariates of included studies

Appendix d: assessment of relevance table