Notes
Article history
The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 11/1022/01. The contractual start date was in November 2012. The final report began editorial review in February 2014 and was accepted for publication in October 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
none
Permissions
Copyright statement
© Queen’s Printer and Controller of HMSO 2015. This work was produced by Blank et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
Chapter 1 Background
Demand management defines any method used to monitor, direct or regulate patient referrals. This includes the methods by which patients are referred from primary care to specialist, non-emergency care provided in hospital. This interface between primary and secondary care is a pivotal organisational feature in many health-care systems, including the NHS. In the UK, primary-care physicians act as the gatekeeper for patient access to secondary care and are responsible for deciding which patients require referral to specialist care. Similar models are found in health-care systems throughout the developed world, for example Australia, Denmark and the Netherlands. 1 Elsewhere, self-referral dominates (e.g. France), or the colocation of primary and specialist services leads to a variety of referral pathways (e.g. the USA). As demand outstrips resources in the UK, the volume and appropriateness of referrals from primary care to specialist services has become a key concern within the NHS. Worldwide, shifts in demographics and disease patterns, accompanied by changes in societal expectations and the relationship between professionals and patients (including the influence of the internet), are driving up treatment costs. As a result of this, several strategies have developed to manage the referral of patients to secondary care, with interventions that target primary care, specialist services or infrastructure (such as referral management centres).
Recent reviews of referral management interventions
The effectiveness of interventions to improve outpatient referrals from primary to secondary care has been the subject of a Cochrane review. 1 The Cochrane review searched for only high-quality, controlled studies and found 17 published papers. The authors concluded that there was insufficient evidence on organisational and financial interventions aimed at primary care, and also inconclusive evidence on effective educational interventions. They did, however, suggest that focusing on potentially effective interventions such as secondary care provider-led education activities, structured referral management sheets, enhancement of primary care and in-house second opinions should guide further research. A previous review on the effects of service innovation on the quality and pattern of referrals from primary care predates recent innovations such as referral management centres. 2 This previous review concluded that professional interventions such as guidelines and education, although able to affect clinical behaviour, had limited effect on referral rates, whereas organisational innovations were more likely to affect referral rates. Further to this, Dunst and Gorman3 reanalysed the Faulkner review along with the previous Cochrane review4 and concluded that interventions that more actively involved primary-care physicians were more effective in influencing rates and patterns of referral.
More recently, referral management in the general practitioner (GP) context has been the subject of work funded by The King’s Fund. 5 Their report highlights the concerns of many with regard to the risks of managing demand without taking account of patient safety, acknowledging that referral management has the capacity to increase clinical risk as well as to reduce it. In considering whether or not one approach to referral management is ‘better’ than another, they suggest that ‘light touch interventions’ such as peer review and feedback, alongside the use of guidelines and structured referral sheets, may offer the most cost-effective approach. However, although the report contributes important insights, it does not suggest best practice examples of these interventions or how they would best be implemented in practice.
Theoretical/conceptual framework
It is increasingly recognised that most interventions in health care can be considered to be complex, with individual and organisational factors affecting how and if interventions lead to improved outcomes. 6 This recognition of the complexity of interventions has been accompanied by a corresponding growth in the challenges for standard methods of evaluation and synthesis. Evidence-based practice requires policy-makers and practitioners to have readily available access to information on interventions that have been shown to work or not work, or indeed have the potential to cause harm. Systematic reviews are an established way of exploring the effectiveness of interventions and a cornerstone of evidence-based practice in order to identify, evaluate and summarise the findings of all available research evidence. Methods for carrying out systematic reviews have become increasingly refined, led by Cochrane, the National Institute for Health and Care Excellence (NICE) and the Centre for Reviews and Dissemination which details the formal procedures required. Conventional systematic review methods, however, face challenges in establishing clear intervention-outcome links when complex multifactorial processes are operating, and there are few experimental studies to draw upon.
As much of the international evidence in the area of referral management is observational in nature and lacks control comparators, our work builds on previous reviews by taking broader inclusion criteria (to include all study designs and grey literature, as well as evidence from other industries). The review findings are presented via a conceptual model (a logic model), which details the range of interventions identified, evidence of their effectiveness and factors which may influence how and if interventions lead to demand management outcomes. The work not only explores the effectiveness of interventions for demand management, but also aims to uncover detail of the processes whereby interventions may lead to an impact on health-care systems in order to determine applicability to the UK context.
Logic models
Logic model methods are a form of theory-based evaluation that focus on relating hypothesised links between an intervention and its constituent parts to its outcomes and long-term impacts. Logic models are concerned with examining the processes of implementation, mechanisms of change and participant responses in order to develop hypothesised links or a ‘theory of change.’7 In order to develop a theory of change, it is necessary to understand the moderator and mediator variables in the process. 8 These factors are the key to understanding how an intervention works and how interventions may work in different health-care contexts. Logic model evaluation methods begin by mapping out an intervention and then examining conjectured links between the intervention activities and anticipated outcomes to develop a summarised theory of how an intervention works, usually in diagrammatic form. Outcomes are conceptualised as being the end of a chain of intermediate changes which the evaluation process seeks to track, with each intermediate point predicting the outcomes that may occur in the future. 9 Logic models have been suggested as a means to help to provide a strategic perspective on complex programmes and to understand the relationships between various elements of an intervention and outcomes. 10 In particular, they are recommended for evaluating highly complex, multisite interventions with multiple and/or indeterminate outcomes. 11
The area of referral/demand management has many of the same challenges as other complex interventions. A key issue relates to the diversity of the many different referral management approaches that have been investigated, which involve varying degrees of active intervention in referral systems and processes. Understanding how these interventions operate is important when evaluating applicability between different systems and contexts. Logic model methods are underpinned by a systems perspective and provide a mechanism for evaluating system impacts, and for supporting managers in presenting a logical argument for how and why an intervention will address a specific need. There has been growing interest in applying the approach to evaluation of health care. It has been highlighted, for example, that hospitals need to look at the logistics of their patient-pathway processes and use a systems perspective to examine flows through the process. Referral management entails moving from a system that reacts in an ad-hoc way to meet increasing needs to one that is able to plan, direct and optimise services in order to optimise demand, capacity and access across an area. Uncovering the assumptions and processes within a referral management intervention, therefore, requires an understanding of system operation and assumptions which the logic model methodology is well placed to address.
Research questions
This research was designed to conduct an inclusive systematic review and develop a logic model to answer the following research questions:
-
What can be learned from the international evidence on interventions to manage referral from primary to specialist care?
-
How can international evidence on interventions to manage referral from primary to specialist care be applied in a UK context?
-
What factors affect the applicability of international evidence in the UK?
-
What are the pathways from interventions to improved outcomes?
Chapter 2 Review methods
A review protocol was developed for the project and can be found at www.nets.nihr.ac.uk/__data/assets/pdf_file/0007/81178/PRO-11–1022–01.pdf.
Inclusion and exclusion criteria
Participants: all primary care medical physicians, hospital specialists and their patients.
Interventions: interventions that aim to influence and/or affect referral from primary care to specialist services by having an impact on the referral practices of the primary physician; in addition, interventions that aim to improve referral between specialists or have the potential to impact on primary care to specialist referrals.
Comparators: the main comparator condition for intervention studies was the usual method of referral practice which is undertaken in the location where the intervention is being implemented. However, alternative comparators have not been excluded. We also included studies with no concurrent comparator (e.g. non-controlled before-and-after studies), as well as qualitative studies where comparators are not relevant.
Outcomes: all outcomes relating to referral were considered, including referral rate, referral quality, appropriateness of referral, impact on existing service provision, costs, mortality and morbidity outcomes, length of stay in hospital, safety, effectiveness, patient satisfaction, patient experience and process measures (such as referral variation and conversion rates). All qualitative outcomes were also considered for the relevant papers.
Study design: with the increasing recognition in the literature that a broad range of evidence is needed to inform review findings, no restrictions were placed on study design. The criterion for inclusion in the review was that a study is able to answer or inform the research questions. We have, however, taken note of how quality of study design and execution may affect the reliability of the results generated, as discussed below.
Identification of evidence
Search strategy
Searches were limited by date (January 2000 to July 2013). Articles generated by our searches that consisted of English abstracts only, with full papers published in other languages, were considered for translation, but none was found to meet the inclusion criteria for the review. Our international collaborators did not identify any key articles in other languages, which might have required translation.
All of the literature identified using the above methods were imported into Reference Manager Version 12 (Thomson ResearchSoft, San Francisco, CA, USA) and key-worded appropriately. An audit table of the search process was kept, with date of search, search terms/strategy, database searched, number of hits, keywords and other comments included, in order that searches were transparent, systematic and replicable. Searches took place between November 2012 and July 2013. Search strategies and a full list of data sources are given in Appendices 3 and 4.
At the outset of the project a steering group of our international collaborators, relevant patient representatives and other stakeholders was formed. This group had the opportunity to suggest terms to be considered for inclusion in the initial search strategy as well as identifying key articles for potential inclusion.
Initial search
Systematic searches of published and unpublished (grey literature) sources from health care and other industries were undertaken to identify recent, relevant studies. An iterative (i.e. a number of different searches) and emergent (i.e. the understanding of the question develops throughout the process) approach was taken to identify evidence. 12,13
An initial search was generated to address the project research questions, with free-text and subject-heading terms combined to address the concepts of ‘primary care’ and ‘referral’. A broad range of electronic database, including MEDLINE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO and Health Business Elite, was searched in order to reflect the diffuse nature of evidence (see Appendices 3 and 4).
Databases that focus on health management literature, such as the Health Management Information Consortium and Health Business Elite, and management databases such as Business Source Premier and Emerald Management Reviews, were also searched using the initial search strategy.
Additional searches
After the initial search a phrase search was undertaken for ‘referral management centres’ in MEDLINE and CINAHL (for full details of data sources see Appendix 3). This was to make sure that papers had not been missed which described this particular referral method.
As the work progressed, further searches were required in order to seek additional evidence where there were gaps and implicit assumptions that particular outcomes would result following interventions described later.
Citation searches
Citation searches of included articles and systematic reviews were undertaken in the Science Citation Index and Social Science Citation Index and respective conference papers indices. Where a search returned no results, a search in Scopus was undertaken to double check for any registered citations. Relevant reviews articles were also used to identify studies.
Grey literature
Grey literature (in the form of published or unpublished reports, or data published on websites, in government policy documents or in books) was searched for using the OpenGrey (www.opengrey.eu), Greysource (www.greynet.org) and Google Scholar (http://scholar.google.com; Mountain View, CA, USA) electronic databases.
Reference list checking
Hand-searching of reference lists of all included articles was also undertaken, including relevant systematic reviews.
Selection of papers and data extraction
Citations were uploaded to Reference Manager, and titles and abstracts (where available) of papers were independently screened for inclusion by two reviewers, with disputes resolved by consulting other team members. Full-paper copies of potentially relevant articles were retrieved for systematic screening. A data extraction form was developed using the previous expertise of the review team, trialled using a small number of papers and refined for use here. Data extractions were completed by one reviewer and checked by a second.
Extraction data included country of the study, study design, data collection method, aim of the study, detail of participants (number; any reported demographics), study methods/intervention details, control details, length of follow-up, response and/or attrition rate, context (referral from what/who to what/who), outcome measures, main results and reported associations between elements for the logic model.
Data synthesis
The heterogeneity of the interventions’ aim, design and outcome measures used precluded a meta-analysis of their results. We therefore completed a narrative synthesis of the data, primarily in terms of type of intervention and outcomes. In addition, we built on our previous methodological work14,15 and thematic synthesis methods,16 and used the data to develop a diagrammatic representation (logic model) of the factors that may influence the pathway from interventions to system-wide impacts. The model aimed to portray how interventions operate in order to change practice at individual, local and system-wide levels.
Quality appraisal
Individual studies
The critical appraisal of included evidence is a key part of the review process; however, it is the subject of debate in the field, with no single recognised tool. There is also variation in views regarding the use of scoring systems, with Cochrane discouraging the use of systems which total elements on a checklist, as a single item may jeopardise an entire study. In this review, the quality of studies was assessed using a checklist based on work by Cochrane (see Appendix 2). This approach considers risk of bias and, as it is usually used with experimental studies, required some modification for use with our wider range of study designs. Qualitative papers were evaluated using an adaptation of the Critical Skills Appraisal Program tool. Each paper was assessed by one reviewer and checked for accuracy by a second. Each paper was graded on a three-point scale as being at higher risk of bias, lower risk of bias or unclear risk of bias. The rating was based on not only an aggregate (the number of items) but also an overall judgement of risk of bias. It is important to note that our rating was comparative (higher vs. lower) across the set of papers, with a study classed as being at lower risk not meaning that it was necessarily low risk (see the assessment of each study detailed in Appendix 2). Study design criteria for inclusion in the review were not set as the work was intended to be broad-based and inclusive. Inclusion required only that the paper was able to answer the research question; however, we took account of quality standards in the synthesis and presentation of the evidence as will be outlined below.
Appraising the strength of the evidence
Although there is debate regarding rating of quality of individual studies, there is also considerable variation in views regarding methods for appraising strength of evidence across studies, with a higher number of papers in an area indicating not necessarily greater strength of evidence but only that more work has been carried out. We adopted a system that combined consideration of volume of evidence, and also consistency of evidence, with quality of evidence, based on work by Hoogendoorn et al. 17 Evidence strength appraisal was undertaken by the research team at a series of meetings to establish consensus. Each group of papers was graded as (i) stronger evidence, (ii) weaker evidence or (iii) inconsistent/no evidence.
Stronger evidence (i) was defined as generally consistent findings in multiple higher-quality studies.
Weaker evidence (ii) was defined as generally consistent findings in one higher-quality study and lower-quality studies, or in multiple lower-quality studies.
No evidence or inconsistent evidence (iii) was defined as only one study available or inconsistent findings in multiple studies. Study findings were considered to be inconsistent if fewer than 75% of studies reported the same conclusions.
Validation and applicability of the findings
Following completion of the evidence appraisal and draft logic model synthesis, we undertook a period of stakeholder consultation to seek feedback on the evidence that we had identified and the applicability of the findings to the UK health-care context. This consultation was carried out via presentations to practitioners and patient representatives, via individual meetings to discuss the findings, and by circulating the model to experts in the field (including practitioners, commissioners and academics). In total, 44 individuals contributed to this validation stage. In order to assess how our findings resonated with other work in the field, we also carried out a review of other reviews in the area.
Chapter 3 Results of the review
Quantity of the evidence available
In total, our searches generated a database of 8327 unique papers. Of these, 580 papers were selected for consideration at the full-paper stage. After considering these, searching reference lists and completing the validation stage of the project, 290 full papers were included in the review (Table 1). 18–308 The included papers consisted of 140 intervention papers and 150 non-intervention papers (looking at the views of patients and professionals on the referral process, and factors which predict referral). The 150 non-intervention papers included qualitative studies (n = 33) and non-intervention quantitative studies such as surveys and research reporting associations (n = 117). Grey literature searches generated 69 potentially relevant articles but no additional articles were subsequently found to be within the scope of the review. This was probably due to the fact that a number of grey literature reports had already been identified in the previous searches.
| Source | Number of hits | Number of papers included |
|---|---|---|
| Initial searches | 6431 | 253 |
| Additional searches | 876 | 7 |
| Citation searches of included papers | 814 | 16 |
| Reference list of included papers and systematic reviews | 137 | 12 |
| Grey literature | 69 | 0 |
| Validation stage | 1 | 1 |
| Total | 8328 | 290 |
Of the intervention papers, 114 were identified through the initial database searches, 14 were identified through citation searches, one was identified through additional targeting searching and 10 additional papers were identified through scrutinising reference lists (including those of systematic reviews). One further study was identified at the validation stage of the logic model.
Of the non-intervention studies, 140 were identified through the initial database searches, two were identified through citation searches and six were identified through additional targeting searching, with two additional papers identified through scrutinising reference lists.
In addition, 30 systematic review papers in relevant topics were identified and a synthesis of these was developed in parallel with, but independently to, the logic model development. Comparison with the logic model synthesis is considered in Appendix 6 of this report as part of the validation stage.
We excluded a total of 286 papers which were obtained as full papers but were subsequently found to be outside the scope of the review. A list of these papers and the reasons for their exclusion are given in Appendix 5. Figure 1 details the process of identification of studies.
FIGURE 1.
The process of identification of studies.
Quality of the evidence available
Of the 140 intervention studies, the vast majority (n = 126) were considered to be at lower risk of bias. 19,21–24,27–40,43–71,73–89,92–96,98–100,102–139,141,142,144–150,152,156–160 Fifteen intervention studies were considered to be at higher risk of bias,25,26,42,72,90,91,97,129,140,143,151,153,154 including two studies where the risk of bias was unclear. 19,41 The main risks for bias related to a lack of participant details, only narrative results, percentages reported without supporting statistics, data reported as charts only, inconsistencies in data reporting, poor response rates, attrition rate not reported, weak outcome measures, unclear study design, and evaluation tools which asked questions that strongly led respondents towards positive answers.
Of the 33 qualitative studies, 32 were considered to be at lower risk of bias. 176,177,182,192,194,201,204,207,209,210,212,213,217,218,221,226,228–230,232,237,239,249,252,253,256–258,273,293,306 Only one was considered to be at higher risk of bias due to unclear aim, unclear process for selection of participants and data not clearly distinguished from report of other authors’ work. 20
Of the 117 non-intervention qualitative studies (surveys, etc.), 96 were considered to be at lower risk of bias,98,101,138,161,163–181,183,187–189,191,193,195–200,206,211,215,216,219,220,222,223,225,231,234,235,238,240–243,245–248,250,251,254,259–270,272,274–276,278–294,297,299–305,307 with 21 studies considered to be at higher risk of bias. 162,165,184–186,190,202,203,205,208,214,224,227,233,236,244,255,271,277,295,306,308 The main risks for increased bias were attributable to studies being completed in one small sample only, limited recruitment details, poor response rate, leading questions, recall bias, unpiloted survey tools, unclear methods, limited data presentation, possible overstatement of findings and over-reliance on self-reported outcomes.
Although the higher-risk studies were not excluded from the synthesis and model, the risk of bias was accounted for in assessing the strength of evidence for each element of the model. The detailed quality assessment for each study is provided in Appendix 2.
Study designs
Of the 140 intervention studies, there were 44 randomised controlled trials (RCTs)23,26,27,29–32,36,39,53,54,58–60,63–68,76,77,79,82,85–87,92–95,107,109,111,114,116,117,120,125,126,131,135,144,159 (including 19 of cluster design30–32,39,53,58,63,65–68,77,79,86,111,114,117,120,131), five non-RCTs (nRCTs),62,108,127,130,134 43 before-and-after studies (without a concurrent control group),24,33–35,38,42,43,45,47–52,55,57,69,72–74,89,90,102,103,105,110,112,115,119,122,129,133,136,137,143,145,146,149,154,156–158,160 three controlled before-and-after studies,56,70,81 one case–control study,57 one economic analysis,151 five cohort studies28,46,71,104,128 and 38 evaluation studies (described variably as audits, review, evaluation and retrospective data analysis). 18,19,21,22,25,27,40,41,44,61,75,78,80,83,84,88,91,97–99,106,113,118,121,123,124,132,135,138,140–142,147,148,152,153,155,158
Of the non-intervention views and predictors studies, the 33 qualitative studies consisted of qualitative interview studies (n = 2520,163–165,171,177,178,180,183,192,194,196,201,204,207,210,212,213,237,239,245,249,253,258,260), focus group studies (n = 5217,230,232,252,257), studies using both interviews and focus groups (n = 2196,239) and one study which used transcriptions of video tapes. 182 The non-intervention quantitative studies (n = 117) were mostly cross-sectional surveys (n = 8229,108,161,168–175,178,179,181,183–185,187–191,193,195,198,200,202,203,205,206,208,209,211,214–216,219,220,222,224,225,227,231,232,234–236,238–240,242,244,246,248,250,251,259,261,263,264,268–282,284–287,289,291,292). In addition, one study employed a follow-up survey; two studies used surveys and interviews,176,186 and one further study also included a focus group. 233 There were also 29 studies which consisted of an analysis of patient records, documents, case notes, admissions data and referral forms. 138,166,167,173,197,219,223,235,241,243,254,256,263,265–267 Most of these studies (n = 23) were retrospective designs, but four employed a prospective cohort design. 173,223,254,266 In addition, one study employed Delphi methods196 and one final study used a group-based assessment of referral appropriateness. 255
Populations and settings
Of the 140 interventions, the majority were conducted in the UK (n = 8218,19,21–23,26,28,30–32,34,37,38,41–62,64,65,68,70,71,73,74,76–80,82–85,94,96,99,103,104,106,109,114,116,117,119,122,124–126,128,129,131,133,139,140,142,143,152–157,159,160) or the USA (n = 2024,33,63,87,89,93,98,100,102,112,115,121,132,138,144–147,155,158). There were 10 studies from the Netherlands36,67,86,90,120,123,134,135,141,149 and nine from Australia. 49,72,91,97,105,111,118,136,148 Additional studies were conducted in Canada (n = 327,107,110), Israel (n = 3130,137,150), Italy (n = 369,113,127), Denmark (n = 229,92), Spain (n = 235,75), Finland (n = 195), Norway (n = 1151), Hong Kong (n = 181) and UK/China (n = 125), with one final study where the country of origin was unclear. 101
Of the non-intervention views and predictors studies, the 33 qualitative studies were conducted mostly in the UK (n = 18177,180,182,192,194,201,204,207,209,210,218,228,229,249,252,253,257,258), with additional studies from Australia (n = 5169,176,221,226,245), USA (n = 5170,183,200,202,208), the Netherlands (n = 3212,237), Norway (n = 2164,217), New Zealand (n = 120) and Belgium (n = 1230). The non-intervention quantitative studies (n = 117) were mostly from the UK (n = 35157,174,175,177,187,189,190,193,195,197,198,207,220,224,233,236,241–243,247,251,254–256,265,266,272,273,279,282,284,285,287,291,294) and USA (n = 3198,108,138,171,172,178,184,200,205,214,216,218,219,222,223,225,231,232,235,238,240,246,260,264,267,270,271,274,277,283,286,290,304,305,307), with additional studies from Canada (n = 13107,165,179,196,203,206,227,234,248,263,275,292,299), Australia (n = 1040,91,105,148,162,185,186,188,215,268), the Netherlands (n = 4163,191,212,250), Norway (n = 4164,168,239,244), Israel (n = 3167,261,269), Germany (n = 2173,211), Denmark (n = 229,181), New Zealand (n = 2288,302), France (n = 1161), Ireland (n = 1280), Belgium (n = 1209), Lithuania (n = 1166) and Spain (n = 1276). In addition, two studies were conducted in more than one country, namely the UK/Australia (n = 1169) and USA/Canada/Puerto Rico (n = 1183).
Types of interventions
In total 140 intervention papers were identified and used to create a typology of studies by intervention type. The intervention studies identified may be grouped into four categories: GP education interventions (n = 4919,21,22–69); process change interventions (n = 4770–87,98–120); system change interventions (n = 4118,121–157); and patient-focused interventions (n = 3158–160). It is accepted that this grouping of interventions may have some overlap; however, focus is on the content. Table 2 provides a summary of the intervention studies grouped by typology.
| Intervention category | Intervention type | Studies reporting a positive effect on referral outcomes (first author and year) | Studies reporting no effect on referral outcomes (first author and year) | Strength of evidence |
|---|---|---|---|---|
| GP education | Peer review and training/feedback | Cooper 2012,19 Evans 2009,21 Evans 2011,22 Jiwa 200423 | i | |
| GP training: professional development | Adams 2012,33 Bennett 2001,30 Donohoe 2000,31 Hands 2001,34 Hilty 2006,24 Kousgaard 2003,29 Ramsay 2003,27 Suris 2007,35 Watson 2001,32 Wolters 200536 | Bhalla 2002,37 Ellard 2012,38 Emmerson 2003,40 Lam 2011,25 Lester 2009,39 Rowlands 2003,26 Shariff 201028 | iii | |
| Guidelines (no training/feedback) | Cusack 2005,43 Idiculla 2000,44 Lucassen 2001,45 Malik 2007,41 Imkampe 2006,47 Potter 2007,46 Twomey 200342 | Fearn 2009,48 Hill 2000,49 Matowe 2002,50 Melia 2008,51 West 200752 | iii | |
| Guidelines with training/feedback/specialist support | Banait 2003,53 Eccles 2001,54 Elwyn 2007,55 Glaves 2005,57 Griffiths 2006,58 Julian 2007,62 Kerry 2000,59 Robling 2002,60 Walkowski 2007,63 White 2004,61 Wright 200656 | Dey 2004,66 Engers 2005,67 Jiwa 2006,68 Morrison 2001,64 Spatafora 2005,69 Wilson 200665 | iii | |
| Process change | Direct access to screening/diagnostic testing | DAMASK 2008,76 Shaw 2006,77 Simpson 2010,78 Thomas 2003,79 Thomas 2010,80 Wong 200081 | Dhillon 2003,82 Eley 2010,83 Gough-Palmer 200984 | iii |
| Designated appointment slots/fast-track clinic | Bridgman 2005,70 Hemingway 2006,73 Khan 2008,71 Sved-Williams 201072 | McNally 2003,74 Prades 201175 | iii | |
| Specialist consultation prior to referral | Eminovic 2009,86 Harrington 2001,93 Hockey 2004,91 Jaatinen 2002,95 Knol 2006,90 Leggett 2004,85 McKoy 2004,89 Nielsen 2003,92 Tadros 2009,96 Wallace 2004,94 Whited 200287 | i | ||
| Electronic referral | Chen 2010,100 Dennison 2006,99 Gandhi 2008,108 Jiwa 2012,105 Kim 2009,98 Kim-Hwang 2010,102 Nicholson 2006,97 Patterson 2004,104 Stoves 2010103 | Kennedy 2012106 | i | |
| Decision support tool | Akbari 2012,110 Emery 2007,111 Junghams 2007,109 Knab 2001,112 Mariotti 2008,113 McGowan 2008107 | Greiver 2005,114 Magill 2009,115 Slade 2008,117 Tierney 2003116 | iii | |
| Waiting list review | Stainkey 2010118 | King 2001,119 van Bokhoven 2012120 | iii | |
| System change | Community provision of ‘specialist’ services by GPs | Callaway 2000,121 Ridsdale 2008,124 Salisbury 2005,125 Sanderson 2002,126 Sauro 2005,127 Standing 2001,122 Van Dijk 2011123 | Levell 2012,129 Rosen 2006128 | i |
| Additional primary care staff | Simpson 2003,143 Van Dijk 2010,141 White 2000142 | i | ||
| Outreach: community provision by specialists | Campbell 2003,131 Felker 2004,132 Gurden 2012,133 Hermush 2009,137 Hughes-Anderson 2002,136 Leiba 2002,130 Schulpen 2003,134 Vlek 2003135 | Johnson 2008,139 Pfeiffer 2011138 | i | |
| Return of inappropriate referrals | Tan 2007,140 Wylie 200118 | ii | ||
| Gatekeeping | Ferris 2001,145 Ferris 2002,146 Joyce 2000,147 Schillinger 2000144 | iii | ||
| Payment system | McGarry 2009148 | Iversen 2000,151 Van Dijk 2013,149 Vardy 2008150 | iii | |
| Referral management centre | Maddison 2004,154 Watson 2002,152 Whiting 2011153 | Cox 2013,156 Ferriter 2006,157 Kim 2004155 | iii | |
| Patient inventions | Patient education | Lyon 2009160 | Heaney 2001159 | iii |
| Patient concerns and satisfaction | Albertson 2002158 | iii |
General practitioner education interventions
The GP education intervention group included peer-review and feedback (n = 4) interventions, which consisted of formal GP training (including continued professional development) (n = 17) and the issuing of guidelines [with (n = 18) and without (n = 11) additional formal training and support for practitioners].
Peer review
Peer-review training/feedback was offered to GPs (plus advanced health-care practitioners and practice managers) in one study19 either in face-to-face meetings19,21,22 or via written feedback. 23 Follow-up was for a minimum of 1 year in all cases. Details of each study are outlined in Table 3.
| Study | Intervention | Design | Country | Specialty | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Cooper 2012 19 | Face-to-face peer review | Audit | UK | Orthopaedics | NR | 5 years |
| Evans 200921 | Face-to-face peer review | Audit | UK | Emergency, orthopaedics | Nine GP practices | 1 year |
| Evans et al. 201122 | Face-to-face peer review | Audit | UK | Seven specialties | 10 GP practices (53 GPs) | 1 year |
| Seven specialties | ||||||
| 21 female GPs, median aged 44 years | ||||||
| Jiwa et al. 200423 | Written peer review | nRCT | UK | Specialists | 26 GPs in intervention group | 18 months (6 months) |
Two studies were at lower risk of bias. Evans21 reported, on average, a significant drop in referrals between the first and fourth quarters (z = 2.25, p = 0.025). The quality of referrals as judged by doctors’ peers improved and referral rates in orthopaedics showed a reduction of up to 50%. However, variability between practices decreased and referral to local services increased. In 2011 they further reported a reduction in variation in individual GP referral rates (from 2.7–7.7 to 3.0–6.5 per 1000 patients per quarter), and a related reduction in overall referral rates (from 5.5 to 4.3 per 1000 patients per quarter). 22 Although the highest individual referrers showed a decrease, the lowest referrers may show an increase in referrals [and a significant negative correlation comparing the first month’s data with the change from first to last month (r = 0.719, p = 0.019)]. 22 Jiwa et al. 23 reported a difference of 7.1 points [95% confidence interval (CI) 1.9 to 12.2 points] in the content scores between the feedback group and the controls after adjusting for baseline differences between the groups. There was a considerable improvement in the content of the referral letters from the feedback group from before to after feedback (mean score 34.1 vs. 39.5). There was no improvement in the scores for the control group in the same period [mean score 34.1 vs. 28.2; mean difference 5.3 (95% CI 1.5 to 9.2)/mean difference 0.55 (95% CI –1.4 to 2.5); t-test degrees of freedom (df) 20/36; p = 0.008/0.6].
One further study was at higher risk of bias. Cooper19 conducted a peer-review scheme for referrals with two guiding principles: the review would benefit the practice and the commissioning group; and there was no blame. GPs, nurses, advanced health-care practitioners and practice managers attended a workshop event and each practice bought two or three trauma and orthopaedic referral letters. Participants worked at mixed tables to understand each practice’s referral profile, share how each practice would handle each situation and then identify any gaps or areas of changed needed. As a result they reported that trauma and orthopaedic expenditure in 2010–11 was 17% less than in 2006–7; in addition, one practice cut ear, nose and throat (ENT) referrals by 20% in the first year and 40% overall.
Formal general practitioner training
Seventeen interventions consisted of formal GP training. Overall, 11 studies reported a positive impact on referral,24,27–36 with six showing no effect or a negative change. 25,26,37–40 Three studies were considered to be at higher risk of bias. 24–26 Overall, the strength of this evidence was graded as inconsistent.
The interventions themselves were varied and it was challenging to separate them further for analysis given the diversity of the interventions delivered. However, seven interventions were delivered in one single session (Table 4) and 10 sessions were delivered over a number of weeks or months (Table 5). The single-session interventions consisted of educational reminders added to radiographs requested by GPs;27 an educational module and 12-page printed guide;28 a structured information pack sent to GPs when their patients attended the department of oncology for the first time;29 an education video;30 in-practice education session plus information pack;31,32 and a 1-day interactive chronic obstructive pulmonary disease (COPD) programme. 33
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Adams 201233 | One-day CME | BA | USA | COPD | 351 primary care clinicians | (3–6 months) |
| Bennett 200130 | Video; checklist | cRCT | UK | ENT (glue ear) | 50 practices | (1 year) |
| 177 GPs | ||||||
| Donohoe 200031 | Practice visits; leaflets | cRCT | UK | Diabetic foot | 10 towns | (6 months) |
| 1939 patients | ||||||
| Aged 18+ years | ||||||
| Kousgaard 200329 | Information pack to GPs on first referral | RCT (unblind) | Denmark | Oncology | 248 patients | NR |
| 199 GPs | ||||||
| Ramsay 200327 | Educational reminders on radiographs | RCT | Canada | Radiology (knee and spine) | 81 GP practices | 12 months |
| 2324 referrals | ||||||
| Shariff 201028 | Educational module | Cohort | UK | Oncology (skin cancer) | 460 referrals | 15 months (12 months) |
| Watson 200132 | Practice education session ± information pack | cRCT | UK | Oncology (familial breast/ovarian cancer) | 170 GP practices | 9 months |
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Bhalla 200237 | Three or four ENT sessions over a 2-week period once a year | Case control | UK | Otolaryngology (ENT) | Two GP practices | 3 years |
| 1073 referrals | ||||||
| One partner in each GP practice | ||||||
| Ellard 201238 | Six 2-hour interactive sessions on common skin conditions | CBA | UK | Dermatology | 30 GPs from 26 practices | (3 months) |
| Emmerson 200340 | Psychiatric appointments in primary care | Audit | Australia | Psychiatry | Five psychiatrists, 200 GPs | 1 year |
| Hands 200134 | GPs trained at outpatient sessions | BA | UK | All specialties | 22 consultants, 21 GPs | (6 months) |
| Hilty 2006 24 | Regular CME peer review; consultation notes for GPs | BA | USA | Psychiatry | 400 consultations | NR |
| Lam 2011 25 | Diploma in Community Geriatrics | CX | UK/China | Geriatrics | 98 GPs | 1 year |
| Lester 200939 | Video, question and answer, two refresher sessions | cRCT | UK | Psychiatry | 179 patients | (4 months) |
| Two GP practices | ||||||
| Rowlands 2003 26 | Educational referral meetings | CX (part of RCT) | UK | All specialists | 13 GP practices | NR |
| Four or more partners | ||||||
| Suris 200735 | Biweekly educational sessions by specialists | BA | Spain | Rheumatology | 117 GPs | 1 year |
| Wolters 200536 | Distance-learning programme | RCT | Netherlands | Urology | 142 GPs | (14 months) |
Six of the ‘one-session’ interventions (see Table 4) showed positive effects on referral outcomes and were at lower risk of bias.
Adams et al. 33 delivered a 1-day interactive COPD continuing medical education programme. Knowledge/comprehension significantly improved {mean [standard deviation (SD)] pre-test percentage correct, 77.1% (16.4%); 95% CI 76.2% to 78.9%; and mean (SD) post-test percentage correct, 94.7% (8.7%); 95% CI 94.2% to 95.2%; p < 0.001)}, with an absolute percentage change of 17.6% (13.2%). Of the follow-up survey respondents, 92 of 132 (69.7%) reported completely implementing at least one clinical practice change, and only 8 of 132 (6.1%) reported inability to make any clinical practice change after the programme.
Bennett et al. 30 delivered a training video, a checklist or both to three intervention groups. At 1 year post intervention, there was significant improvement in the positive predictive value, adjusted for patient waiting time between GP referral and appointment at the ENT department. The improvement in positive predictive value pre and post intervention was 15% (95% CI –12.1 to 41.7) for the practices receiving both interventions, compared with 20% (95% CI –32.9 to –6.4) for practices receiving only one intervention and a degradation of 34% for those receiving no intervention.
Donohoe et al. 31 delivered an educational intervention aimed at clarifying management of the diabetic foot, referral criteria and the responsibilities of professionals. The intervention included practice visits and education of the whole practice team. Leaflets outlining patients’ role and responsibility were disseminated to the practices. Appropriate referrals from intervention practices to the specialist foot clinic rose significantly (p = 0.05), compared with control practices (p = 0.14).
Kousgaard et al. 29 provided a structured information pack to GPs when their patients attended the department of oncology for the first time. Intervention group practitioners gave a significantly higher score to the information value of the discharge letter than did control group practitioners. The most pronounced difference was seen for psychosocial conditions (p = 0.001) and information about what the patient had been told at the department (p = 0.001).
Ramsay et al. 27 reported that after 6 months of adding educational reminders to radiographs (adjusting for seasonal variation) the frequency of knee radiographs showed a relative risk (RR) reduction of 0.65 and lumbar spine radiographs showed one of 0.64. The mean number of referrals per practice per month for the control group was 2.97 (SD 3.22) knee and 2.88 (SD 3.05) spine, compared with intervention group mean referrals of 1.87 (SD 2.4) knee and 1.76 (SD 2.38) spine.
Watson et al. 32 randomised 170 practices to group A (receiving an in-practice educational session plus information pack), group B (receiving an information pack alone), or group C (receiving neither an educational session nor a pack). There was a 40% (95% CI 30 to –50, p < 0.001) improvement in the proportion of GPs who made the correct referral decision on at least five of six vignettes in group A (79%) compared with the control group (39%) and a 42% (95% CI 31 to 52%, p < 0.001) improvement in group B (81%) compared with the control group (39%). There was no significant difference between groups A and B.
A further ‘one-session’ intervention was not effective. Shariff et al. 28 delivered an educational module that was aimed at building confidence in the diagnosis of lesions not requiring an urgent referral, especially basal cell carcinomas and seborrhoeic keratoses, referred through the ‘2-week wait’ route. After 11 months, the proportion of appropriately referred skin cancers (squamous cell carcinomas and melanomas) was 20.6%, compared with 23.2% before the intervention. The remaining 10 interventions were delivered over several sessions (see Table 5), although the exact number and timing of sessions was not always well described.
Hands et al. 34 reported an intervention where GPs attended outpatient sessions in different clinical specialties of their choice. GPs reported changes in their clinical behaviour which appear to have been maintained at 6 months. GPs stated that referral was discussed/taught in 83% of interactions. Immediately after the session, 25% of GPs reported that this would change their referral behaviour. After 6 months, 29% reported behaviour change in reference to referral.
Hilty et al. 24 implemented the following educational strategies. (1) Regular continuing medical education lectures. (2) GP participation in consultations: GPs present their patients at the beginning of the sessions, and get direct feedback at the end. (3) Consultation notes for GPs: a note by the psychiatrist was sent within 10 minutes of each consultation in a deliberately educational style. A dictation of two to three pages was sent in about 5 working days. (4) Telephone consultations with the psychiatrist. Among the first 200 consultations, only 47.4% of the medication doses for depressive and anxiety disorders were adequate, according to national guidelines. Among the second 200 consultations, dosing adequacy improved to 63.6% (p < 0.001). GPs rated the quality of consultation as significantly higher over time (95% CI 4.45 to 4.83, p < 0.001), as with overall satisfaction (95% CI 4.49 to 4.73, p < 0025). This study was considered to be at higher risk of bias.
Suris et al. 35 carried out biweekly educational sessions with GPs for 1 year (a total of 120 sessions carried out by four rheumatologists). At the end of the pilot year the total number of GP referrals was 31% lower than the previous year (1141 vs. 1652, no significance levels reported). The referral rate to the rheumatology unit decreased significantly from 8.13 per 1000 to 5.53 per 1000 (2.6, 95% CI 2.09 to 3.10; p < 0.001).
Wolters et al. 36 delivered a distance-learning programme accompanied with educational materials or a control group only receiving mailed clinical guidelines. The distance-learning programme comprised: (1) a package for individual learning developed by the Dutch College of General Practitioners; (2) consultation supporting materials: a voiding diary, the international prostate symptom score (IPSS) and Bother score; (3) the guideline summarised into two decision trees [one on clinical management of lower urinary tract symptoms (LUTS) and one on prostate-specific antigen (PSA) testing] and a brief explanation; and (4) two information leaflets for patients (on PSA testing and on treatment for LUTS). The intervention group showed a lower referral rate to a urologist [odds ratio (OR) 0.08, 95% CI 0.02 to 0.40], but no effect on PSA testing or prescription of medication.
Six further studies delivered over several sessions did not show a clearly positive effect on referral outcomes. Four of these were at lower risk of bias: Bhalla et al. 37 delivered three or four clinical ENT sessions over a 2-week period, once a year for 3 years to one partner in a GP practice. There was no statistical difference in referral rates (Kruskal–Wallis: p = 0.63) for the trained partner when compared with the other three partners in the same practice. There was also no statistical difference in referral patterns between the intervention and the control practice (Mann–Whitney U-test p = 0.50).
Ellard et al. 38 completed six 2-hour interactive sessions on common skin conditions in early 2011. Appropriate referrals from participants increased from 37.2% in 2010 to 51.8% after training, accompanied by an increase in the mean number of referrals from 20.7 to 25.7. Furthermore, the overall number of appropriate referrals increased from 37.8% to 49.5% at participating surgeries. However, these results were compared with the 36 other local GP practices that did not participate in the training programme, which also displayed an increase in appropriate referrals from 40.8% to 56.4% from 2010 to 2011.
Lester et al. 39 reported an intervention consisting of a 17-minute video, a 15-minute question-and-answer session, and two refresher educational sessions conducted over 4 months. Ninety-seven people with a first episode of psychosis were referred by intervention practices and 82 people from control practices during the study: RR of referral 1.20 (95% CI 0.74 to 1.95, p = 0.48). No effect was observed on secondary outcomes except for ‘delay in reaching early-intervention services’, which was statistically significantly shorter in patients registered in intervention practices (95% CI 83.5 to 360.5, p = 0.002).
Emmerson et al. 40 developed a psychiatric assessment and advisory service for local GPs. Five full-time psychiatrists dedicated a 1-hour appointment per week in their hospital private practice clinics to assess patients referred by local GPs. After 12 months referrals to the clinic were disappointing (n = 30, with 10 referrals from one GP). Feedback from GPs who had used the service showed high levels of satisfaction with the service (mean score 6.2 out of 7). Feedback from GPs who had not used the service showed a strong endorsement of the concept (94%), but there was poor awareness of the service’s existence (26%).
There were also two studies of interventions delivered over several sessions which were at higher risk of bias. Lam et al. 25 conducted an evaluative study to examine the impact of a 1-year part-time Postgraduate Diploma in Community Geriatrics. The diploma includes the components of clinical attachment (20 sessions of clinical geriatric teaching and five sessions of rehabilitation and community health services), interactive workshops, locally developed distance-learning manual, written assignments and examination as well as a clinical examination. Most respondents did not refer elderly patients to private geriatricians and would refer them to public geriatricians or other specialists. After the course, the average percentage of elderly patients being referred to private geriatricians increased from 2.8% to 6.1% and to other specialists decreased from 53.4% to 49.1%. The changes in the referrals to private geriatricians and other specialists were statistically significant. However, no significant change was found in the referrals to public geriatricians. The average percentage remained around 44%. It is unclear which of those outcomes were beneficial or how this study could be applied in a UK context.
Finally, Rowlands et al. 26 implemented an educational intervention consisting of referral meetings. Fewer than half of doctors became involved with development of formal referral or clinical protocols. Eighty-eight per cent noted a change in their referral practice. Overall, there was no change on referral rate in the intervention group. This study was considered to be at higher risk of bias.
Guidelines (no training or feedback)
Interventions that consisted of guidelines mailed to GPs (with no further training, support or feedback) were reported in 12 studies (Table 6). 41–52 The guidelines were for a range of referral conditions and procedures including genetic screening, orthopaedics, complications of diabetes, dementia, dermatology (two studies43,49), radiography (two studies42,50) and cancer (three studies41,46,47). Overall, seven studies reported at least some positive impact on referral,41–47 with five showing no effect or a negative change. 48–52 Two of the positive impact studies were considered to be at higher risk of bias41,42 with all other studies at lower risk of bias. Overall, the strength of this evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Cusack 200543 | NICE guidelines and a pro forma | BA | UK | Dermatology | 36 GPs | (18 months) |
| 150 referrals | ||||||
| Fearn 200948 | QOF Depression Indicators | BA | UK | Dementia clinic | NR | (18 months) |
| Hill 200049 | Local guidelines | Audit | UK | Dermatology | 33 GP practices | (2 years) |
| 422 patients | ||||||
| Idiculla 200044 | Local guidelines | RCT | UK | Outpatient infertility clinic | 214 GP practices | 1 year |
| 689 referrals | ||||||
| Most aged over 34 years, 84% female only | ||||||
| Imkampe 200647 | Pro forma for breast cancer referral | BA | UK | Oncology (breast cancer) | 2354 referrals | (8 months) |
| Lucassen 200145 | Local guidelines | BA | UK | Regional genetics service | NR | 14 months (6 months) |
| Malik 2007 41 | 2-week wait cancer guidelines | Audit | UK | Oncology (bone or soft tissue tumour) | 40 patients | 2 years |
| Matowe 200250 | Royal College of Radiology referral guidelines | BA | UK | Radiology | 376 GPs in 87 practices | (3 years) |
| 117,747 referrals | ||||||
| Melia 200851 | Prostate Cancer Risk Management Programme guidelines | BA | UK | Urology | 200 GP partners in 48 practices | 1–2 years |
| Male patients aged 45–84 years, n = 1520 | ||||||
| Potter 200746 | 2-week wait cancer guidelines | Cohort | UK | Oncology (breast cancer) | 24,999 new referrals | (7 years) |
| Twomey 2003 42 | Local guidelines | BA | UK | Radiology | NR | 2 years |
| West 200752 | Local guidelines | BA | UK | Orthopaedic outpatient department | 471 referrals | 29 weeks |
Seven studies showed a positive effect on at least one referral outcome (although results were often borderline or mixed). Five of these studies were considered to be at lower risk of bias.
Cusack and Buckley43 analysed dermatology referral letters from GPs prior to guidelines and 60 following guideline introduction. NICE guidelines and a pro forma for future referrals were sent to GPs. The percentage of referrals in accordance with NICE guidelines increased from 31% to 45% after introduction of guidelines (p = 0.041). The percentage of inappropriate referrals decreased from 69% to 55%, and 22% of GPs (8 of 36) fully complied with guidelines. However, over 50% of referrals were still inappropriate. The pro forma was used in only 23% of referrals and the provision of data in referral letters remained poor. The number of referrals per month only marginally decreased.
Idiculla et al. 44 analysed 200 GP referral letters submitted before (set 1) and 200 submitted after (set 2) local guidelines on the management of adult diabetes had been issued to local GPs. Following the distribution of the guidelines there was no significant change in the frequency with which specific conditions were documented in referral letters (set 1 vs. set 2): for example, hypertension 72% versus 79%, cerebrovascular disease 89% versus 80%. However, the guidelines did appear to have encouraged the active treatment of hyperglycaemia by GPs before referral.
Lucassen et al. 45 sent referral guidelines for a regional genetics service family cancer clinic to GPs and subsequent content of referral letters was analysed and compared with the previous 6 months. Post guidelines, more referrals met the criteria than before (χ2 = 15.79, p < 0.001). Fewer lower-risk referrals were made: 34% of letters (36/103) were high risk pre guidelines, whereas 47% (46/110) were high risk post guidance (not significant: χ2 for change in proportion of low risk pre and post = 1.34; p = 0.24, and for high risk χ2 = 3.33, p = 0.07). The description of the risk in the GP letter improved so that a greater proportion of generic clinic risks agreed with those described in the GP letter.
Potter et al. 46 used routine data to consider the effect of the introduction of the 2-week wait guideline for cancer referrals. The annual number of referrals increased over 7 years from 3499 in 1999 to 3821 in 2005, a significant increase of 1.6% (95% CI 1.0% to 2.2%). The number of 2-week wait referrals increased by 42% (n = 739) from 1751 in 1999 to 2490 in 2005, an estimated increase of 5.8% per year (5.0% to 6.7%, p = 0.001). By contrast, the number of routine referrals has declined over the same period by an estimated 4.3% a year (3.3% to 5.2%, p < 0.001), giving an apparent reduction of 24% (n = 417) from 1999 to 2005. The percentage of patients diagnosed with cancer in the 2-week wait group decreased from 12.8% (224/1751) in 1999 to 7.7% (191/2490) in 2005 (p < 0.001), whereas the number of cancers detected in the ‘routine’ group increased from 2.5% (43/1748) to 5.3% (70/1331) (p < 0.001) over the same period. About 27% (70/261) of people with cancer are currently referred in the non-urgent group. Waiting times for routine referrals have increased with time.
Imkampe et al. 47 determined whether or not GP grading of referrals into urgent and non-urgent had improved after the introduction of the 2-week rule was introduced. A retrospective review of GP referrals over 8 months, between September 2003 and April 2004, with regard to their urgency, subsequent diagnosis and the use of standardised referral formats was carried out. The results were compared with the 1999 audit. Eighty-two of 1178 patients referred by GP had breast cancer versus 115 of 1176 patients referred in 1999. Sixty-eight per cent (56/82) of breast cancer patients were referred as urgent, compared with 47% (54/115) in 1999 (p = 0.005). A pro forma was used in 47% (548/1178) of GP referrals, while no pro forma was used in 1999. Sixty-five of the 82 cancer patients were referred with a pro forma and 85% (55/65) were referred as urgent.
Two further studies which showed a positive effect on at least one referral outcome were at higher risk of bias. Malik et al. 41 determined if the 2-week wait referral guidelines for suspect cancer referrals had been followed and what proportion of patients referred under the guideline had malignant tumours. Referral letters were evaluated to see if they met Department of Health guidelines for referral of a suspected bone or soft tissue tumour. Most (31 of 40: 78%) ‘2-week’ referrals met the published referral guidelines. However, in 9 of the 40 cases, the patient did not meet the criteria for urgent referral, and none of the nine patients had malignant tumours. Of 40 patients referred under the guideline, 10 of these patients (25%) had malignant tumours, but this was compared with 243 of 507 (48%) of those referred from other sources. Twomey42 assessed GP referral for plain radiography in the areas of hip, knee, cervical spine and lumbar to establish a procedure for the development of care pathways. The proposed guidelines were circulated to all GPs. GP referrals to radiology for plain radiography declined from 2365 the year before the intervention to 1077 the year after intervention, a total reduction of 288 (54%). Similarly, referrals for plain radiography requests declined from 6650 to 4291, a reduction of 2359 (35.5%).
Five further studies (all at lower risk of bias) of dissemination of referral guidelines showed no effect, or a negative effect, on referral outcomes.
Fearn et al. 48 looked at whether or not the introduction of Quality and Outcomes Framework (QOF) Depression Indicators changed the pattern of referrals from primary care to a dedicated dementia clinic. The percentage of all referrals originating from primary care was about half in both time periods and did not differ significantly between the two time periods (χ2 = 0.88, df = 1, p > 0.1; z = 0.77, p > 0.05). Of the referrals from primary care, about one-third referred in both time periods had dementia. The RR of a diagnosis of dementia in a primary care referral pre and post QOF was 0.55 (95% CI 0.40 to 0.74) and 0.66 (95% CI 0.49 to 0.89), respectively. The proportion of patients referred from primary care with dementia was the same in the cohorts seen both before and after introduction of the QOF Depression Indicator (χ2 = 0.54, df = 1, p > 0.05), a finding corroborated by the z-test (z = 0.60, p > 0.05).
Hill et al. 49 evaluated referral guidelines for dermatology compiled by the dermatologist at the Royal Surrey County Hospital in consultation with local GPs. A 40% increase was seen in the numbers of referrals recorded by the dermatologist as appropriate immediately after the guidelines were sent (from 57% to 80%). The 2-year follow-up audit, however, demonstrated that the improvement had not been sustained, with a decline to 48% appropriate referrals.
Matowe et al. 50 mailed copies of the Royal College of Radiology referral guidelines for chest, limb and joint, and spine radiographs to GPs. There were no significant effects of the intervention on total number of general practice imaging requests. Total referrals decreased by 32 (95% CI –226.7 to 291.4) in the month following guideline dissemination, while the trend decreased by –1.82 requests per month (95% CI –11.8 to 8.2 requests per month). Referral only decreased by average 1.2 per month for the entire 35-month period.
Melia et al. 51 disseminated the Prostate Cancer Risk Management Programme (guidelines for GPs on age-specific PSA cut-off levels in asymptomatic men). One year after intervention, awareness of the pack was acknowledged by 112 (56%) GPs (24 were unaware and 64 did not know if they had seen it). The proportion of asymptomatic men referred who had raised antigen levels did not increase significantly from baseline to intervention (24% pre intervention, 29% post intervention; p = 0.42) There was no significant difference in referral rate by area (p = 0.33).
West et al. 52 completed a 13-week audit of referral letters for six specific orthopaedic complaints, namely anterior knee pain, back pain, carpal tunnel syndrome, in-toeing in children, sciatica and tennis elbow. Paper copies of referral guidelines produced by orthopaedic consultants were distributed to all local GPs. After a period of 4 weeks for distribution, the process was repeated for a further 13 weeks. The first 13-week period had 195 (64%) referrals that consisted of patients who had not received the recommended management or to whom this had not been mentioned in the referral letter. The second period had 103 (61%). There was no statistically significant difference between the two (p = 0.49).
Guidelines with additional training or feedback
Interventions consisting of guidelines with additional training or feedback were reported in 18 studies (all lower risk of bias), of which 11 showed a positive association with referral outcomes53–63 and six did not (Table 7). 64–69 The guidelines were for a range of referral conditions and procedures including mental health, infertility clinic, dermatology, gynaecology, oncology, colorectal surgeon, urology, cardiology (two studies56,63), low-back pain (two studies66,87), endoscopy (two studies53,55) and radiology (four studies54,57,59,60).
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Banait 200353 | Educational outreach/dyspepsia management guidelines | cRCT | UK | Open-access endoscopy (GI) | 114 practices | (6 months) |
| 233 GPs | ||||||
| Dey 200466 | RCGP guidelines plus outreach visits | cRCT | UK | Low-back pain | 24 health centres | (8 months) |
| 2187 patients; age 18–64 years (mean 42.2 years, SD 12.1) | ||||||
| 54% female | ||||||
| Eccles 200154 | RCGP guidelines, audit and feedback, or educational messages | RCT | UK | Radiology | Six radiology departments; 244 general practices | (1 year) |
| Elwyn 200755 | NICE guidelines plus feedback | BA | UK | Endoscopy (dyspepsia) | 215 GPs | (5 months) |
| Three endoscopy units | ||||||
| Engers 200567 | National guidelines plus workshop | cRCT | the Netherlands | Low-back pain | 41 GPs | NR |
| 531 patients | ||||||
| Glaves 200557 | Guidelines plus return of referrals | BA | UK | Radiology (spine and knee) | Three community hospitals | (1 year) |
| Griffiths 200658 | Local guidelines and training sessions | cRCT | UK | Dermatology | 165 health centres | NR |
| Patients 18+ years with psoriasis n = 188 | ||||||
| Jiwa 200668 | Local guidelines plus visit | cRCT | UK | Colorectal surgeon | 44 practices | (6 months) |
| 180 GPs | ||||||
| 504 patients | ||||||
| GPs 30–60 years | ||||||
| Julian 200762 | Shared care guidelines | nRCT | UK | Gynaecology | 193 GP practices | (8 months) |
| One hospital | ||||||
| Kerry 200059 | Royal College of Radiology guidelines plus feedback | RCT | UK | Radiology (spinal exam) | 69 GP practices | 2 years (9 months) |
| Morrison 200164 | Local guidelines plus meeting | RCT | UK | Outpatient infertility clinic | 214 GP practices | 1 year |
| 689 referrals | ||||||
| Age 34+ years | ||||||
| 84% female | ||||||
| Robling 200260 | Local guidelines plus seminar or newsletter | RCT | UK | Radiology (MRI) | 121 GP practices | NR |
| 182 referrals | ||||||
| Spatafora 200569 | Local guidelines plus meeting | BA | Italy | Urology (outpatients) | 45 urological centres, 263 GPs | NR |
| GPs’ mean age 47 years | ||||||
| 18% female | ||||||
| Walkowski 200763 | Local guidelines, telephone call, e-mail, or in-person visit | cRCT | USA | Cardiology | Five US states | 15 months (3 months) |
| White 200461 | Local guidelines plus implementation strategy | Audit | UK | Mental health | NR | (2 years) |
| Wilson 200665 | Local guidelines plus education meetings and outreach | cRCT | UK | Oncology (familial breast cancer) | GP in Grampian | 4 years (11 months) |
| Wright 200656 | Guidelines, educational meetings, outreach visits | CBA | UK | Cardiology (post TIA for stroke prevention) | One PCT | 50 months (22 months) |
Eleven studies showed a positive relationship between the intervention and referral-related outcomes. 53–63
Banait et al. 53 implemented educational outreach as a strategy for facilitating the uptake of dyspepsia management guidelines in primary care for open-access endoscopy. All groups received the guidelines by post and the intervention groups began to receive education outreach 3 months later. The outreach included practice-based seminars with hospital specialists at which guidelines recommendations were appraised and implementation plans formulated, and was reinforced by visits after 12 weeks. The proportion of appropriate referrals was higher in the intervention group in the 6-month post-intervention period (practice medians: control = 50%, intervention = 63.9%; p < 0.05). The proportion of major findings at endoscopy did not alter significantly, but there was an overall rise in acid-suppressing drugs in the intervention, compared with the control group (+ 8% vs. + 2%, p = 0.005).
Eccles et al. 54 compared two methods of reducing GP requests for radiological tests in accordance with the UK Royal College of Radiologists’ guidelines on lumbar spine and knee radiographs. GPs and consultant radiologists wrote referral guidelines and educational messages for lumbar spine and knee radiographs [based on the Royal College of Radiologists’ guidelines and the Royal College of General Practitioners’ (RCGP) back-pain guidelines]. The referral guidelines were then sent by post to all study GPs. Each practice was randomly allocated to receive audit and feedback or control; and educational messages or control. Feedback covered the previous 6 months’ referrals and was sent to GPs at the start of the intervention period and 6 months later. Educational messages were attached to the reports of every knee or lumbar spine radiograph requested during the intervention. The effect of educational reminder messages (i.e. the change in referral rate after intervention) was an absolute change of 1.53 (95% CI 2.5 to 0.57) for lumbar spine and of 1.61 (2.6 to 0.62) for knee radiographs (relative reductions of ≈20%). The effect of audit and feedback was an absolute change of 0.07 (1.3 to 0.9) for lumbar spine and 0.04 (0.95 to 1.03) for knee radiograph requests (relative reductions of 1%). Requests from doctors who had received audit and feedback were no more likely to be appropriate than requests from other doctors: OR 0.75 (95% CI 0.52 to 1.07) for lumbar spine radiographs and 0.82 (0.50 to 1.33) for knee. For doctors who had received educational reminder messages, the equivalent values were 0.95 (0.63 to 1.67) and 1.36 (0.86 to 2.23).
Elwyn et al. 55 evaluated a system of providing feedback to clinicians following referral requests not adhering to NICE guidelines. Letters were sent to GPs stating that two GPs would be employed part-time to assess all endoscopy letters and referrals for dyspepsia and they would be judged against recently issued NICE guidelines. Where referrals did not meet the criteria, the referring doctor would be informed by letter giving a reason for non-adherence to guidelines. The All Wales Dyspepsia Guidelines based on NICE criteria were circulated to GPs 2 weeks earlier. Adherence to NICE guidelines for referral criteria increased significantly among GPs following the intervention (mean 55% to 75%; 95% CI 13.6 to 26.4; p < 0.001). No similar effect was seen for hospital doctors. The number of gastroscopy referrals for dyspepsia declined after the intervention, but not significantly after inclusion of seasonal effects (p = 0.065). Intervention significantly reduced the referral to procedure time for gastroscopy (mean 52.1 to 39.4 days, 95% CI 6.6 to 18.6 days; p < 0.001).
Wright et al. 56 completed an evaluation of a quality improvement programme for transient ischaemic attack (TIA) referral in three primary care trusts (PCTs). Four local consensus group meetings for relevant stakeholders (including service users and carers) were used to adapt national guidelines to local context and identify barriers and incentives for changing practice. Guideline reminders for clinicians included laminated posters, desktop coasters and electronic referral templates. Guidelines were disseminated via education meetings in each PCT and further education outreach visits to 19 practices. Guidelines were disseminated by post to other practices not requesting a visit. There was a 41% increase in referrals from trained practices, compared with control practices (RR 1.41, p = 0.018). Adherence to best-practice standards was significantly higher in practices that had received the training programme than in the controls.
Glaves57 undertook an intervention where GPs referring to three community hospitals and a district general hospital were circulated with referral guidelines for radiography of the cervical spine, lumbar spine and knee. All requests for these three examinations were checked and requests that did not fit the guidelines were returned to the GP with an explanatory letter and a further copy of the guidelines. If the GP maintained the opinion that the examination was indicated, they had the option of supplying further information in writing or speaking to a consultant radiologist to reach agreement. The total number of examinations fell by 68% in the first year (95% CI 67% to 69%) and 79% in the second year (95% CI 78% to 80%). Knee radiographs fell by 64% in the first year (95% CI 62% to 65%) and 77% in the second year (95% CI 75% to 79%). Lumbar spine radiographs fell by 69% in the first year (95% CI 68% to 71%) and 78% in the second year (95% CI 77% to 80%). Cervical spine radiographs fell by 76% in the first year (95% CI 74% to 78%) and 86% in the second year (95% CI 84% to 88%) (p = 0.001 for all measures).
Griffiths et al. 58 evaluated the effectiveness of guidelines and training sessions on the management of psoriasis in reducing inappropriate referrals from primary care. Guidelines on the management of psoriasis in primary care, developed by local dermatologists, were sent to health centres in the intervention arm, and supplemented by the offer of a practice-based nurse-led training session. Patients in the intervention arm (82/105) were significantly more likely to be appropriately referred than patients in the control arm (49/83), a difference of 19.1% [OR 2.47; 95% CI 1.31 to 4.68; intracluster correlation coefficient (ICC) 0]. Only 25 (30%) health centres in the intervention arm took up the offer of training sessions. There was no significant difference in outcome between health centres in the intervention arm that received a training session and those that did not (OR 1.28, 95% CI 0.50 to 3.29; ICC 0).
Kerry et al. 59 evaluated the introduction of radiological guidelines into general practices, together with feedback on referral rates, to see whether or not this reduced the number of GP radiological requests over 1 year. A GP version of the Royal College of Radiologists guidelines was sent to each GP in the 33 practices in the intervention group. Guidelines for examination of chest, hips, knees, spine, skull and sinuses were printed verbatim on two sides of a sheet of A4 paper, which was then laminated. After 9 months’ intervention, practices were sent revised guidelines with individual feedback on the number of examinations requested in the past 6 months. A total of 43,778 radiological requests were made during the 2-year intervention. The number of referrals for all spinal examinations fell by 18% in the intervention group, compared with a 2% rise in the control group (p = 0.05). Taking requests for the lumbar spine alone, there was a reduction of 15% in the intervention group, compared with a rise of 5% in the control group, giving a difference of 20% between the groups (95% CI 3% to 37%). Overall, an 8% reduction in total numbers of radiological requests was observed in the intervention group, compared with a 2% increase in the control group (10% between the two groups, not significant).
Robling et al. 60 investigated whether or not method of access or method of guideline dissemination affects GP compliance with referral guidelines for magnetic resonance imaging (MRI) in two sequential trials: (1) one group of practices requesting MRI by telephone was compared with a second group requesting in writing using a standard request form. A third group could refer as wished; and (2) one group of practices receiving guidelines via a seminar was compared with a second group who received feedback via a newsletter with practice-specific data on referrals. A third group received both a seminar and feedback, and a fourth group received guidelines only by post. The seminars were facilitated by an academic GP and a researcher. In trial 1, 65% of requests were judged to be compliant with the guidelines and there were no statistical differences between the three groups. Telephone access proved unpopular among participants and written access more cost-effective. In trial 2, 74% of referrals were judged to be compliant with the guidelines and there was no association between method of dissemination of guidelines and compliance. Requests made after dissemination of guidelines were more likely to be compliant: 74% versus 65% (OR 1.62, p < 0.005).
White et al. 61 aimed to use guidelines to improve communication between GPs and community mental health teams (CMHTs). Following a baseline audit of referrals and assessment letters, locally agreed good practice protocols were developed and shared widely, accompanied by a dissemination and implementation strategy (updates at 6-monthly intervals throughout the project). Significant improvements occurred in both the GP and the CMHT letters. These were most dramatic after 1 year but tailed off considerably in the second year despite continued efforts to implement the protocol’s standards. Annual GP referrals (percentage of total) reduced from 661 (63%) to 550 (58%), p-value not significant, and new referrals completing CMHT assessment increased from 369 (66%) to 423 (89%) (p < 0.001).
Julian et al. 62 examined the outcomes of an integrated model. Women attending the new ‘Bridges’ pathway were compared with those attending a consultant-led one-stop menstrual clinic. The Bridges pathway involved the use of shared care evidence-based guidelines for the management of dysmenorrhoea patients in primary and secondary care, which determined the timings for investigations and surgical treatment. Management decisions were made by GPs in all but atypical/complex cases. At 8 months, there were no significant differences between the groups in terms of surgical and medical treatments of in the use of GP clinic appointments. Significantly fewer hospital outpatient appointments were made in the Bridges group than in the one-stop menstrual clinic (p < 0.001). Patient diaries demonstrated a significant improvement in the Bridges group for patient information, ease of access (p < 0.001), choice of doctor (p < 0.002), waiting time (p < 0.001) and less ‘limbo’ between primary and secondary care (p < 0.001).
Walkowski et al. 63 tested the effect of different strategies to inform GPs of the high performing cardiac specialists in their community and facilitate increased referrals to these specialists. This initiative involved sending letters to primary care physicians which requested that when the physician had a patient needing referral to a cardiac specialist or facility, they refer that patient to a physician or facility that had earned the ‘United Health Premium designation for both Quality and Efficiency of care’. To facilitate those referrals, the primary care physicians were provided with a hard-copy referral list of cardiac specialists and hospitals. Participants were divided into four test groups: (1) letter and referral list (LRL) only (n = 3537); (2) LRL plus follow-up telephone call from the local health plan (n = 252); (3) LRL plus e-mail reminder (n = 1187); or (4) LRL plus in-person follow-up visit from the local market medical director (n = 65). The initial 3-month pilot data showed an overall 6.3% increase of patients referred to United Health Premium-designated quality and efficient cardiac specialists overall, compared with a baseline period of 12 months prior to the mailing. Intervention effects ranged from 17% change (letter plus call) to 22% change (letter plus visit), versus 0.3% change in the control group. The applicability of this study in the UK may be limited.
Six further studies (all at lower risk of bias) of dissemination of referral guidelines with additional support or training showed no effect or a negative effect on referral outcomes. 64–69
Morrison et al. 64 evaluated the effect of clinical guidelines on the management of infertility in general practice. Local guidelines were developed and a management pack was sent to intervention practices with an invitation to attend a meeting to discuss using the guidelines. Seventeen per cent of doctors attended a meeting. Individual visits were also offered but were taken up by only two practices. There was no difference between the control and intervention practices with regard to whether or not a management plan was made (OR 1.239, 95% CI 0.869 to 1.765; p = 0.236). There was also no difference in duration between first appointment and date of management plan, no difference in mean number of outpatient visits before a management plan was put in place, and no significant difference in total costs to the NHS (£349.78 vs. £327.48, p > 0.05).
Wilson et al. 65 considered the effectiveness of an intervention to improve GP confidence in managing patients concerned about genetic risk of breast cancer. Components of the intervention software included a list of the key patient information needed in order to use the guidelines. A risk assessment module was presented as a set of short checklists, in which the Scottish referral guidelines for breast, ovarian and colorectal cancer were embedded. This was provided along with the following: background information on cancer genetics and the evidence underlying the guidelines (prepared by local geneticists); printer-ready, locally customised patient information leaflets; selected web-links for professionals and patients; and a contact e-mail link with the Cancer Genetics Service, with a guaranteed response time. The system automatically produced a draft referral letter using the regionally recommended template. All partners in intervention practices were invited to interactive workshops on cancer genetics designed to complement the software. No statistically significant differences were observed between intervention and control arms in the primary or secondary outcomes. Only a small proportion of intervention GPs attended the educational session, were aware of the software or made use of it in practice. In the pre-intervention period, intervention GPs were less likely than control GPs to refer patients who were eventually assessed as having elevated genetic risk (0.70, 95% CI 0.50 to 0.99), with the opposite trend observed in the post-intervention period (1.18, 95% CI 0.88 to 1.37), although these results did not reach statistical significance.
Dey et al. 66 evaluated the impact on patient management of an educational strategy to promote the RCGP’s low-back-pain guidelines among GPs. Practices in the intervention arm were offered outreach visits to promote national guidelines on acute low-back pain, as well as access to fast-track physiotherapy and to a triage service for patients with persistent symptoms. At least two members of the guideline team attended each visit; these included senior representatives from the musculoskeletal directorate, physiotherapy services and the health authority. Members of the guideline team facilitated a structured interactive discussion with the GP to raise awareness of the RCGP guidelines, adapted to the local context; emphasise the key messages in the guidelines; identify potential barriers to implementation; and suggest strategies for overcoming the barriers identified. GPs were given a poster reinforcing guideline recommendations and a copy of a text recommended by the RCGP for patients. The estimated annual consultation rate for acute low-back pain was 35 per 1000 adults in the intervention group, compared with 38 per 1000 in the control group. There were no significant differences between study groups with respect to the proportion of patients who were referred for radiography (a difference of 1.4%, 95% CI –4.1% to 6.8%), issued with a sickness certificate (a difference of –1.5%, 95% CI –10.3% to 7.3%), prescribed opioids or muscle relaxants (a difference of –0.03%, 95% CI –5.5% to 5.4%) or referred to secondary care (a difference of 1.1%, 95% CI –0.3% to 2.6%). Significantly more patients in the intervention group were first referred to physiotherapy or to educational programmes at the back pain unit than in the control group (a difference of 12.2%; ICC = 0.0563; χ2 = 6.49; 1 df; p = 0.01; 95% CI 2.8% to 21.6%).
Engers et al. 67 assessed the effectiveness of the Dutch low-back-pain guideline for GPs with regard to adherence to guideline recommendations. GPs in the intervention group received a tailored interventions consisting of the Dutch low-back-pain guideline for GPs, a two hour educational and clinical practice workshop; two scientific articles on low-back-pain management; the guideline for occupational physicians; a tool for patient education; and a tool for reaching agreement on low back care with physical, exercise and manual therapists. The participating GPs were asked to recruit consecutive patients with a new episode of low-back pain as the main reason for consultation. The intervention was delivered by a psychologist-physiotherapist. Forty-one of the 67 randomised GPs reported on a total of 616 consultations for 531 patients with non-specific low-back pain. The advice and explanation provided by the GPs, the prescription of paracetamol (33% vs. 21%) or non-steroidal anti-inflammatory drugs (54% vs. 62%), and prescription of pain medication on a time contingent (70% vs. 69%) or a pain contingent basis (30% vs. 31%), showed no statistically significant differences between the intervention and control groups. There were also no differences in overall referral rate (23% vs. 28%; OR 0.8, 95% CI 0.5 to 1.4). However, in follow-up consultations fewer patients were referred to a physical or exercise therapist by the GPs in the intervention group than in the control group (36% vs. 76%; OR 0.2, 95% CI 0.1 to 0.6).
Jiwa et al. 68 evaluated a referral guideline intervention for lower bowel symptoms. GP practices were offered one of an electronic interactive referral pro forma, an educational outreach visit by a local colorectal surgeon, both or neither. They developed and piloted an interactive electronic pro forma for processing referrals to colorectal surgeons (General Practice Referral Assessment Facilitator or G-RAF). The interactive pro forma requested information on drop-down menus for 15 clinical signs and symptoms previously identified by GPs and colorectal surgeons as those of significant colorectal disease. The interactive software offered the practitioner guidance on which cases needed urgent referral with reference to current UK Department of Health guidelines. A referral letter was automatically produced seeking an appropriate appointment at a hospital clinic. The educational outreach visit was delivered by a colorectal surgeon. During the 45-minute meeting, the presenter summarised the features of significant organic colorectal disease and encouraged questions. There were 716 consecutive referrals recorded over a 6-month period, for which a diagnosis was available for 514. There was no significant difference in proportion of cases with significant pathology for either intervention or compared with no intervention. In the combined software arms 14% (37/261) had significant pathology, compared with 19% (49/253) in the non-software arms: RR 0.73 (95% CI 0.46 to 1.15). In the combined educational outreach arms 15% (38/258) had significant pathology, compared with 19% (48/256) in the non-educational arms (RR 0.79, 95% CI 0.50 to 1.24).
Spatafora et al. 69 developed a short algorithm on procedures to be used with men with LUTS. The algorithm was developed by urologists and approved by a panel of experts. It was presented at a meeting with local GPs and revised in line with feedback, and the revised protocol was presented at each centre. The protocol was a clinical report form containing history, examination, use and outcome of tests, and diagnosis. Sixteen per cent of centres accepted the original protocol with no changes. There was no significant change in referral pattern from baseline to intervention: 51.2% of patients were managed entirely by their GP, 44.3% were referred to urologist after some diagnostic procedures and 4.5% were referred without any diagnostic testing. Use of digital rectal exams increased significantly from 32% to 41% (p < 0.001) and this was predominantly in centres that endorsed this test.
Process change interventions
We defined process changes as small-scale changes to some aspect of the individual referral process which did not involve the movement of staff or relocation of clinics, the methods in which referrals were triaged at hospital or financial arrangements for referral.
Process change interventions included designated appointment slots and fast-track clinics for primary care referrals (n = 6), interventions that provided direct access to screening (n = 9), specialist consultation prior to referral (n = 11), electronic referral systems (n = 10), the provision of decision support tools to assist GPs in making referrals (n = 10) and interventions that consisted of waiting list review or watchful waiting (n = 3).
Designated slots/fast-track clinics
The provision of designated appointment slots and fast-track clinics for primary care referrals were reported in six studies (Table 8). 70–75 The speed of referral varied from the same day to within 2 weeks (to meet the 2-week cancer referral guidelines). Four studies showed a positive effect,70–73 with two studies showing a negative or no effect. 74,75 One effective study was considered to be at higher risk of bias. 72 The evidence overall was rated as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Bridgman 200570 | Quota appointment slots | CBA | UK | Orthopaedic (outpatients) | 36 practices | (18 months) |
| 33 GPs | ||||||
| 30 full-time | ||||||
| Three single practices | ||||||
| Hemingway 200673 | Protocol-driven fast-track referral system | BA | UK | Colorectal cancer screening | Eight surgeons, 10 GI physicians | (2 years) |
| Khan 200871 | Direct referral to Hot Clinic | Cohort | UK | Respiratory (COPD) | 173 patients | 6 months |
| 97 (57%) men, 75% current or ex-smokers | ||||||
| McNally 200374 | Fast-track clinic | BA | UK | Oncology (ovarian cancer) | 242 patients | 6 years |
| Prades 201175 | Seven fast-track hospital indicators | Mixed method | Spain | Oncology (breast, lung, colorectal) | 56,020 patients | NR |
| 83 health professionals from 18 clinics | ||||||
| 38% GPs | ||||||
| Sved-Williams 2010 72 | Single entry point for psychiatry | BA | Australia | Psychiatry | 45 psychiatrists | 28 months |
| 301 GPs | ||||||
| 824 patients |
Bridgman et al. 70 evaluated a slot system for referrals. GPs and orthopaedic consultants were invited to a meeting to discuss and input into the design of the system. The number of slots available was based on the registered practice population. Quota of slots and their use was fed back to practices on a monthly basis. If a practice went beyond their quota they were told that they might not be allowed to refer any more patients that month. GPs guaranteed a maximum of 8 weeks’ assessment for patients and the backlog of waiting patients was removed. GPs received guidelines on appropriate referrals and routes of referrals for musculoskeletal problems. A clerical officer was appointed to answer queries and make appointments. After a modification to the hospital software, referrals were made using a special pro forma, which included a prioritisation score. In total, 15,439 referrals were made, and 90% attended their appointments. The mean monthly referral rate in the intervention group declined 22% in year 1 and was maintained in year 2. The difference in mean referral rate between the control and intervention was –1.59 intervention; –2.61 control; and –4.39 other comparator. The relative mean rate in reductions in mean referral rates were: 14.5%, –23.7% and –39.5% in period 0, year 1 and year 2, respectively.
Khan et al. 71 evaluated the efficacy of direct GP referral to a hospital respiratory specialist team to a ‘Hot Clinic’ in avoiding hospital admissions. GPs and community nurses directly referred patients threatening an acute hospital admission, by fax, for a rapid assessment. The Hot Clinic service operated Monday to Friday, 09:00–16:00 hours. Patients were seen within 24 hours of the receipt of the referral letter. The consultation included clinical assessment, chest radiograph, laboratory data and a decision whether to treat the patient in the community or to admit the patient to the hospital. The GP would be informed by a typed and faxed letter returned the same day. In total, 27 patients (16%) were admitted directly from the Hot Clinic and 146 (84%) were treated in the community. Of those 146 patients, nine (5%) were later admitted within 1 week and 12 (7%) admitted over 1 week to 1 month after the Hot Clinic appointment. Overall, 125 (72%) were treated successfully in the community without the need for hospitalisation. However, it is unclear if all would have been hospitalised without the clinic.
Sved-Williams and Poulton72 described and evaluated a service that provided a single point of entry for GPs wishing to refer their patients for one-off psychiatric consultations. All psychiatrists in the region were invited to provide reserved appointments to an administrative officer based at the Department of General Practice. They could specify the number of appointments and withdraw unfilled appointments at any time. To make an appointment, a GP or practice nurse phoned a dedicated number Monday to Friday 09:00–17:00 hours. There was no paperwork, and the GP was supplied with the appointment time, along with the name and contact details of the psychiatrist over the phone. From August 2005 to March 2007, 84% of offered appointments were filled. Use of the service rose from six referrals to 10 per week over the course of the study, and 55% of psychiatrists continued to provide regular appointments after the study period. This study was at higher risk of bias.
Hemingway et al. 73 evaluated a protocol-driven rapid-access referral system for colorectal cancer tests. The Leicester Colorectal Test Protocol included a list of presenting symptoms, age criteria for test and the appropriate diagnostic test for each symptom. Patients had investigations either before seeing an outpatient clinician or on the day of the clinic. Referrals were processed by ‘2-week wait’ administration staff using the protocol and assessments booked by these administration staff. There was protection of time slots within the testing suites. Referrals not complying with protocol were redirected to appropriate test without referral back to GP. The data that relate to the intervention period are not clear as they are reported by year rather than before and after. At baseline, the year 1 median time to diagnosis for non-emergencies was 35 days (interquartile range 13–80 days), compared with fast-track (categorised as 2-week wait or ‘soon’) 21 days (interquartile range 10–48 days). Sixty-two per cent of cancers referred as either 2-week wait or ‘soon’ were diagnosed within 31 days. After introduction of the intervention (pilot and full implementation) year 3 median time to diagnosis for non-emergencies was 20 days (interquartile range 10–59 days) and for emergencies was 13 days (interquartile range 8–29 days) [year 4 non-emergencies 20 days (interquartile range 10–51 days) and emergencies 13 days (interquartile range 9–23 days)]. During the 2-month full implementation period in year 3, the service received 256 referrals: 64% came through the 2-week wait protocol office and 36% were referred directly to consultants. In these referrals 70% were diagnosed with a pathology and 19 patients were diagnosed with cancer, all within 31 days. Overall during year 3, 79% of patients with colorectal cancer diagnosed who were referred as 2-week wait or ‘soon’ were diagnosed within 31 days. In year 4, the figure was 82%.
Two studies showed no association with referral outcomes. McNally et al. 74 implemented clinic appointments within 2 weeks to a fast-track breast cancer clinic. GPs were informed of the clinic and referral criteria by individual letter, GP newsletter and meetings. The median waiting time for referral to specialist was 3 days (range 0–188 days). This did not change significantly after clinic introduction (p = 0.05). The impact of fast-track clinic on referral and diagnosis time variables was not significant.
Prades et al. 75 analysed the implementation and effectiveness of a fast-track referral system for cancer which included clinical criteria for primary care referral and patient pathway management in hospital. There was an increase in completeness of hospital data during the intervention period (74% to 96%). Adherence to clinical criteria for including patients in the fast-track system was more than 70% (no specific data reported). About half of all new patients were diagnosed via the fast-track system and the cancer rate declined during the period. The mean time to treatment from primary care was 32 days for breast cancer, 30 for colorectal cancer and 37 for lung cancer. There are no data for patients not referred via the programme to compare these results with.
Direct access to diagnostic testing
Nine interventions provided direct access to diagnostic testing for a range of conditions and, as such, included a range of screening tests [including MRI, dual-energy X-ray absorptiometry (DXA) scanning, computerised tomography (CT) brain scanning, audiology screening, endoscopy, CT (for chronic daily headache) and an open-access urology unit] (Table 9). 76–84 Six studies showed a positive effect,76–81 but three studies were very unclear as to the effect on referral outcomes82–84 and all studies were considered to be at lower risk of bias. The strength of the evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| DAMASK 200876 | GP direct referral for MRI | RCT | UK | Radiology (MRI) | 386 patients | NR |
| Dhillon 200382 | Direct access to DXA scan | RCT | UK | Rheumatology | 330 patients | NR |
| 18 practices | ||||||
| Patients aged 31 to 89 years | ||||||
| Eley 201083 | Direct audiology referrals | Audit | UK | ENT (audiology) | 353 patients | (4 months) |
| 178 female, 175 male | ||||||
| Mean age 77 (60–96) years | ||||||
| Gough-Palmer 200984 | Direct MRI access | Audit | UK | MRI | 1798 scans | 12 years |
| 209 GPs | ||||||
| Shaw 200677 | Open-access serology | cRCT | UK | Serology (dyspepsia) | 47 practices | NA |
| Simpson 201078 | Direct access to head CT | Audit | UK | Neurology | 4404 referrals | NA |
| 986 GPs | ||||||
| Thomas 201080 | Direct access to head CT | Audit | UK | Neurology | 232 referrals | (1 year) |
| 72 practices, 309 GPs | ||||||
| Patient age range 20–85 years | ||||||
| Thomas 200379 | Open-access urology | cRCT | UK | Urology | 66 GPs | (12 months) |
| 959 patients | ||||||
| Wong 200081 | Open-access endoscopy | CBA | Hong Kong | Endoscopy (dyspepsia) | 1334 patients | (2 years 10 months) |
| Mean age 74 years |
DAMASK76 looked at a process to allow direct referral from general practice to a local radiology department for MRI to allow early access to imaging. Early MRI was associated with higher NHS cost by £294 per patient and a larger number of quality-adjusted life-years (QALYs) by 0.05. There was an incremental cost per QALY gained of £5840 below the cost threshold of £20,000 per QALY commonly used in the NHS. This was, therefore, considered to be a cost-effective use of NHS resources.
Shaw et al. 77 evaluated the effect of providing a Helicobacter pylori serology service for GPs who requested open-access endoscopy. General practices were stratified by endoscopy referral rate and randomised into two groups. The intervention group was provided with access to H. pylori serology testing and encouraged to use it in place of endoscopy for patients aged < 55 years with dyspepsia. They were sent written information promoting the use of the serology service in place of endoscopy for patients aged < 55 years suffering from dyspepsia without alarm symptoms and were issued with a summary of the Maastricht consensus statement on the management of H. pylori. The GPs remained free to refer for open-access endoscopy as they felt necessary. The number of endoscopy referrals fell in both groups during the study period, but fell by a greater amount in the intervention group than in the control group. During the 2-year study period, 626 referrals were received from the intervention group, compared with 771 from the control group. This accounted for a significant reduction in referrals for endoscopy in the intervention group compared with the control group: 18.8% difference (95% CI 5.0% to 30.6%, p = 0.009).
Simpson et al. 78 assessed GP direct access to CT for patients with chronic daily headache; 10.5% of scans indicated abnormalities. GPs reported that if direct-access CT had not been available then 44% would have referred to neurology and 38% to general medicine. Ten per cent of patients would not have been referred. Following scans, 86% did not require further specialist referral. Sixty-seven per cent of reports issued following scans were received in 1 week and 79% were received within 14 days. Without direct access, 90% of patients would have been referred at a cost estimate of £503,428. The cost of scans and outpatient appointments for patients in the study was estimated to be £602,026. A specialist headache clinic where, typically, 29% are referred for scans would cost £131,991, with a further review appointment costing total £688,708. Therefore, the cost saving of this intervention was estimated at £86,681.
Thomas et al. 79 evaluated the effectiveness and efficiency of a guideline-based open-access urological investigation service. General practices were randomised to receive either referral guidelines and access to the investigation service for LUTS or referral guidelines and access to the investigation service for microscopic haematuria. Participating GPs were offered a 2-hour educational meeting and were mailed a guideline package which included a guideline booklet, a quick reference flow chart and structured referral checklists. GPs’ compliance with referral guidelines increased (difference in means 0.5, 95% CI 0.2 to 0.8; p = 0.001). Approximately 50% of eligible patients were referred through the new system. The number and case mix of referrals were similar. The intervention reduced the waiting time from referral to initial outpatient appointment (ratio of means 0.7; 95% CI 0.5 to 0.9, patients with LUTS only) and increased the number of patients who had a management decision reached at initial appointment (OR 5.8, 95% CI 2.9 to 11.5; p < 0.001, both conditions). Patients were more likely to be discharged within 12 months (OR 1.7, 95% CI 0.9 to 3.3; p = 0.11). There were no significant changes detected in patient outcomes.
Thomas et al. 80 evaluated the referral rate of patients with chronic headache to open-access CT and the effect on neurology referral rates at three sites. Scanned patients had a lower referral rate to neurology immediately and in the year following the scan. The referral rate to open-access service was 1.2% of headache consultations by GPs. Open-access scans accounted for 4% of the annual number of scans. Of 215 scans, three lesions were identified which may have caused chronic headache (1.4% yield for significant findings and 10.2% for non-significant findings), and 88.4% of scans were normal. The service was used by 45% of GPs from 82% of practices. At 1-year follow-up, 14% (30) were subsequently referred to neurology clinic because of headaches; of these, 40% were referred at the same time as the CT scan request and 60% were referred after their brain-scan CT. Of these later referrals, 17 of the 30 had normal CT findings.
Wong et al. 81 evaluated a system of open-access endoscopy for dyspepsia. Family physicians were able to arrange upper endoscopy directly with the endoscopy unit in addition to conventional referrals. Extra sessions each week were allocated to open-access requests to ensure waiting time not affected. Waiting time for the intervention group was a mean of 6 weeks. For the control group the mean waiting time was 17.5 weeks to consultation and then another 4.5 weeks to procedure (a total of 22 weeks). During this waiting time only antacids were prescribed. There were abnormal findings in 19% of patients from the intervention group and 22% from consultant referral (difference not significant). Only two patients (0.2%) referred via open access were considered inappropriate. There were no significant differences in intervention versus control in peptic ulcer and cancer detection rate, but significantly more non-ulcer non-cancer abnormal findings in referrals via consultant (0.5% vs. 5%, p < 0.005). Of the intervention patients, 76% required no further consultation for at least 4 weeks after endoscopy, 12% attended a GP, and 12% were referred to specialist or were admitted to hospital.
Three further studies of direct-access screening interventions showed no clear effect on referral outcomes. Dhillon et al. 82 evaluated the impact of GP direct access to DXA scanning for patients at risk of osteoporosis; no specific guidelines were issued. They reported mostly clinical outcomes, but included some limited referral rate data. Before intervention, the range of number of referrals for scanning was 0.01% to 0.6% (median 0.2%). The number of referrals to a specialist clinic was 24 in the intervention group, compared with 12 in the control group. The study also concludes that direct access is more economically efficient, but it is not fully explained in the data how this is evaluated. Eley et al. 83 assessed the effectiveness of direct referral to audiology clinics on ENT appointments and appropriate GP use of the clinics. Direct-referral audiology clinics (DRACs) for the assessment and provision of hearing aids in those > 60 years were introduced as a means of decreasing outpatient waiting times and demand on ENT appointments. Of the 353 patients seen within the DRAC clinics, 320 were ultimately provided with a hearing aid. Fifty-five patients require review by an otolaryngologist, either by direct referral or via their GP. The greatest lack of adherence to the referral criteria for DRAC appointments related to appropriate treatment of wax within the community. Gough-Palmer et al. 84 looked retrospectively at GP access to MRI scans. There was no protocol, guidance or formal consultant or radiologist vetting in place. GP-requested scans, as a percentage of the workload of the department, were low (around 2.6%). While workload of the department increased over the study period, this percentage remained stable. Forty-eight per cent of scans requested were normal or minor degenerative changes; 26% demonstrated serious pathology warranting hospital referral.
Enhanced referral information
These interventions were dominated by studies conducted in dermatology where images were sent electronically or by post to the specialist to assist in determining whether or not a referral was necessary (Table 10).
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Knol 2006 90 | Electronic consultation with images | BA | Netherlands | Dermatology | 505 consultations | (2 years) |
| 29 GPs | ||||||
| Leggett 200485 | Referral letter with images | RCT | UK | Dermatology | 136 referrals | NR |
| 20 GPs | ||||||
| McKoy 200489 | Electronic consultation with images | BA | USA | Dermatology | 52 patients | NR |
| Aged 25–89 years | ||||||
| 46% female | ||||||
| Tadros 2009 96 | Electronic referral with images | RCT | UK | Oncology (skin cancer) | 300 referrals | NR |
| Whited 200287 | Electronic consultation with images | RCT | USA | Dermatology | NR | NR |
Leggett et al. 85 compared outcomes of referral for dermatology appointments between patients whose referral letters did and did not include instant photograph(s) taken by the GP. The GP took photograph(s) of the skin condition and sent them with a referral letter to the dermatologist in a numbered, sealed envelope. If a diagnosis was not possible, patients were given an appointment. If diagnosis was possible, a letter was sent to the GP with advice on management; some patients were also given an appointment for further management. Control group patients were given outpatient appointments in the usual way. For 63% of the study group (45/71), a diagnosis and a management plan were made without the patient requiring an appointment. This included 38% (27/71) of patients who, after diagnosis and initial management, needed an appointment, and 25% (18/71) who did not. The remainder of the study group (37%: 26/71) required a face-to-face consultation. This reduced the numbers requiring an outpatient appointment by 25% compared with the control group. The mean time for formulation of a management plan for patients without an appointment was 17 days (SD 11 days); waiting times for appointments in study and control groups were similar (mean 55 days; SD 40 days).
Whited et al. 87 compared usual care (text-based electronic consultation) with teledermatology (usual care plus digital images and standardised history). The standardised history included demographic information, patient-reported medical history, dermatology history, lesion location, size and duration of presence. The consultant answered by scheduling an appointment or by relaying a diagnosis and management plan back to the GP. Patients in the intervention arm reached time to initial defined intervention sooner than those in the usual-care arm (median 41 vs. 127 days, p < 0.001) and 18.5% of patients in the intervention arm avoided the need for a dermatology clinic visit, compared with no patients in the usual-care arm (p < 0.001). A further satisfaction survey as part of the RCT was also reported. 88
McKoy et al. 89 evaluated the accuracy, access time, cost and acceptance by patients and physicians of an asynchronous teledermatology referral intervention in primary care. GPs in a multispecialty group referred patients for teledermatology consultation. Same-day history and digital images taken by a nurse were electronically sent to a dermatologist who returned a diagnosis to the referring physician. History was adequate for diagnosis in 81% of cases and images were adequate in 75% of cases. Accuracy of the teledermatology diagnosis in cases with adequate images was 97%; accuracy for all cases was 92%. A dermatology visit was recommended in 26% of cases with adequate images and in 42% of all cases. Access time for a teledermatology opinion was 1.9 days, compared with 52 days for a regular dermatology appointment.
Knol et al. 90 aimed to reduce dermatology referrals using teledermatology. One overview and two detailed digital photographs of the skin problems were taken on a digital camera and attached to an e-mail message containing standard clinical information. The e-mail was sent to a dermatologist who replied after evaluation. Using teledermatology, 163 patients were not referred, a reduction of 163 out of 306 or 53%. There was no significant difference between dermatologists for secondary referral (χ2 = 1.6, p = 0.45), and patient sex did not affect secondary referral (χ2 = 0.8, p = 0.36). This study was at higher risk of bias.
Hockey et al. 91 examined the feasibility of a low-cost store-and-forward teledermatology service for GPs in regional Queensland. GPs were required to decide whether to refer for electronic consultation with the hospital or whether to refer to outpatients as usual. Electronic communication with the hospital was through a secure web-based system. Over 6 months, 63 referrals were processed by the teledermatology service. In the majority of cases, the referring doctors were able to treat the condition after receipt of e-mail advice from the dermatologist. In 10 cases (16%) additional images or biopsy results were requested because image quality was inadequate. The average time between a referral being received and clinical advice being provided was 46 hours. This study was at higher risk of bias.
Specialist consultation prior to referral
Specialist consultation prior to referral was the basis of six interventions (Table 11). 86,89,92–95 The interventions varied from a shared care programme with an oncologist, a system to contact a spine orthopaedist for red flag symptoms, to a virtual outreach intervention to share medical records between GP and specialist. All interventions showed a positive effect on at least one referral-related outcome, although results were sometimes mixed. The evidence for these interventions was rated as stronger.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Eminovic 200986 | Teledermatology to confer with specialist | cRCT | the Netherlands | Dermatology | 85 GPs from 35 practices | (1 month) |
| Five dermatologists | ||||||
| Harrington 200193 | Flow chart/algorithm for care | Case series | USA | Orthopaedics (low-back pain) | 581 patients | 3 years |
| Jaatinen 200295 | Consideration of teleconsultation | RCT | Finland | Specialists | 78 patients | 5 months |
| McKoy 200489 | Electronic consultation with images | BA | USA | Dermatology | 52 patients | NR |
| Aged 25–89 years | ||||||
| 46% female | ||||||
| Nielsen 200392 | Knowledge transfer GP/oncologist | RCT | Denmark | Oncology | 248 referrals | (3 and 6 months) |
| Wallace 200494 | Virtual outreach between GP and specialist | RCT | UK | Specialists | 134 GPs from 29 practices and 20 consultant specialists | (6 months) |
Eminovic et al. 86 determined whether or not teledermatological consultations can reduce referrals to a dermatologist by GPs. The GPs randomised to the intervention used a teledermatological consultation system to confer with a dermatologist, whereas those in the control group referred their patients according to usual practice. A training programme for the intervention GPs included instructions on taking digital images, downloading images to the computer, managing files and using the website. Dermatologists were taught how to use the website and complete the study forms. All patients, regardless of their condition, were seen in the office by a dermatologist after approximately 1 month. The five dermatologists considered a consultation preventable for 39.0% of patients who received teledermatological consultation and 18.3% of 169 control patients, a difference of 20.7% (95% CI 8.5% to 32.9%). At the 1-month dermatologist visit, 20.0% of patients who received teledermatological consultation had recovered, compared with 4.1% of control patients. No significant differences in patient satisfaction were found between groups.
Nielsen et al. 92 conducted an intervention to determine the effect of a shared care programme on the attitudes of newly referred cancer patients towards the health-care system and their health-related quality of life and performance status, and to assess patients’ reports on contacts with their GP. The shared care programme included transfer of knowledge from the oncologist to the GP, improved communication between the parties and active patient involvement. The shared care programme had a positive effect on patient evaluation of co-operation between the primary and secondary health-care sectors. The effect was particularly significant in men and in younger patients (18–49 years) who felt that they received more care from the GP and were left less in limbo. Younger patients in the intervention group rated the GP’s knowledge of disease and treatment significantly higher than younger patients in the control group. The number of contacts with the GP was significantly higher in the intervention group. The quality of life questionnaire and performance status showed no significant differences between the two groups.
Harrington et al. 93 developed an algorithm for referral to a spine orthopaedists which included a flow chart for care and a system for separating urgent cases from others. GPs were encouraged to contact the surgeon or physician manager for advice on patients with red flag symptoms. This resulted in a receptionist taking information which was verified by a nurse co-ordinator. The physician manager then reviewed the information to determine a care plan, which was instigated by the nurse co-ordinator. Following introduction of the guidelines little change was documented from traditional referral patterns (no other information provided on this). Three years later, in response to long waiting lists, the referral management programme was put in place, resulting in a shift of care from spine orthopaedists to primary physicians. Before implementation, 28% of patient visits for low-back pain were referred to a specialist and 72% were treated in primary care. During the transition year, 13% of patient visits were referred to a specialist and 87% were treated in primary care. In the year after implementation, 17% were referred to a specialist care and 83% treated in primary care.
Wallace et al. 94 considered whether or not virtual outreach would reduce offers of hospital follow-up appointments and reduce numbers of medical interventions and investigations, reduce numbers of contacts with the health-care system, have a positive impact on patient satisfaction and enablement, and lead to improvements in patient health status. Joint teleconsultation between GPs, specialists and patients prior to referral was compared with standard outpatient referral. Fifty-two per cent of patients in the virtual outreach group were offered a follow-up appointment, compared with 41% in the standard outpatient group. The overall proportion of patients receiving an offer of follow-up was 46% in the virtual outreach group and 42% in the standard outpatient group (OR 1.19, 95% CI 0.99 to 1.44), but significant heterogeneity remained for both site and specialty (p = 0.001 and p < 0.001, respectively). Fewer tests and investigations were ordered in the virtual outreach group, by an average of 0.79 per patient. In the 6-month period following the index consultation, there were no significant differences overall in number of contacts with general practice, outpatient visits, accident and emergency contacts, inpatient stays, day surgery and inpatient procedures or prescriptions between the randomised groups.
Jaatinen et al. 95 considered teleconsultation as a replacement for referral to an outpatient clinic. GPs had to decide whether to refer for electronic consultation with the hospital or whether to refer to outpatients as usual. Electronic communication with the hospital was through a secure web-based system. All patients treated by teleconsultation said that they wanted the same procedure in the future and 63% of the control group said that they would prefer a teleconsultation next time (p = 0.02), although they were nearly as satisfied as those who had received a teleconference (p = 0.37). The doctors quickly learned to exploit the telecommunication model. The responsibility for treatment was maintained, with the primary-care centre in 52% of cases using teleconsultation without any hospital visit required. The GPs and doctors agreed on follow-up treatment.
Tadros et al. 96 compared referral of suspect skin cancers as well as non-malignant symptomatic skin lesions using high-quality digital images transferred via a secure electronic referral system versus conventional pathways. A comparison of the diagnoses made from digital images with the diagnoses confirmed on pathology reports for lesions excised is described using a random selection of patients’ images and referrals. The study concludes that digital image referral for skin malignancy and other cutaneous lesions reduced the interval between referral and diagnosis by 81% and referral to commencement of treatment in suspect lesions by 30%. Diagnostic accuracy in a random sample of 30 patients was comparable with that reported for patients seen in face-to-face consultations. High levels of GP and patient satisfaction were recorded. This study was at higher risk of bias.
Electronic referral systems
Electronic referral systems were reported in 10 studies (Table 12). 97–106 Although each system differed, and included referral to different specialties, they all consisted of referral via an online system as opposed to via letter or e-mail. In one case a clinical reviewer assessed the referral for appropriateness,100 and in a second study a referral pro forma was included to try to guide appropriate referral, but in all other studies all referrals were accepted. Two studies103,104 also included aspects of specialist consultation but this was not the main focus of the intervention. Nine of the interventions reported a positive effect97–105 and all studies were considered to be at lower risk of bias. The evidence was rated as stronger.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Chen 2010100 | E-referral consultation requests compared with paper | BA | USA | Specialists | One hospital: 500,000 outpatients annually | 6 months |
| GPs in five hospitals: 11 community GPs and 10 independent GPs | ||||||
| Dennison 200699 | Electronic surgical referral | Cross-sectional | UK | Colorectal and gastroenterology clinics | 243 referrals | NR |
| 22 GPs in four practices | ||||||
| 54 patients electronically referred, 189 referred on paper | ||||||
| Gandhi 2008108 | Electronic referral tool | RCT | NR | Specialists | 430 referrals | 2 years |
| Jiwa 2012105 | Referral Writer software | BA | Australia | Six specialties | NR | (4 months) |
| Kennedy 2012106 | Electronic referral system | Audit | UK | Oncology (head and neck cancer) | 190 patients | NR |
| 55% female, aged 19–92 years, mean age 58 years | ||||||
| Kim 200998 | Impact of electronic referrals | Cross-sectional | USA | Clinical care | 298 GPs | NR |
| Kim-Hwang 2010102 | E-referral compared with paper | BA | USA | Specialists | 505 specialists | 2 years |
| Nicholson 200697 | Design and delivery of electronic referral system | Audit | Australia | Oncology | NR | 1.5 months |
| Patterson 2004104 | Structured form for neurology referrals | Cohort | UK | Neurology | 76 referrals | 14 months (6 months) |
| 27 male, 48 female | ||||||
| Mean age 44 years, range 16–80 years | ||||||
| Stoves 2010103 | Electronic sharing of health records | BA | UK | Nephrology | 17 practices | NR |
Nicholson et al. 97 completed an evaluation of an online referral and booking system for oncology referrals which included the design, development and deployment of the software in a new approach to information management (similar to choose-and-book system) for suspected cancer referrals. GP satisfaction with the new system was high. Hospital specialists were supportive; however, they noticed little difference in the processes from their perspective. All participants agreed that the system had meant that referrals were being efficiently actioned and that it made the process easy for patients. Patients perceived no major disadvantage.
Kim et al. 98 evaluated GPs to assess the impact of electronic referrals on workflow and clinical care. They distributed an 18-item, web-based questionnaire to 368 GPs who had the option of referring to San Francisco General Hospital. They asked participants to rate the time spent submitting a referral, guidance of work-up, wait times and change in overall clinical care compared with prior referral methods using five-point Likert scales. Over half (55.4%) worked at hospital-based clinics, 27.9% worked at county-funded community clinics and 17.1% worked at non-county-funded community clinics. Most (71.9%) reported that electronic referrals had improved overall clinical care. Providers from non-county-funded clinics (OR 0.40, 95% CI 0.14 to 0.79) and those who spent more than 6 minutes submitting an electronic referral (OR 0.33, 95% CI 0.18 to 0.61) were significantly less likely than other participants to report that electronic referrals had improved clinical care.
Dennison et al. 99 implemented an electronic surgical referral pro forma system, including patient details, symptoms, urgent/routine, provisional diagnosis and a free-text box, for referral to colorectal and gastroenterology clinics. Patients were 21% less likely to change their appointment when referred electronically. Time from referral to appointment was 8 weeks for the electronic system and 10 weeks for the paper system. Time from referral to booking was 0 days for the electronic system, compared with 7 days for the paper system (significantly different; data not given). There was an 8.5% rate of non-attendance in the electronic system, compared with 22.5% in the paper system (significantly different; data not given).
Chen et al. 100 evaluated a new consultation request process, called e-Referral, which was integrated into a hospital’s electronic health record. Clinician reviewers screen requests to evaluate urgency, choice of specialties, whether or not sufficient workup information is provided, and whether a specialist needs to see the patient or can guide the primary care clinician through the e-Referral system. Waiting times for non-urgent visits declined in seven of eight medical specialty clinics by up to 90% during the first 6 months of use. The percentage of referrals deemed inappropriate by medical and surgical specialists was cut by more than half (no data given). For clinics that had been plagued by long waiting times, implementation of e-Referral resulted in dramatic improvements. For example, in rheumatology, the median waiting time for a non-urgent appointment initially dropped from 126 days to 29 days. The majority of primary care clinicians reported that e-Referral improved patient care, but those with poorer access to the electronic health record found it more time-consuming than the previous paper-based system.
Gandhi et al. 108 reported on implementation of an electronic referral tool to analyse its impact on communication between primary care and specialists. They studied one practice site that implemented the referral tool and one that did not, and surveyed affiliated specialists, GPs and patients about referral communication. Specialists more often received information before the referral visit from intervention GPs versus non-intervention GPs (62% vs. 12%, p < 0.001), a finding that persisted after adjustment (RR = 3.3, p = 0.008). Intervention GPs more often received communication from specialists (69% vs. 50%, p = 0.08). Patients of intervention GPs were more likely than patients of control GPs to report that specialists had received information before their visit (70% vs. 43%, p = 0.007).
Kim-Hwang et al. 102 aimed to determine the impact of ‘e-Referral’, compared with paper-based referral, on specialty referral rates. The study was based on a visit-based questionnaire appended to new patient charts at randomly selected specialist clinic sessions before and after the implementation of e-Referrals (using a web-based system). A specialist reviewer (physician or nurse) reviewed the referrals and determined whether or not it was appropriate to schedule an appointment. It was difficult to identify the reason for referral in 19.8% of medical and 38.0% of surgical visits using paper-based methods versus 11.0% and 9.5% of those using e-Referral (p = 0.03 and p < 0.001). Of those using e-Referral, 6.4% and 9.8% of medical/surgical referrals using paper methods versus 2.6% and 2.1% were deemed not completely appropriate (p = 0.21 and p = 0.03). Follow-up was requested for 82.4% and 76.2% of medical and surgical patients with paper referrals versus 90.1% and 58.1% of e-Referrals (p = 0.06 and p = 0.01). Follow-up was considered avoidable for 32.4% and 44.7% of medical/surgical follow-ups with paper-based methods vs. 27.5% and 13.5% with e-Referral (p = 0.41 and p < 0.001).
Stoves et al. 103 evaluated an intervention where the electronic sharing of primary care electronic health records with the nephrology service was introduced to intervention practices. Participating GPs attended education workshops and received paper and e-guidance about the new service. The service allowed GPs to send electronic referrals and share patient electronic health records with a renal specialist after first obtaining verbal patient consent. GPs use criteria agreed in local guidelines to ‘request advice’ or ‘question the need’ for hospital clinic review. There was a significant reduction in paper referrals from intervention practices. The mean [standard error (SE)] interval between the GP sending an e-consultation referral and the renal specialist submitting an electronic response was 7 (0.8) days. This contrasted with a mean wait of 55.1 (1.6) days between the GP sending a paper referral and the patient attending a hospital clinic. When GPs were requesting clinic review by letter, only 56% of referrals were appropriate according to local criteria (71% and 52% for intervention and non-intervention practices, respectively), but 98% of these were accepted for hospital clinic review. By contrast, 90% of e-consultations that questioned the need for clinic review were appropriate, and clinic assessment was recommended in only 27% of cases.
Patterson et al. 104 conducted an intervention to determine if an e-mail triage system between GPs and a neurologist for new outpatient referrals was feasible, acceptable, efficient, safe and effective. A structured form was devised for GPs to refer patients. This set out the required history and examination and was either sent as an e-mail attachment or incorporated into the body text of the e-mail. When the neurologists received the e-mail referral they decided whether or not advice alone was appropriate, whether or not investigations were needed, or whether or not a clinic visit was necessary. When the investigation results were available, either a clinic appointment was made or further advice was given. Forty-three per cent of participants required a clinic appointment, 45% were managed by e-mail advice alone and 12% were managed by e-mail plus investigations. Forty-four per cent of the neurologist’s time was saved, compared with conventional consultation; total time spent was, therefore, 1270 minutes (mean of 16.7 minutes per patient). No deaths or significant changes in diagnosis were recorded during the 6-month follow-up period.
Jiwa et al. 105 explored if increasing the amount of relevant information in referral letters between GPs and hospital specialists helps in the scheduling of appointments for patients. They used Referral Writer software, a software system to assist referral writing, consisting of a pro forma that selects relevant information from the electronic patient record and requests the doctor to choose one of six specialties for referral: urology, breast, gynaecology, upper gastrointestinal (GI), colorectal and respiratory. The doctors were finally prompted to enter details about the patient’s condition. Each GP referred 5.6 patients on average (range 1–14) before the intervention and 4.8 patients (0–14) after it. The amount of relevant information in the referrals improved substantially (mean difference 37%, 95% CI 30% to 43%; p < 0.001). For 91% of referrals after the intervention both specialists in each specialty were confident or very confident that they had enough information to decide when the patient should come to their clinic; this was an increase from 50% before the intervention (p = 0.001). There was no association between the amount of relevant information and the final diagnosis.
One further study of an electronic referral system showed no effect on referral. Kennedy et al. 106 evaluated a fast-track electronic referral system (including referral guidelines) for suspected head and neck cancer. Fifty-two per cent of urgent referrals required no further investigation following assessment and were discharged. Head and neck cancer detection rate (percentage of patients with confirmed diagnosis from total number of referrals) was 8%. Overall cancer detection rate was 15%. During the time period of system operation (1 year), only 14% of the total number of head and neck cancers diagnosed were referred via the electronic system. All others had been referred by non-urgent referral channels (by the same group of practitioners). Twenty-seven different GP practices used the system to refer; however, one city-centre practice accounted for 17% of referrals. Therefore, in this case, 86% of patients diagnosed with cancer bypassed the system.
Decision support tools
The 10 decision support tools all aimed to assist GPs in making referrals and included real-time computer or internet-based systems, as well as a librarian consultation service, the effect of patient-specific ratings versus conventional guidelines, and automatic reporting of estimated glomerular filtration rate (eGFR) to inform referral decision (Table 13). Six studies showed a positive effect,107,109–113 but four reported a negative effect or no effect. 114–117 All studies were considered to be at lower risk of bias. The strength of the evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Akbari 2012110 | Automatic reporting eGFR | BA | Canada | Nephrology | 2672 patients | 2 years (1 year) |
| 12.5% aged 65+ years | ||||||
| Emery 2007111 | Computer decision support system | cRCT | Australia | Regional cancer genetics service | 45 GPs practice teams | (12 months) |
| Greiver 2005114 | PDA software to diagnose angina | cRCT | UK | Cardiology (angina) | 18 GPs | (7 months) |
| 65 patients; patients aged 30–75 years | ||||||
| Junghams 2007109 | Patient-specific ratings | RCT | UK | Cardiology (angina) | 145 GPs | NR |
| Knab 2001112 | Computer-based decision support | BA | USA | Chronic pain referral | 100 patients | 1 year |
| Magill 2009115 | Computer-based referral enhancing | BA | USA | Colonoscopy | NR; patients aged 50+ years | NR |
| Mariotti 2008113 | Prioritisation by GP and specialist | Audit | Italy | Gastroscopy colonoscopy | 438 outpatients | 7 months |
| McGowan 2008107 | Librarian consultation | RCT | Canada | Specialists | 82 GPs; five nurses; one specialist | (24 hours) |
| Slade 2008117 | Referral threshold assessment | cRCT | UK | Mental health | 281 GPs | NR |
| 1061 referrals | ||||||
| Tierney 2003116 | Computer-based care suggestions | RCT | UK | Cardiology | 706 patients | 1 year |
McGowan et al. 107 evaluated whether or not information provided by librarians to answer clinical questions positively impacted time, decision-making, cost savings and satisfaction. The ‘just-in-time information’ librarian consultation service was designed to provide a rapid response to clinical questions during patient visit hours. The questions were submitted by the participants and each question was randomly assigned to the intervention (librarian information) or control (no librarian information) group. If the question was randomised to the control group, participants received a message within 1 minute that their question would not be answered. Each participant had clinical questions randomly allocated to both intervention (librarian information) and control (no librarian information) groups. Participants were trained to send clinical questions via a hand-held device. The average time for ‘just-in-time information’ librarians to respond to all questions was 13.68 minutes per question (95% CI 13.38 to 13.98 minutes). The average time for participants to respond their control questions was 20.29 minutes per question (95% CI 18.72 to 21.86 minutes). Using an impact assessment scale rating cognitive impact, participants rated 62.9% of information provided to intervention group questions as having a highly positive cognitive impact. They rated 14.8% of their own answers to control question as having a highly positive cognitive impact, 44.9% as having a negative cognitive impact and 24.8% as having no cognitive impact at all.
Junghams et al. 109 assessed the effect of patient-specific ratings versus conventional guidelines on appropriate investigation of angina. Intervention physicians received patient-specific ratings (online prompt stating whether the specific vignette was considered appropriate or inappropriate for investigation, with access to detailed information on how the ratings were derived) and control physicians received conventional guidelines from the American Heart Association and the European Society of Cardiology. Physicians made recommendations on 12 web-based patient vignettes before and on 12 vignettes after these interventions. Decisions for exercise electrocardiography were more appropriate with patient-specific ratings [819/1491 (55%)], compared with conventional guidelines [648/1488 (44%)] (OR 1.57, 95% CI 1.36 to 1.82). The effect was stronger for angiography [1274/1595 (80%) with patient-specific ratings compared with 1009/1576 (64%) with conventional guidelines (OR 2.24, 95% CI 1.90 to 2.62)]. Within-arm comparisons confirmed that conventional guidelines had no effect but that patient-specific ratings significantly changed physicians’ decisions towards appropriate recommendations for exercise electrocardiography (55% vs. 42%; OR 2.62, 95% CI 2.14 to 3.22) and for angiography (80% vs. 65%; OR 2.10, 95% CI 1.79 to 2.47).
Akbari et al. 110 assessed whether or not automatic reporting of the eGFR, along with an ad hoc educational component for primary care physicians, would increase the number of appropriate referrals to nephrology. Concurrent with the introduction of automatic reporting of the eGFR, the nephrology service mailed an algorithm to all primary care physicians in the Champlain Local Health Integration Network. This algorithm explained the interpretation of the eGFR and appropriate parameters for referrals to nephrology, based on the value. In addition, ad hoc educational sessions (lectures and workshops) were provided to the primary care physicians to discuss interpretation of the eGFR results and parameters for referral to nephrology. In the year after automatic reporting began, the number of referrals from primary care physicians increased by 80.6% (95% CI 74.8% to 86.9%). The number of appropriate referrals increased by 43.2% (95% CI 38.0% to 48.2%). However, there was no significant change in the proportion of appropriate referrals between the two periods (−2.8%, 95% CI −26.4% to 43.4%). In the year after automatic reporting of the eGFR was introduced, the total number of referrals increased significantly among patients ≥ 80 years (percentage-point change 8.0, p < 0.001) and among women (percentage-point change 12.6, p < 0.001).
Emery et al. 111 evaluated the effect of an assessment strategy using the computer decision support system [the Genetic Risk Assessment on the Internet with Decision Support (GRAIDS) software] on the management of familial cancer risk in British general practice in comparison with best current practice. Training in the new assessment strategy and access to the GRAIDS software (GRAIDS arm) was conducted and compared with an educational session and guidelines about managing familial breast and colorectal cancer risk. All GPs and practice nurses attended a 45-minute educational session on cancer genetics, delivered at their general practice. They were also introduced to the principles of the GRAIDS intervention. There were more referrals to the Regional Genetics Clinic from GRAIDS than to control practices (mean 6.2 and 3.2 referrals per 10,000 registered patients per year; mean difference 3.0 referrals; 95% CI 1.2 to 4.8; p = 0.001). Referrals from GRAIDS practices were more likely to be consistent with referral guidelines (OR 5.2, 95% CI 1.7 to 15.8; p = 0.006). Patients referred from GRAIDS practices had lower cancer worry scores at the point of referral (mean difference 1.44, 95% CI 0.23 to 2.64; p = 0.02).
Knab et al. 112 determined whether or not computer-based decision support (CBDS) could enhance the ability of GPs to manage chronic pain. Structured summaries were generated for 50 chronic pain patients referred by GPs to a pain clinic. A pain specialist used a decision support system to determine appropriate pain therapy and sent letters to the referring physicians outlining these recommendations. Separately, five GPs used a CBDS system to ‘treat’ the 50 cases. One year later, the hospital database provided information on how the actual patients’ pain was managed and the number of patients rereferred by their GP to the pain clinic. On the basis of CBDS recommendations, the GP subjects ‘prescribed’ additional pain therapy in 213 of 250 evaluations (85%), with a medical appropriateness score of 5.5 ± 0.1. Only 25% of these chronic pain patients were subsequently rereferred to the pain clinic within 1 year. The use of a CBDS system may improve the ability of GPs to manage chronic pain and may also facilitate screening of consults to optimise specialist utilisation.
Mariotti et al. 113 evaluated a new method of prioritisation of patients suffering from significant GI disorders needing rapid access to diagnostic procedures. GPs used a ranking of waiting times for different levels of clinical priority called homogenous waiting groups. Specialists assigned a priority level for each patient as well as evaluating the appropriateness of the referral and the presence of significant endoscopic disorders. Agreement between GP and specialist was evaluated. Most referrals (74.4%) were deemed low priority by GPs, with no maximum waiting time assigned. The level of agreement between GPs and specialists with regard to patient priorities was poor to moderate; for gastroscopy the kappa was 0.31 and for colonoscopy it was 0.44. There was an association between the proportion of significant disorders identified with endoscopy and the priority assigned to the referral (χ2 = 18.9; 1 df; p < 0.001). The overall proportion of referrals deemed inappropriate by specialists was 22.1%.
Four further decision support studies showed no positive association with referral outcomes. Greiver et al. 114 determined the effectiveness of a personal digital assistant (PDA) software application to help family physicians to diagnose angina among patients with chest pain. Intervention GPs received a Palm PDA (which included the angina diagnosis software). They prospectively recorded the process of care for patients presenting with suspected angina over seven months. Fourteen of the 28 patients in the control arm (50%) and 30 of the 37 patients in the PDA arm (81%) were referred for cardiac stress tests (p = 0.007), an absolute difference of 31% (95% CI 8% to 58%). There was a trend towards more appropriate use of stress testing (48.6% with the PDA vs. 28.6% control), an increase of 20% (95% CI –11.54% to 51.4%; p = 0.284). There was also a trend towards more appropriate use of nuclear cardiology following cardiac stress testing (63.0% vs. 45.5%), an absolute increase of 17.5% (95% CI –13.9% to 48.9%; p = 0.400). Referrals to cardiologists did not increase (38.2% with the PDA vs. 40.9%, p = 0.869). A referral was more likely to have been made if the final diagnosis was angina (likelihood ratio for referral 15.455, 95% CI 2.124 to 112.431), so GPs appeared to refer appropriately.
Magill et al. 115 evaluated a computer-based system to enhance referral for colonoscopy. The intervention had three components: (1) a pop-up prompt for screening colonoscopy on electronic medical records (EMRs) was modified; (2) education sessions for primary care providers comprising epidemiology of colon cancer, strategies for early detection, how to use EMRs and optimal clinic workflow to facilitate screening were provided; and (3) medical assistants were asked to discuss screening with eligible patients before they were seen by a physician and to initiate preliminary orders for tests. There were also best practice alerts, computerised documentation of referral status and individual physician feedback, which were implemented later. Individual site providers experienced very different local conditions and changes during the course of the project, for example relocation, new services, personnel change, and introduction of revenue for screening site and physician from referrals. At baseline, monthly referral rates were 5–7%. The pop-up prompt and provider education introduced over a 2-month period showed little or no immediate correlation with referral. Initiation of medical assistant workflow change 2 months later was associated with an 11% increase in referral rate. Small increases were observed after best practice alerts and computerised documentation of referral status was implemented 2.5 years after the initial intervention (no details given of these intervention methods). At 4 years, referral rates remained above baseline.
Tierney et al. 116 assessed the effects of computer-based cardiac care suggestions. Evidence-based cardiac care suggestions, approved by a panel of local cardiologists and general internists, were displayed to GPs and pharmacists as they cared for enrolled patients. Evidence-based guidelines published by the Agency for Health Care Policy and Research and national professional organisations were used to develop the cardiac care rules. The cardiac care suggestions were printed at the end of the medication list on the encounter form and displayed as ‘suggested orders’ on GPs’ workstations. GPs could view the guidelines and references via the ‘help’ key. Subjects were followed for 1 year, during which they made 3419 primary care visits and were eligible for 2609 separate cardiac care suggestions. The intervention had no effect on physicians’ adherence to the care suggestions (23% for intervention patients vs. 22% for controls). There were no intervention–control differences in quality of life, medication compliance, health-care utilisation, costs or satisfaction with care.
Slade et al. 117 investigated whether or not introducing a standardised assessment of severity improved referral agreement. Prior to a mental health referral, GPs completed a threshold assessment grid, a one-page assessment of mental health severity, which was then attached to the referral form/letter. Implementation was low and the grid was used with only 25% of referrals. There were no significant differences between trial arms (p = 0.05) for any of the comparisons: appropriateness of referral was 64% versus 60% (intervention vs. control, p = 0.41 adjusted), rating of urgency was 81% intervention versus 76% control (p = 0.15), identification of appropriate professional was 89% intervention versus 87% control (p = 0.46), and time to discuss referral by mental health team was 2.08 versus 2.15 minutes (p = 0.37).
Waiting list interventions
We identified three interventions that consisted of waiting list review (reviewing the condition of patients awaiting a specialist appointment to see if that appointment was still appropriate and required) or watchful waiting (delaying referral to see how a condition developed) (Table 14). 118–120 Only one of these interventions was shown to be effective, although all were considered to be at lower risk of bias. 118 The strength of the evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| King 2001119 | Review of waiting list | BA | UK | Any specialty | 109 referrals | NR |
| Stainkey 2010118 | Review of waiting list | Audit | Australia | Five specialties | 872 patients | NR |
| van Bokhoven 2012120 | Watchful waiting | cRCT | the Netherlands | 498 patients, 63 GPs | NR | |
| Patient mean age 43 years, 28% male | ||||||
| GP mean age 45 years, 74% male |
The effective intervention118 evaluated a specialist appointment service for long-waiting patients. Letters were sent to patients who had been waiting for hospital appointments for 2 years or more (triaged by the hospital as non-urgent). Patients responded and, if they felt that the appointment was still needed, they were seen at specially arranged clinics. In the first wave 16 patients required procedures (of the 101 who had responded to the letter and been seen in a clinic). In the second wave 532 patients responded to the letter and were seen in a clinic. One hundred and seventy-seven patients had surgical procedures resulting from these appointments.
Two further waiting list interventions had no effect on referral: the first119 considered whether or not, in practices with high referral rate, an invitation to review referrals could identify patients on the waiting list who considered their referral unnecessary, leading to a negotiated cancelling of their appointment. Four to seven weeks after referral, selected patients were sent a questionnaire and an invitation to a review their appointment. Exclusion criteria were symptoms that raised the possibility of significant disease; patient’s mental state precluded consent or co-operation; the referring doctor preferred the patient not to participate; and such urgency that an outpatient appointment could be expected within 3 weeks. Of those patients who were contacted, 77 (72%) responded and, of those, 10 (13% of responders) indicated uncertainty that a referral was still needed. Eight of these attended for review, but in none of these cases was the appointment subsequently cancelled. Therefore, taking cancellation of a hospital appointment as an end point, the effect shown is 0 out of 435 referrals and 0 out of 109 in the intervention group (95% CI 0 to 3).
The second study120 evaluated the feasibility of watchful waiting compared with immediate blood test ordering in patients presenting with unexplained complaints that did not cause alarm for the GP, including fatigue, abdominal complaints, weight change, musculoskeletal complaints and itch. Group A took a watchful-waiting approach. Group B included watchful waiting plus a ‘quality improvement strategy’, which consisted of two small group meetings including an explanation of the diagnostic value of tests, a discussion of the difficulties in dealing with patients with unexplained complaints, and goal setting to change GPs’ behaviour. There was no statistically significant difference between the two intervention groups in terms of the number of patients for whom tests were ordered, or GP performance (performs adequate examination, explains findings to patient). First consultation GPs ordered a mean of seven tests in the control group and trained intervention group, and six tests in the untrained intervention group. Fifty-two of the 498 patients returned to the GP after 2 weeks for a further consultation.
System change interventions
We defined system changes as large changes impacting on all referrals made which involved the movement of staff or relocation clinics, the methods in which all referrals were triaged at hospital or financial arrangements for referrals.
System change interventions included the community provision of specialist services by GPs (n = 9), outreach or community provision by specialists (n = 10), return of inappropriate referrals (n = 2), the provision of additional primary care staff (n = 3), the addition or removal of gatekeeping systems (n = 4), changes to payment systems (n = 4), and referral management centre or other major triage systems (n = 6).
Community provision of specialist services by general practitioners
Community provision of specialist services by GPs was reported in nine studies (Table 15). 121–129 The services provided included dermatology services delivered by primary care, ambulatory electrocardiogram (ECG) monitoring in general practice, GP providing minor surgery, a GP headache service, GP with special interest (GPwSI) clinics in primary care, spirometry, and loop electrical excision procedure (LEEP) for cervical dysplasia. Seven studies showed a positive effect on referral outcomes, but two reported a negative effect or no effect. Eight studies were considered to be at lower risk of bias,121–128 with only one study at higher risk of bias (this study showed no effect on referral outcomes). 129 The strength of the evidence was graded as stronger.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Callaway 2000121 | LEEP training for GPs | Audit | USA | Gynaecology | 272 patients; female | (6 years) |
| Levell 2012 129 | Dermatology clinics | BA | UK | Dermatology | NR | 6 years |
| Ridsdale 2008124 | GPwSIs in headache | Audit | UK | Neurology | 117 patients | NR |
| Mean age 41.1 years, 57% female | ||||||
| Rosen 2006128 | GPwSIs | Cohort | UK | All specialties | Four sites | NR |
| Salisbury 2005125 | Primary dermatology service | RCT | UK | Dermatology | 30 practices | (9 months) |
| 556 patients | ||||||
| Sanderson 2002126 | Dermatology in primary care | RCT | UK | Dermatology | 556 patients | (9 months) |
| Sauro 2005127 | GP spirometry | nRCT | Italy | Respiratory (COPD) | 24 GPs | NR |
| 32,785 patients | ||||||
| Standing 2001122 | ECG monitoring by GPs | BA | UK | Cardiology | 73 patients | NR |
| 26 male, 47 female | ||||||
| 71% no cardiac history | ||||||
| Van Dijk 2011123 | Minor surgery by GPs | Audit | the Netherlands | Surgical specialties | 14,202 patients | NR |
| Mean age 39 years; 51% female |
Seven studies showed a positive association between the intervention and referral outcomes. 121–127
The first from the USA121 evaluated whether or not LEEP training for family physicians could impact on referral to gynaecology. Prior to training all patients were referred. After training, the LEEP for cervical dysplasia was carried out by family physicians in a cervical dysplasia clinic. During the study period, 283 women were seen in the clinic, and 26 individuals (9%) were referred by the GP to a consulting gynaecologist. Of the 9% referred to gynaecologist, all but one were subsequently treated with a laser or a combination of a laser and LEEP.
A UK study122 investigated whether or not ambulatory ECG monitoring in general practice could decrease unnecessary referrals and pick up unsuspected cardiac abnormalities. Patients were recruited to use a novel ambulatory ECG machine designed to detect arrhythmias in general practice. Patients were selected if they had signs and symptoms indicative of cardiac abnormalities including dizzy spells, fainting, palpitations or pounding chest, as well as considering their medical history and general profile. Patients made two GP visits. On the first they underwent a normal consultation and the GP recorded any diagnosis made, whether he or she would refer the patient and, if so, what test he or she would request. The ECG device was fitted and the patient was given a diary card and general advice about the equipment. The patient’s ECG signal was then analysed for 24 hours. The patient was instructed to return to the surgery the next day where the GP reviewed the report generated by the equipment and decided whether or not to refer the patient to the cardiology clinic. Following GP assessment prior to using the ECG machine, GPs were intending to refer 49 (68%) to cardiology outpatients for further tests. Of these, three cases were considered to need urgent appointments. The ECG data identified 22 patients with cardiac abnormalities. In seven patients no abnormality was detected, and three further cases gave non-diagnostic results (probably attributable to poor fitting). The number of patients the GPs decided to refer to cardiology outpatients reduced by 60%, from 49 to 19 patients. However, the number of patients identified as urgent increased from three to seven. Thirty-six (of 49) were unlikely to need cardiology referral.
The most recent study123 retrospectively examined associations between the number of minor GP surgical interventions undertaken and hospital referral rates. Electronic medical record data were examined for patients where benign neoplasm skin/naevus, sebaceous cyst or laceration/cut and/or minor surgery was performed by GPs. GP practices that performed more minor surgery had a lower referral rate for patients with a laceration/cut (–0.38, 95% CI –0.6 to –0.11) and for patients with a sebaceous cyst (–0.42, 95% CI –0.63 to –0.16) but not for those with benign neoplasm skin/naevus (–0.26, 95% CI –0.51 to 0.03). Minor surgery was more often performed in older patients. The presence of a primary care nurse only affected referral for benign neoplasm. There was a significant negative correlation between minor surgery intervention and referrals at a practice level (no data given). For laceration/cut and sebaceous cysts, GP practices that perform more minor surgery interventions refer fewer patients to a medical specialist. Performing five more minor surgery interventions per 100 care episodes would result in 4.3 fewer referrals for sebaceous cyst.
Another UK study124 evaluated the training of GPwSIs in headache and the setting up of a GPwSI clinic in general practice, compared with the existing neurology service. A questionnaire survey was conducted, measuring headache impact, satisfaction and cost estimates. There was no significant difference in headache impact between hospital (mean score 61.2, SD 10.4) and GPwSI clinic attendees (mean score 64.3, SD 9.3) after adjustment for age, sex and ethnicity (mean difference 2.7, 95% CI 1.6 to 7.0). Patients were significantly more satisfied with the GPwSI service, particularly that the service was effective in helping to relieve their symptoms (89% vs. 76%; OR 7.7, 95% CI 2.7 to 22.4). The cost per first appointment was estimated to be £136, with £68 for subsequent contacts. These are lower than costs for neurologist contacts.
A further study from the UK125 investigated the effectiveness, cost-effectiveness, accessibility and acceptability of a primary care dermatology service (PCDS) in comparison with a hospital outpatient clinic for dermatology. The PCDS was staffed by two GPwSIs and a specialist nurse, and provided from a suburban health centre. Patients were referred by their GPs to the outpatient dermatology department as usual. Those who appeared on the basis of their referral letter to be suitable for management in the PCDS were given an appointment there rather than at the outpatient department. There were no marked differences between the PCDS and hospital care in respect of clinical outcome (ratio of geometric means 0.99, 95% CI 0.85 to 1.15; p = 0.9, adjusting for baseline and stratification). The PCDS was more accessible [the difference between means on the access scale (scored out of 100) was 14, 95% CI 11 to 19; p < 0.001] and patients had reduced waiting times by a mean of 40 days (95% CI 35 to 46 days, p < 0.001). Fewer PCDS patients (6%) than hospital patients (11%) failed to attend their initial appointment, but overall did-not-attend rates for new and follow-up appointments were similar in both sites (PCDS 8%; hospital 11%). Of those patients seen initially at PCDS, 12% were referred to the hospital for one or more follow-up appointments.
The fourth UK study in this group126 assessed the effectiveness, accessibility and acceptability of a GPwSI service for skin problems compared with a hospital dermatology clinic. The GP clinic was staffed by two GPwSIs and a specialist nurse. A consultant dermatologist provided clinical support for two sessions per month. No noticeable differences were found between the groups in clinical outcome (median dermatology life quality index score of 1 both arms, ratio of geometric means 0.99, 95% CI 0.85 to 1.15). The GPwSI service was more accessible (difference between means on access scale 14, 95% CI 11 to 19) and patients waited a mean of 40 (95% CI 35 to 46) days less. Patients expressed slightly greater satisfaction with consultations with a GPwSI (difference in mean satisfaction score 4, 95% CI 1 to 7), and at baseline and after 9 months 61% said that they preferred care at the service.
An Italian study127 considered the effect of training GPs to perform spirometry on the management of COPD and asthma. There were three study groups (it is not clear if they were randomly allocated): group 1 GPs received a spirometer and practice training in its use, including information on guidelines (n = 11,050); group 2 received only guidelines (no spirometer or training) (n = 11,040); and group 3 was the control group (n = 1049). COPD was diagnosed in 5.8% of group 1, 1.5% of group 2 and 2.3% of group 3 (p < 0.001). Group 1 performed the test in 65.7% cases of COPD or asthma. Group 2 referred 7.8% of patients. The control group requested the test in 96.8% of the cases. There were significant differences between prescribing and/or utilising spirometry between all three groups (p < 0.001, data not given). Group 1 referred 7.5% to a specialist and diagnosis was confirmed in 91.8% of cases. Group 2 sent 7.8% to the specialist and diagnosis was confirmed in 75.8%. The control group referred 96.8% of patients, of whom 27.2% only had a confirmed diagnosis.
Two other UK studies showed no association with referral outcomes (one showed a strong negative effect on referral numbers). 128,129
The first128 compared referrals from GP practices that had access to GPwSI clinics and those that did not. They found that the association between the introduction of GPwSI clinics and hospital referral rates was variable and unpredictable. There were no significant changes in hospital referral rates following the introduction of GPwSI clinics in any of the sites studied. Overall referrals to hospital and GPwSI clinics combined increased in the three sites for which data were available. The likelihood of referral, calculated as the RR, adjusted for baseline and linear time trend, did not change after the launch of the GPwSI clinics in any of the sites studied. Small changes in risks of referral from studying control practices did not reach statistical significance. In one site, where all practices had access to GPwSI clinics, there was a significant (p = 0.08) 13% increase in overall referrals.
The second129 assessed the effect of introducing dermatology integrated intermediate care services on the numbers of dermatology referrals to secondary care. The dermatology intermediate care service was set up in 2005, providing services in two locations by two GPwSIs in dermatology. The GPwSIs were supported by experienced dermatology nurses and in total six clinics weekly were held, seeing approximately 30 new patients weekly. The numbers of dermatology new patients seen in secondary care, which had been stable for 5 years, showed an increase in 2007 followed by a substantial increase in 2008 and then 2009. The mean number of new patients seen in dermatology in 2004–6 was 6927 patients per year; in 2007, the mean number was 7844 patients; and the mean number of new patients seen between 2008 and 2010 was 11,535 patients per year. This was an increase of 67% in the number of new patients seen. Overall, over this period, there was a 23% increase in new dermatology patients seen in secondary-care dermatology in England. This study was at higher risk of bias.
The majority of interventions in which GPs were trained to provide specialist services in the community were effective at preventing referrals to secondary care. The two studies which did not show a positive effect consisted of GPwSIs rather than GPs who were trained to undertake a specific procedure. However, three other GPwSI interventions were shown to be effective. This could not be separated by condition, as of the two GPwSIs in dermatology studies, one was shown to be effective and one was not.
Community provision by specialists
Community provision by specialists was reported in 10 studies (Table 16). 130–139 The interventions consisted of specialist outreach clinics for diagnosis and treatment where appropriate, development of multidisciplinary mental health teams in primary care, acupuncture in primary care, manual therapy as part of a community-based musculoskeletal service, and an outreach surgical service offering open-access endoscopy to rural areas. Of the 10 studies, eight reported positive effects,130–137 with two reporting a negative effect or no effect138,139 (including one at higher risk of bias139). Nine of the studies were considered lower risk for bias,130–138 and the strength of the evidence was graded as stronger.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Campbell 2003131 | Specialist outreach clinic | cRCT | UK | Cancer genetics service | 203 GPs | NR |
| Women: family breast cancer | ||||||
| Felker 2004132 | Multidisciplinary mental health team | BA | USA | Mental health | 9656 patients | 2 years (1 year) |
| Mean age 53 years; 90% male | ||||||
| Gurden 2012133 | Community musculoskeletal service | BA | UK | Musculoskeletal | 696 patients | (≈8 weeks) |
| Back or neck pain | ||||||
| Mean age 52 years; 66% female | ||||||
| Hermush 2009137 | Caring for the elderly in the community | BA | Israel | Geriatrics | 512 patients | (3 years) |
| Mean age 79 years; 66% female | ||||||
| Hughes-Anderson 2002136 | Outreach endoscopy | BA | Australia | Endoscopy | 4400 patients | 5 years |
| Mean age 50.8 (15–94) years; 45% female | ||||||
| Johnson 2008 139 | Acupuncture in primary care | Audit | UK | Acupuncture | 109 practices | NR |
| Leiba 2002130 | Specialist outreach clinic | nRCT | Israel | All specialists | 136 patients; 20 GPs | NR |
| Pfeiffer 2011138 | Primary mental health services | Audit | USA | Mental health | 49,957 patients | NR |
| Mean age 55.7 years; 93% male | ||||||
| Schulpen 2003134 | Joint consultation sessions | nRCT | the Netherlands | Rheumatology | 17 GPs | 2 years |
| Mean age 48.5 years, 12% female | ||||||
| Vlek 2003135 | Joint consultation sessions | RCT | the Netherlands | Cardiology | 49 GPs | 1 year |
| 13 cardiologists | ||||||
| 306 patients; mean patients age 58 years |
A study from Israel130 evaluated a specialist outreach clinic established in a home-front military primary-care clinic. Patients were initially referred, but no further referral was required for continuity of specialist care. The same analysis was applied to a similar clinic employing only GPs, which refers to military specialist centres or hospital outpatient clinics. The incorporation of specialists did not result in a significant increase in the overall consumption of medical services (p < 0.05). It reduced the number of referrals out of the clinic to specialist centres from 1449 to 421 per month (p < 0.05). In the control clinic, referrals to distant specialist centres and outpatient clinics showed a slight and non-significant increase. Number of work-days lost was reduced from 2891 days per month to 1938 days per month (p < 0.001). The total cost of all medical interactions and referrals did not significantly increase after the introduction of the outreach specialist clinic (p < 0.05). Primary physicians graded their satisfaction with the new clinic as 4.5 (out of 5).
Campbell et al. 131 evaluated specialist outreach clinics in rural Scotland. Women with a family history of breast cancer were referred to a clinic held in a community setting near to the GP practice rather than receiving an appointment to see a consultant geneticist and breast surgeon at a regional centre. Referral rates rose from 2 years before the trial to during the trial (0.21 to 0.31), a 48% increase in referral rate (p < 0.001). Forty-three per cent of women asked to be referred and younger women were more likely to have taken the initiative to request referral (p = 0.001). There was a substantially greater increase in referral rates to community clinics than to the regional centre (64% increase vs. 38% increase), suggesting that providing a service in the community resulted in a change in GP referral behaviour. This was particularly apparent in practices in relatively deprived communities. There were higher referral rates from practices with more female partners before and during the trial (p < 0.005 and p < 0.02).
A study from the USA132 evaluated the effect of a multidisciplinary mental health care team in primary care. A multidisciplinary mental health team was created consisting of a psychologist, a psychology intern, psychiatry residents, clinical social workers and a chaplain. Before implementation 543 consultations occurred over the year. Of these, 543 (38%) were subsequently referred to specialty mental health care services. The following year, 560 consultations occurred, but only 81 (14%) were referred. The change in referral rate was significant (χ2 = 77.85, df = 1; p < 0.001).
The most recent study133 evaluated a community-based musculoskeletal service. Patients still having pain after 4–6 weeks of ‘usual GP care’ were offered a course of manual therapy and referred to a private provider of their choice for chiropractic, osteopathy and physiotherapy services. The percentage change in scores from baseline to discharge were as follows: Bournemouth Questionnaire, 64.6% patients categorised as improved; Bothersomeness scale, 69.9% patients categorised as improved; and Global Improvement Scale, 67.8% patients categorised as improved. Overall, 99.5% were satisfied or very satisfied with the treatment and only 3% were referred back to the GP with a recommendation for referral to secondary-care services (97% were given self-management advice and recommended for discharge).
Schulpen et al. from the Netherlands134 evaluated joint consultation sessions between GPs and a consultant held 6-weekly which consisted of three GPs and one visiting rheumatologist at the practice of a host GP. The GPs presented each patient, and the consultant examined the patient and formulated a diagnosis and therapy policy together with the GP. Prior to intervention there was an increasing referral rate to the hospital rheumatology department. By the end of the study period, the number of patients referred by each GP per year differed by –62% in the intervention group, compared with the controls. The average reduction in referral rate to rheumatology was –2.8 (SD 3.9) at the end of the second year of the intervention period, compared with the first year in the intervention group. In the control group the referral rate difference was zero (SD 2.1). The difference in referral rate between the intervention and control groups both before and after the intervention was significant (p = 0.024, Mann–Whitney U-test). Based on referral rates prior to the intervention, if all patients had been referred to a normal outpatient clinic they would have taken 307.8 hours of consultant time. If all referrals during the study period had been seen via the joint clinic system this would have used 166.7 hours. The authors argue that there was, therefore, a decrease of 46% in time spent by rheumatologist consultants.
A second study from the Netherlands in this group135 evaluated monthly joint consultation sessions between GPs and cardiology specialists held over 18 months in the surgery of the GP. Three to four patients could be examined and discussed at each session and there were an average of seven sessions per GP (range 2–13 sessions per GP). Fewer patients in the intervention group than the control group were referred to a cardiologist (33% vs. 52%, p = 0.001). The difference in referral rates showed an average decrease of referrals to cardiology of 6 per 1000 patients in the GPs from the intervention group. Further diagnostic procedures were required for 7% in the intervention group versus 16% in control group (p = 0.013).
Hughes-Anderson et al. 136 assessed whether or not an Australian outreach surgical service offering open-access endoscopy to rural areas was being overutilised. Indications for referral between the GPs and the visiting surgeons were reviewed in patient records and assessed for compliance with guidelines. Two groups of patients were defined: those referred directly for open-access endoscopy and those selected by the surgeons. A total of 772 endoscopies were performed and 75% were booked as open-access services. The referral rate for procedures was greater for GPs (583: 75%) than for the visiting surgeons (189: 25%). The overall compliance rate for approved indications using the guidelines for both groups was 92%. There was no significant difference in pathology found between groups. The difference between GPs and visiting surgeons for the number of appropriate indications for endoscopy was 3.2% (95% CI 1.8% to 8.2%; p = 0.348, not significant). The difference between GPs and visiting surgeons (appropriate indications) for colonoscopy was 6.8% (95% CI 1.8% to 15.4%; p = 0.148, not significant).
A study from Israel137 evaluated a new model used in caring for the elderly in the community. GPs referred difficult or complex cases to a geriatrician who carried out a clinic in the same primary-care location. Referrals to a geriatrician increased significantly from 133 at baseline to 207 2 years later (p = 0.01). The number of visits to GPs decreased in the 6 months following the consultation with the geriatrician (p < 0.01).
Two further studies did not show clearly positive association with referral outcomes: the first138 determined whether or not the implementation of primary care mental health services is associated with differences in specialty mental health clinic use. The US Veterans Health Administration is a primary care mental health service providing collocated collaborative mental health specialists and managers for screening and managing common mental health conditions (e.g. depression or alcohol misuse). Initiation of treatment at the specialty mental health clinic did not differ between primary-care services with mental health facilities and those without (5.6% vs. 5.8%). Attendance at a primary-care service for mental health was not a predictor of total number of specialist mental health clinic visits.
The second study, from the UK,139 evaluated the provision of acupuncture in primary care and whether or not it resulted in a reduced need for referral to secondary care. They found ‘no evidence from the data that provision of acupuncture is associated with lower referral rates’. The data presented outline mean referral rates for practices providing acupuncture clinics and ‘some’ versus ‘higher’ numbers of acupuncture appointments, but not for practices with no acupuncture, so this conclusion may need modification. They also report a wide variation between different PCTs, possibly associated with local differences in referral patterns and sociodemographic characteristics. This study was at higher risk of bias.
Return of inappropriate referrals
Interventions consisting of the return of inappropriate referrals were reported in two UK studies (Table 17). 18,140 The interventions consisted of a restricted-referral guideline issued to GPs for dermatology, including a list of conditions for which the dermatology service would no longer see patients, and a clinic returning patients referred for erectile dysfunction to the referrer (either in writing or by telephone). Both studies showed positive effects, with one at lower risk18 and the other at higher risk of bias. 140 The strength of the evidence was graded as weaker.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Tan 2007 140 | Refuse referral for certain conditions | Audit | UK | Dermatology | NR | NR |
| Wylie 200118 | Return of referrals for erectile dysfunction | Audit | UK | Erectile dysfunction | 796 referrals | NR |
The first study18 compared the prescribing pattern and attitude of GPs in response to a clinic returning a patient referred for erectile dysfunction with the referrer by two different methods. Referrals on a waiting list for an assessment of erectile dysfunction were reviewed and a subgroup of patients was identified who had criteria enabling them to be eligible for a prescription under the NHS. The GP was informed either in writing or by telephone that the clinic had written to the patient, suggesting that he make direct contact with his GP. The long waiting time for assessment had led to 35% of patients having already tried drug therapy, and by the time the questionnaire was completed, 57% of patients had tried drug therapy. Ten times as many referrers indicated that they were happy to initiate a prescription for drug therapy than not to do so, for those men eligible for an NHS prescription. More GPs who had received a letter returned the completed questionnaire (80%) than those who had received a courtesy telephone call (64%). There were no differences between the groups of GPs in their attitude to contact with their patient and no difference in prescribing pattern.
The second study140 evaluated the impact of a restricted-referral guidance issued to GPs for dermatology referrals inspired by the Oregon Health Plan, a rationing policy. A list of conditions that the service would no longer treat or treat only in exceptional circumstances was circulated to all GPs. Referrals for these conditions were returned. Following the introduction of the new policy, a reduction in the rate of referrals occurred. For a further 3–4 years post intervention the volume of new referrals remained static. The data are presented only in the form of a chart; there was a peak of 800 new referrals per year before the intervention, falling to around 600 referrals per year post intervention. This study was at higher risk of bias.
Additional primary care staff
Three studies reported on the provision of additional primary care staff: primary care nurses, and counsellors (Table 18). 141–143 However, all showed no effect (or very limited effect) on referral outcomes, with one graded as being at higher risk of bias143 and the other two being graded as lower risk. 141,142 The strength of evidence was graded as stronger, but it is important to note that the evidence was in a negative direction here, that is, more staff adversely impacted on demand management outcomes.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Simpson 2003 143 | Counsellors in primary care | BA | UK | Mental health | 85 practices | 8 years |
| Van Dijk 2010141 | Primary care nurses | Audit | the Netherlands | Diabetes referral to internists, ophthalmologists, cardiologists or mental health care | 54 practices | NR |
| 751 patients; 50% male; mean age 61 years | ||||||
| White 2000142 | Counsellors in primary care | Cross-sectional | UK | Mental health | 180 referrals | NA |
A study from the Netherlands141 assessed whether or not the introduction of primary care nurses affected referral rate for diabetes-related hospital treatment (referrals to internists, ophthalmologists, cardiologists or mental health care). Referral rate to internists for newly diagnosed patients decreased for practices both with and without a practice nurse between the two time points (7.3% vs. 3.3%). The trend in referral patterns to internists for known diabetic patients was lower in general practices with primary care nurses than those without (OR 0.59, 95% CI 0.31 to 1.11; p < 0.1). The number of diabetes-related contacts did not differ between practices with and without primary care nurses.
The first of two UK studies142 evaluated the impact of counsellors in primary care on referrals to mental health services. A counsellor was present at 20.3% of practices. A random sample of 180 referrals to community mental health teams was reviewed: 76 (42.2%) from practices that employed a counsellor and 104 (57.8%) from practices that did not. There was a significantly higher referral rate from practices that employed a counsellor (p = 0.003). However, there was no evidence of a difference in rates of appropriateness of referrals between practices that employed a counsellor and those that did not.
The second UK paper143 also investigated the effect of employing counsellors in general practice on referral rates to mental health services. The practice-employed counsellors were well established and practices were allocated 6–12 hours per week. The findings suggest that the cost of the counsellor could be offset elsewhere. The provision of counselling had no statistically significant effect on referrals or the volume and cost of prescribing.
Gatekeeping systems
Interventions that involved the addition or the removal of gatekeeping systems (primary-care control of hospital referral) were reported in four studies (Table 19). 144–147 In two studies (by the same author),145,146 multispecialty primary-care gatekeeping was removed so that patients were able to schedule an appointment directly with any specialist. The other two studies144,147 compared open-access with physician-approved referral. Overall, the studies showed no significant effect (or only a borderline significant effect) on referrals irrespective of whether gatekeeping was added or removed in the intervention. One study was at higher risk of bias,146 with the other three being rated as lower risk of bias. 144,145,147 The strength of the evidence was rated as stronger; however, it is important to note the bidirection of evidence and that all studies were from the USA.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Ferris 2001145 | Removal of gatekeeping | BA | USA | All specialists | 59,997 patients | 6 months |
| Mean age 41.7 years, 53% female | ||||||
| Ferris 2002 146 | Removal of gatekeeping | BA | USA | All specialists | 59,952 patients | NR |
| Joyce 2000147 | Open access vs. gatekeeping | Audit | USA | All specialists | 53,011 patients, working age | 2 years |
| Schillinger 2000144 | Open access vs. physician approved | RCT | USA | All specialists | 2293 patients | 1 year |
The first paper144 evaluated the effect of open-access versus physician approval of referral to specialist services (and to emergency departments). Intervention patients required prior approval from their primary-care physician in order to receive specialty care at the local hospital. A computer programme blocked the scheduling of unapproved appointments for these patients. Primary-care physicians were required to complete a consultation form including clinical information and number of visits requested prior to the unlocking of the system. For control patients, physician approval was not required prior to accessing services, and both self-referral or physician referral were permitted. Intervention patients decreased specialty use by 0.57 visits per year more than control patients (95% CI –1.05 to –0.01; p = 0.04). The intervention group increased primary-care use; however, this change was not significant. Changes in patient satisfaction with care, perceived access to specialists and use of services were similar between the two groups.
The second paper145 evaluated the elimination of a gatekeeping system. The need for referral from a primary-care provider was removed and patients were able to call and schedule an appointment with any specialist in the group. Rates of visits to specialists were stable during the baseline period and during the intervention period. However, first visits to specialists increased slightly from 0.19 to 0.22 per patient per 6-month period (p < 0.001). The average proportion of visits to eligible specialists as a percentage of all visits was 29% during the year before the removal of gatekeeping and 29.6% during the year afterwards (p = 0.39).
The third paper146 also evaluated the elimination of a gatekeeping system in a separate population. Elimination of gatekeeping was not associated with changes in the mean number of visits to specialists (0.28 visits per 6 months before and after gatekeeping was removed), or the percentage of all children visits to specialists (11.6% vs. 12.1%, 95% CI 29.4% to 31.8%, vs. 11.8% to 12.4%). However, new patient visits to specialists by children with chronic conditions as a percentage of all specialist visits increased from 28.1% (95% CI 25.9% to 30.2%) to 32.2% (95% CI 30.1% to 34.5%). This study was at higher risk of bias.
The fourth paper in this group147 assessed utilisation of visits to primary-care physicians and to specialists in two different managed care models: a closed-panel gatekeeper model and an open-panel point-of-service model. Both plans shared the same physician network. There were more annual visits to primary care and a greater number of total physician visits in the gatekeeper model than in the point-of-service plan. However, there was no difference in rates of specialist visits between the systems.
Payment systems
Changes to payment systems were reported in four studies (Table 20). 148–151 The system changes were described as (1) change from a contract system (whereby the GP receives a fixed practice allowance plus charges fee per item to each patient) to a capitation system where GP income is based on the number of patients on their list; (2) all GPs regardless of training or practice location receive higher Medicare rebates to complete GP mental health plans and for mental health consultations; (3) replacing separate remuneration systems for publicly insured patients (capitation) and privately insured patients (fee-for-service) with a combined system of capitation and fee-for service for all; and (4) introducing a co-payment system – patient payment for attending specialist consultation. One study showed a positive effect on referral outcomes, with three studies showing a negative/no effect. One study was graded as being at higher risk of bias151 (with the other three being judged as lower risk of bias148–150). The strength of the evidence was graded as inconsistent, with none of this group reporting UK data.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Iversen 2000 151 | Payment system for GPs | Economic analysis | Norway | All specialties | 150 GPs | NR |
| McGarry 2009148 | Government spending on mental health | Audit | Australia | Mental health | 44 GPs | 5 years |
| Van Dijk 2013149 | Changes to payment systems | BA | the Netherlands | All specialties | 39,828 patients, 52 GPs | 7 years |
| Vardy 2008150 | Copayment system | Audit | Israel | All specialties | 3745 patients, 48 GPs | NR |
| GPs: 54% female, mean age 45.5 years |
McGarry et al. 148 examined changes in patient management and referral for care following the Better Outcomes in Mental Health Care (BOiMHC) programme initiative in Australia. The BOiMHC programme allows all GPs to refer patients for psychological health care under Medicare. GPs working in accredited practices who had completed accredited mental health training were able to receive service incentive payments for providing care to patients with International Classification of Diseases, Tenth Edition (ICD-10)-diagnosed mental illness. All GPs regardless of training or practice location receive higher Medicare rebates to complete GP mental health plans for patients with ICD-10-diagnosed mental illness, as well as higher rebates for mental health consultations. Significantly higher rates of referral for psychological treatments were reported in 2006 than in 2002. Significantly higher proportions of responders in 2006 reported referring half or more of their patients with mild to moderate depression (p < 0.001) for cognitive–behavioural therapy (p < 0.001).
A study from the Netherlands149 investigated the effects of replacing separate remuneration systems for publicly insured patients and privately insured patients with a combined system of payment. Guideline adherence increased between 2002 and 2008 by 7% for (formerly) publicly insured patients and 10% for (formerly) privately insured patients. In general, there were no significant differences in the trends for guideline adherence between privately and publicly insured patients, indicating the absence of an effect of the remuneration system on guideline adherence.
Vardy et al. 150 evaluated a copayment system in Israel which consisted of a payment per patient for attending a specialist consultation. The payment was described only as ‘a relatively low fixed sum to be paid prior to the appointment’. Attendance at planned appointments was 85% for specialist appointments in the community and 91.7% for specialist hospital appointments in the time period when copayment was in operation. There was no difference in self-referral and physician referral rates. Only 2% reported copayment as the reason for not attending, compared with 19% who stated that copayment was a reason for not attending an appointment in the past. Physicians stated that a need for copayment influenced their referral decision, especially with elderly or lower-income patients.
A Norwegian paper151 explored whether or not a payment system for GPs has an impact on referral. The intervention consisted of a change from a contract system (whereby the GP receives a fixed practice allowance, plus charges a fee per item to each patient) to a capitation system where each person registers with a particular GP and GP income is based on the number of patients on their list. In the capitation system where GP income is determined by the number of patients on the list, the GP referral rates to specialists increased by 42%. It was hypothesised that it is less profitable for the GP to provide services themselves and more profitable for them to let the specialists provide the services.
Referral management centres
Referral management centres or other major triage systems were reported in six papers (Table 21). 152–157 All but one155 reported UK studies. The interventions included two city-wide gateways for triage from general practice to specialist care, single-point referral systems for adult learning disability health services and old age psychiatry, a common pathway for all musculoskeletal referrals, and a gatekeeping and appropriateness review for diabetes referral. Three studies showed a positive effect on referral outcomes,152–154 with three studies showing a negative/no effect. 155–157 Two of the studies that showed a positive effect were graded at higher risk of bias153,154 (with the other four being lower risk for bias152,155–157), and the strength of the evidence was graded as inconsistent.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Cox 2013156 | Introduction of referral management centres | BA (retrospective) | UK | All specialties | 376,000 patients | 3 years |
| 85 practices | ||||||
| Ferriter 2006157 | Single assessment process | BA | UK | Psychiatry | 20 referrals | NA |
| Kim 2004155 | Diabetes referral management centre | Audit | USA | Diabetes specialists | 6941 patients | (1 year) |
| Mean age 61 years; 54% female | ||||||
| Maddison 2004 154 | Early access to musculoskeletal services | BA | UK | Musculoskeletal | NR | 18 months |
| Watson 2002152 | Single-point referral system | Audit | UK | Adult learning disability services | NR | NR |
| Whiting 2011 153 | Manchester referral gateway | Audit | UK | Eight specialties | Four practices | 5 months |
The first paper152 evaluated the impact of introducing a multidisciplinary single point of referral (SPR) system for dedicated adult learning disability health services. They completed a retrospective case note review comparing referrals to a SPR system with those to the old referral system. The SPR system used common referral criteria and a streamlined information system. A new referral form and information leaflet were developed and copies distributed to social workers, data centre managers, GPs and colleges of further education. With the introduction of the SPR system, the mean waiting time for referral to assessment was reduced from 46 (15–67) days to 6 (2–9) days. The proportion of inappropriate referrals halved from 26% to 13%. The proportion of appropriate referrals that involved more than one dedicated learning disability health professional increased from 63% to 80%.
Whiting153 evaluated development of a Manchester-wide referral gateway for triage from general practice to specialist care (including referrals to general surgery, ophthalmology, cardiology, ENT, trauma/orthopaedics, gynaecology, urology and dermatology). Referrals were electronically screened at three stages using a single standard referral letter template. At stage 1, GP referrals were checked for completeness (NHS number, date of birth, etc.), and checked against local non-commissioned policy. At stage 2, if data were missing, or the procedure was not commissioned, an electronic advice note was sent back to the GP practice. Stage 3 was clinical triage consisting of three outcomes: referral continues; referral diverted to an alternative service or advice and guidance from Map of Medicine, NICE or the local commissioner; or referral sent back to the GP to encourage more work-up or increase management in primary care. The process was completed within 2 working days. There was a 1.2% reduction in outpatient activity (compared with the 3.8% growth predicted before the intervention). No further data were reported. This study was at higher risk of bias.
The third paper in this group154 assessed the impact of a Targeted Early Access to Musculoskeletal Services (TEAMS) programme on accessibility to musculoskeletal services. The intervention established (with central clinical triage) a common pathway for all musculoskeletal referrals so that patients attended the appropriate department. A back pain pathway led by physiotherapists was developed, and GPwSIs and physiotherapists were trained to provide services for patients with uncomplicated musculoskeletal problems in the community. After the introduction of intervention, there was a major increase (116%) in the total number of referrals for musculoskeletal problems. In contrast, the number of orthopaedic referrals was slightly reduced. Over 18 months the total number of referrals more than doubled. Despite this, waiting times for musculoskeletal services fell; this was noticeable for rheumatology and pain management (primary data not given).
The only non-UK study155 examined the effect of referral management on diabetes care by evaluating Translating Research Into Action for Diabetes (TRIAD), a multicentre US study of managed-care enrolees with diabetes. Prospective referral management consisted of gatekeeping and mandatory authorisation from the management office. Retrospective referral management consisted of referral profiling and appropriateness reviews. Referral management was commonly used by health plans (55%) and provider groups (52%). In adjusted analysis, there were no associations between any of the referral management strategies and any of the referral outcome measures.
The most recent paper156 reported an evaluation to establish whether or not the introduction of referral management centres was associated with a reduction in hospital outpatient attendance rates. Eighty-five GP practices formed five groups to manage referrals. Two groups also carried out peer review of referrals. The referral management interventions were more complex than internal peer-review controls, involved a wider range of activities, and included activities not directly related to referral management (no further information on these differences is given). Four groups showed statistically significant increases in attendance rates, ranging from 0.41 to 1.20 attendances per 1000 persons per month. After correction, only one group (a referral management centre) remained significant (1.05 attendances per 1000 persons per month, 95% CI 0.64 to 1.64; p < 0.005).
The final paper in this group157 aimed to identify changes in the quality of information in referrals to an old age psychiatry service before and after the introduction of the single assessment process. The single assessment process was introduced in response to the National Service Framework for Older People, to facilitate referrals between agencies and reduce duplication for patients, carers and clinicians. All referrals between agencies were expected to be made on designated forms. The referral form consists of several free-text sections: identity of patient and carer; identity of referrer; reason for referral; assessment of urgency; risk factors; current services provided to patient; diagnosis and recent history; current medication; and signature of referrer. Two senior clinicians performed independent and masked rating of each referral, using a five-point Likert scale. The authors report that referrals were worse in all areas of quality of referral information after implementation of the single assessment process. Word count decreased from 240 (SD 120) to 129 (SD 39) (p = 0.005). Time to read in seconds increased from 96 seconds (SD 40 seconds) to 124 seconds (SD 41 seconds) (p = 0.001). Illegible sections (% of) increased from 2 (10%) to 6 (30%) (p = 0.011). The number of raters who strongly agreed or agreed with the statement: ‘I am able to judge the appropriateness of the referral’ decreased from 19 to 5 (p = 0.001). ‘I would need to seek further information before processing this referral’ increased from 3 to 17 (p = 0.001). ‘Overall I think the referral is useful’ decreased from 17 to 3 (p = 0.001).
Patient-focused interventions
We found few examples of patient-focused interventions. The papers we identified comprised two evaluating the provision of health information/education, and one intervention aiming to address patient concerns and satisfaction (Table 22). 158–160 The first study showed no effect and was scored at higher risk of bias. The second showed a positive effect and scored lower risk of bias. The education interventions were graded as inconsistent and the small number of papers led to a strength of evidence grading as ‘no evidence’.
| Study (first author and year) | Intervention | Design | Country | Specialty/treatment | Sample size and details where provided | Study duration (follow-up) |
|---|---|---|---|---|---|---|
| Albertson 2002158 | Recognition of patient concerns | BA | USA | All specialists | 12 GPs | NR |
| 495 patients | ||||||
| Heaney 2001 159 | Patient information booklets | RCT | UK | All specialists | 4878 patients | 12 months |
| 20 GPs | ||||||
| Lyon 2009160 | Raising community awareness | BA | UK | Cancer | NR | 12 months |
A US study158 determined whether or not a brief pre-visit questionnaire about referral concerns could improve primary-care provider recognition of patient concerns and satisfaction with care. Patients were given a pre-visit questionnaire about referral need and rationale and a post-visit questionnaire about referral concern and visit satisfaction. Providers were given a post-visit questionnaire asking whether a referral was discussed or made, and about visit satisfaction. In the control phase, patient pre-visit questionnaires remained confidential, whereas in the intervention phase GPs were shown the pre-visit questionnaire at the time of the encounter. The intervention significantly increased GP referral recognition from 61% to 81% (p < 0.001) and was associated with increased visit satisfaction (p = 0.05). Satisfaction of GPs with the referral discussion, overall rate of referral and visit duration was not affected by the intervention.
The first UK paper159 investigated the effect of patient information booklets on overall use of health services. One of two booklets was posted to participants in intervention groups. Patients randomised to the control group did not receive a booklet. ‘What Should I Do?’ was part of a patient education programme that had been implemented in the Netherlands. The booklet outlines 40 common health problems and provides information on when to consult a doctor and when self-care is appropriate. The ‘Health Care Manual’ was developed by a GP and a practice nurse in Scotland. It outlines 50 common health problems and also provides information about keeping healthy. Receipt of either booklet had no significant effect on health service use, compared with the control group (difference 0.14, 95% CI –0.18 to 0.45).
Lyon et al. 160 conducted a UK intervention which involved local people working in partnership in their communities to raise awareness of cancer symptoms and promote early presentation. The teams worked with primary care, with other statutory organisations and with the voluntary sector. The specific contribution of the local people was in the identification of hard-to-reach groups and the tailoring of effective health messages. Interim results showed an increase in the number of urgent 2-week referrals and the proportion of new cancer cases diagnosed through the urgent 2-week referral route (from 43% to 51%) for all breast, lung and bowel cancers. These results were statistically significant for the bowel cancer (χ2 = 22.193, df = 1; p < 0.001) and lung cancer pathways (χ2 = 8.886, df = 1; p = 0.003). There was also an increase in the proportion with no spread at the time of diagnosis for bowel cancer (38% to 43%) and breast cancer (41% to 44.5%), but these results did not reach statistical significance.
Intervention outcome measures
In addition to synthesising the evidence by intervention type, we examined the main outcome measures reported in each intervention study (Table 23). As with the types of interventions outlined above, we evaluated the strength of evidence which supported interventions having an effect on this range of outcomes (see Figure 2). The outcomes reported in the literature were as follows.
| Primary outcome | Studies reporting positive effect on outcome (first author and year) | Studies reporting no effect on outcome (first author and year) | Strength | ||
|---|---|---|---|---|---|
| Controlled study/RCT/cRCT/nRCT/CBA | Other | Controlled study/RCT/cRCT/nRCT/CBA | Other | ||
| Referral | Bridgman 2005,70 Julian 2007,62 Kerry 2000,59 Ramsay 2003,27 Salisbury 2005,125 Sauro 2005,127 Schulpen 2003,134 Shaw 2006,77 Vlek 2003,135 Wolters 2005,36 Wright 200656 | Albertson 2002,158 Callaway 2000,121 Campbell 2003,131 Cooper 2012,19 Cusack 2005,43 Elwyn 2007,55 Glaves 2005,57 Gurden 2012,133 Hands 2001,34 Hermush 2009,137 Hockey 2004,91 Kim 2009,98 Knol 2006,90 Lam 2011,25 Levell 2012,129 Maddison 2004,154 McGarry 2009,148 McKoy 2004,89 Simpson 2010,78 Standing 2001,122 Stoves 2010,103 Suris 2007,35 Tan 2007,140 Thomas 2010,80 Twomey 2003,42 Van Dijk 2011,123 Whited 200287 | Bhalla 2002,37 Campbell 2003,131 Dhillon 2003,82 Dey 2004,66 Engers 2005,67 Lester 200939 | Emmerson 2003,40 Gough-Palmer 2009,84 Iversen 2000,151 Johnson 2008,139 Joyce 2000,147 Magill 2009,115 Matowe 2002,50 Potter 2007,46 Rosen 2006,128 Rowlands 2003,26 Simpson 2003,143 Van Dijk 2010,141 White 2000142 | iii |
| Attendance rate | Leiba 2002,130 Schillinger 2000,144 Wallace 2004,94 Whited 200287 | Harrington 2001,93 Khan 2008,71 Stainkey 2010,118 Whiting 2011153 | Eley 2010,83 Heaney 2001159 | Cox 2013,156 Ferris 2001,145 Ferris 2002,146 Kim 2004,155 King 2001,119 Pfeiffer 2011,138 Sved-Williams 2010,72 Vardy 2008150 | iii |
| Appropriateness of referral | Banait 2003,53 Bennett 2001,30 Donohoe 2000,31 Griffiths 2006,58 Junghams 2007,109 Walkowski 2007,63 Watson 2001,32 Wong 200081 | Akbari 2012,110 Chen 2010,100 Evans 2009,21 Hughes-Anderson 2002,136 Imkampe 2006,47 Kim-Hwang 2010,102 Knab 2001,112 Lucassen 2001,45 Watson 2002152 | Ellard 2012,38 Greiver 2005,114 Slade 2008117 | Hill 2000, Kennedy 2012,106 Melia 2008,51 Shariff 201028 | ii |
| Appropriate actioning of referral | Eccles 2001,54 Emery 2007,111 Robling 2002,60 Thomas 200379 | Malik 2007,41 Mariotti 2008,113 Van Dijk 2013149 | Julian 2007,62 Tierney 2003116 | Cusack 200543 | i |
| Adequate referral information provided | Jiwa 2004, Jiwa 2012,105 Kousgaard 2003,29 McGowan 2008,107 Gandhi 2008101 | Idiculla 2000,44 White 200461 | Jiwa 200668 | Ferriter 2006,157 West 200752 | i |
| Waiting time | Leggett 2004,85 Morrison 2001,64 Wong 200081 | Dennison 2006,99 Hemingway 2006,73 Tadros 2009,96 Watson 2002152 | McNally 200374 | i | |
| QALYs/cost | Damask 2008,76 Leiba 2002,130 McGowan 2008,107 Morrison 2001,64 Robling 2002,60 Salisbury 2005125 | Harrington 2001,93 Ridsdale 2008,124 Simpson 201078 | Tierney 2003116 | Rosen 2006,128 Spatafora 200569 | iii |
| Satisfaction/attitudes | Jaatinen 2002,95 Kousgaard 2003,29 Leiba 2002,130 McGowan 2008,107 Nicholson 2006,97 Salisbury 2005,125 Sanderson 2002,126 Schillinger 2000, 144 Wallace 2004,94 Whited 2002,87 Wong 200081 | Albertson 2002,158 Gurden 2012,133 Harrington 2001,93 Hilty 2006,24 Maddison 2004,154 Patterson 2004,104 Ridsdale 2008,124 Simpson 2010,78 Stoves 2010,103 Suris 2007,35 Tadros 2009,96 Wylie 200118 | Eminovic 2009,86 Tierney 2003116 | Emmerson 2003,40 Rosen 2006128 | i |
-
Referral outcomes (n = 62). These were outcomes that related to measuring the number of referrals which had been made. The specific outcomes reported by individual papers included referral, number of referrals/number of patients referred, change/differences in referral rates, referral to a particular specialty, referred back to GP with recommendation for referral to secondary-care services, achieving target referral levels and referrals avoided. These measures were usually used in a context in which a reduction was the target of the intervention. However, there were some instances where an intervention aimed to increase referrals (e.g. early diagnosis and referral).
-
Attendance rate/service use outcomes (n = 18). These outcomes related to use of specialist services as a result of referrals from primary care. Very often this outcome measure was reported in the absence of a more direct measure of referral. The specific outcomes reported by individual papers were described as service use; attendance rate; new visits to the clinic; number of patients requiring a clinic appointment; appointment cancellation; admission avoidance/readmission; non-attendance; and self-reported visits to specialist. This measure could be used in both a positive and a negative way, in that an increase in attendance could be the target outcome (decreasing non-attendance), or, conversely, a decrease in service use could be the anticipated effect.
-
Appropriateness of referral outcomes (n = 24). These outcomes relate to measuring the amount or proportion of referrals considered to be ‘appropriate’. Both adequacy of referral (suitable level of urgency and timing) and accuracy of referral (patients referred to the most suitable place) were considered. Most outcome measures were described simply as the amount or proportion of appropriate referrals, but others were also described as the number of inappropriate referrals, quality of referral, proportion of GPs making the correct referral decision, proportion correctly referred, and proportion of asymptomatic referrals. This outcome measure reportedly has some limitations in that consideration of appropriateness could vary between practitioners.
-
Referral quality outcomes (adequate referral information provided) (n = 10). These outcomes included measures of the quality of information provided in the referral. The outcomes were focused on whether or not the information provided by the GP to the specialist was adequate for the specialist’s needs. The outcomes were described in the individual studies as referral quality, referral letter quality, referral letter content, the quality of the referral information, relevant information in the referral, impact of the information provided and referral communication.
-
Appropriate actioning of referral (n = 10). These outcomes related to guidelines and measured compliance with, or adherence to, referral guidelines. Individual studies described outcome measures as compliance/concordance with guidelines, proportion of referrals meeting guidelines, adherence to care suggestions, number of requests for treatment/appointments, and GP/specialist agreement. As with the appropriateness of referral outcome (above), this measure has some limitations owing to variations in judgements of what is considered appropriate.
-
Waiting time (n = 8). These outcomes included all measures of time from the GP making the referral to some subsequent point in the diagnostic process. Most frequently, this was the time from the GP making the referral to the patient seeing the specialist for the first time. Individual papers described the outcomes as waiting time, time from presentation to referral appointment, waiting time from referral to appointment, time from referral to diagnosis, time to diagnosis and speed of referral.
-
Costs (n = 12). Although few papers focused specifically on the cost/cost-effectiveness of an intervention, 12 papers did report cost outcomes along with other measures. The cost-related outcomes reported included cost, cost to the NHS, cost of testing, health-care costs, cost-effectiveness, QALYs and cost saving.
-
Satisfaction/attitudes (n = 27). These outcomes looked at positive impacts on decision-making and patient satisfaction, and therefore include satisfaction of the patient, the referrer or both. Many studies included satisfaction outcomes as secondary measures. The specific outcomes reported were patient satisfaction, user satisfaction, satisfaction of patients and health professionals, practitioner satisfaction and GP attitude.
Non-intervention papers: immediate effects
The non-intervention papers consisted of qualitative studies and papers reporting associations. We scrutinised data from these papers and carried out additional searching to uncover any further evidence regarding the process whereby the different types of interventions we had identified may lead to change in referral outcomes. The key gaps in evidence from the intervention literature related to, firstly, the process whereby providing GP education interventions would change referral outcomes. We carried out further targeted searching to identify evidence here termed the ‘clinical reasoning search’. The second gap related to the process whereby interventions that change processes and systems would impact on referral behaviours and outcomes, and additional searches for this evidence were named the ‘systems search’. Full search strategies are provided in Appendix 4. Full extractions of these papers are to be found in Appendix 1.
Scrutiny of this literature identified two key sections of data, which had not been described in the intervention papers. Firstly, the literature described effects resulting from an intervention at a more immediate or micro level for individuals and, secondly, the papers described a range of predictors that may influence whether or not interventions which achieve effects in the short term lead to long-term change.
The outcomes described could be considered as measuring the ‘active ingredients’ in the interventions; these are the elements that would underpin the intended changed referral practice. These factors are, therefore, of importance in influencing if and how an intervention has an effect. The outcomes described in the literature were change in the doctor’s or patient’s knowledge, attitudes or beliefs, and change in the doctor–patient relationship (Table 24). As with the intervention and outcomes data, we assessed the strength of evidence underpinning these factors being associated with referral outcomes.
| Factor | Studies reporting association with referral outcomes (first author and year) | Studies reporting no association with referral outcomes (first author and year) | Strength |
|---|---|---|---|
| Increased GP knowledge | |||
| Additional training in condition/knowledge level or familiarity with a condition | More referral: Delva 2011,161 Fucito 2003162 | Jorgensen 2001,181 Lakha 2011,179 Montgomery 2006,180 Rowlands 2001,182 Rushton 2002,183 Wassenaar 2007178 | i |
| Less referral: Elhayany 2000,167 Freed 2003,172, Kvaerner 2007,168 Naccarella 2008,169 O’Neill 2005, Ringard 2010,164 Scheerers 2007,163 Swarzrauber 2002,171 Townsley 2003,165 Zielinski 2008166 | |||
| Direction unclear: Dodds 2004,174 Knight 2003,177 Lambert 2001,175 Pomeroy 2010,176 Tzaribachev 2009173 | |||
| Increased knowledge of services/systems | Angstman 2009,184 Coulston 2008,187 Kisely 2002,185 Mitchell 2012186 | ii | |
| Use/awareness/availability of referral guidelines | Blundell 2011,189 Clarke 2010,190 Kasje 2004,191 Ramanathan 2011188 | Abel 2011,20 Baker 2006,192 Bederman 2010,196 Belgamwar 2011,197 Jiwa 2008,193 Ruston 2004194 Tucker 2003,198 Watson 2001195 | iii |
| Use or awareness of specialist service quality indicators | Morsi 2012200 | iii | |
| Increased knowledge of patient responsiveness to treatment | Philichi 2010,202 Sigel 2004201 | ii | |
| Changed GP attitudes/beliefs | |||
| Confidence in management/perceived expertise | Anthony 2010,208 Bruynincksx 2009,209 Calnan 2007,207 Knight 2003,177 Moore 2000,205 Morgan 2007,210 Nandy 2001,204 Olson 2012,206 Rosemann 2005,211 Steele 2012,203 Van der Weijden 2002,212 Wilkes 2009213 | Ahluwalia 2009,214 Pryor 2001215 | i |
| Tolerance of uncertainty/risk | Abel 2011,20 Bruynincksx 2009,209 Calnan 2007,207 Cornford 2004,218 Espeland 2003,217 Franks 2000,216 Morgan 2007,210 Rosemann 2005,211 Rushton 2002,183 Van der Weijden 2002212 | Forrest 2006283 | i |
| Belief regarding peer opinion | Bruynincksx 2009,209 Green 2008,220 Van der Weijden 2002212 | i | |
| Perception of role | Abel 2011,20 Calnan 2007,207 Knight 2003,177 Nandy 2001,204 Young 2010221 | i | |
| Changed views of specialist service: familiarity with service/referral relationship including communication | Allareddy 2007,232 Barnett 2011,222 Beel 2008,226 Berendsen 2007,237 Chew-Graham 2008,229 Clemence 2003,228 Dagneaux 2012,230 Dale 2000,224 Delva 2011,161 Forrest 2002,223 Gandhi 2000,108 Grace 2008,292 Harland 2009,231 Jorgensen 2001,181 Kinchen 2004,238 Knight 2003,177 Massey 2004,236 McKenna 2005,225 Mitchell 2012,186 Morsi 2012,200 Pomeroy 2010,176 Ringard 2010,164 Samant 2007,234 Sigel 2004,201 Taggarshe 2006,233 Thorsen 2012,239 Wakefield 2012,227 Xu 2002235 | Ahluwalia 2009 214 | i |
| Changed GP referral behaviours | |||
| Optimal time of referral | Greer 2011240 | iii | |
| Optimal content of referral | Gandhi 2000,108 Jiwa 2009, Jiwa 2004, Jiwa 2012,105 Kousgaard 2003,29 McGowan 2008107 | Ferriter 2006,157 Harvey 2005242 | i |
| Pre-testing and ability to triage | O’Byrne 2010243 | iii | |
| Changed doctor–patient interaction | |||
| Doctor–patient relationship | Baker 2006,192 Berendsen 2007,237 Carlsen 2008,244 Forrest 2007,246 Hyman 2001,248 Johnson 2011,245 Knight 2003,177 Nandy 2001,204 Ramchandiani 2002,247 Rosemann 2005211 | i | |
| Shared decision-making | Carlsen 2008,244 Clarke 2010,190 Knight 2003,177 Nandy 2001204 | i | |
| Appropriate response to patient pressure | Calnan 2007,207 Little 2004,251 Morgan 2007,210 Rosen 2007,252 Stavrou 2009,249 Vulto 2009250 | i | |
| Changed patient attitudes/beliefs | |||
| Patient wishes/patient pressure | Albertson 2000,270 Anthony 2010,208 Bekkelund 2001,259 Berendsen 2007,237 Blundell 2010,253 Brien 2008,258 Dale 2000,224, Davies 2007,256 Edwards 2002,257 Espeland 2003,217 Forrest 2002,223 Glozier 2007,254 Gross 2000,261 Knight 2003,177 Lakha 2011,179 Lewis 2000,260 Little 2004251 Morgan 2007,210 Morsi 2012,200 Musila 2011,255 Philichi 2010,202 Pomeroy 2010,176 Stavrou 2009,249 Townsley 2003165 | i | |
| Appropriate service use: number of patient visits to GP/previous referral | Albertson 2000,270 Bertakis 2001,264 Cohen 2013,267 Dearman 2006,265 Harris 2011,268 Morgan 2007,210 Ridsdale 2007,266 Shadd 2011263 | Pfeiffer 2011,138 Vinker 2007269 | i |
General practitioner knowledge
The first immediate effect of an intervention described in the literature was a change in the referrer’s level or type of knowledge. Within this, a number of subfactors were categorised.
Additional training in the presenting condition
Additional training in the presenting condition (resulting in a higher knowledge level or familiarity with the patients’ symptoms) was reported in 23 studies (Table 25). 161–183 Of these, 17 studies reported a positive association between greater knowledge of the presenting condition and better referral outcomes (including only one study at higher risk of bias,165 the others being at lower risk of bias). 161–177 A further six studies (at lower risk of bias) reported no association. 178–183 The evidence for this association was graded as inconsistent.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and participant details where reported | Response |
|---|---|---|---|---|---|
| Delva 2011161 | Survey | France | Oncology | 436 GPs | NR |
| 75% male | |||||
| Dodds 2004174 | Survey | UK | Oncology | 331 GPs; 80% practices with four or more doctors | 65% |
| Elhayany 2000167 | Audit | Israel | All specialties | 44 GPs | NA |
| 67,577 patients | |||||
| Freed 2003172 | Survey | USA | Juvenile RA | NR | 49% |
| Jorgensen 2001181 | Survey | Denmark | Physiotherapy | 38,231 referrals | 90% |
| 410 GPs | |||||
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Kvaerner 2007168 | Survey | Norway | ENT | 1633 GPs | 48% |
| Lakha 2011179 | Survey | Canada | Pain clinic | 47 GPs | 32% |
| Lambert 2001175 | Survey | UK | Epilepsy | 312 GPs | 67% |
| Montgomery 2006180 | Interviews | UK | Nephrology | 51 GPs | 65% |
| Naccarella 2008169 | Survey | Australia | Mental health | 89 projects | 81% |
| O’Neill 2005170 | Survey | USA | All specialties | 2455 GPs | NR |
| Pomeroy 2010176 | Interviews/survey | Australia | Dietitian | 248 GPs (survey) | 30% |
| Ringard 2010164 | Survey | Norway | All specialties | 3493 GPs | 48–50% |
| Rowlands 2001182 | Video transcript | UK | All specialties | NR | NA |
| Rushton 2002183 | Survey | USA, Canada, Puerto Rico | Psychosocial services | 4012 patients | NR |
| Scheerers 2007163 | Survey | the Netherlands | Mental health | 301 GPs | NR |
| Swarzrauber 2002171 | Survey | USA | Neurology | 609 GPs | NR |
| 1116 specialists | |||||
| Townsley 2003 165 | Survey | Canada | Oncology | 2089 GPs | 24% |
| Tzaribachev 2009173 | Cohort | Germany | Paediatric rheumatology | 132 patients | NA |
| Wassenaar 2007178 | Survey | USA | Oncology | 672 GPs | 59.4% |
| Zielinski 2008166 | Audit | Lithuania | All specialties | 18 practice | NA |
Seventeen studies presented data suggesting an association between GP knowledge from training in the presenting condition and referral patterns. 161–177 Three studies suggested that GPs with training in a particular condition would refer more. 161–163 A study from France161 presented data which suggested that GPs’ attendance at a training course was associated with being more likely to refer for advanced cancer (OR = 1.85, 95% CI 1.01 to 3.38). Fucito et al. 162 reported that GPs who stated they regularly obtained information (training) about drug and alcohol use were more likely to refer patients for these problems (χ2 = 7.0, p < 0.01). Scheerers et al. 163 found that, in the Netherlands, GPs who received written training materials encouraging them to refer for chronic fatigue syndrome had higher referral rates.
However, nine studies suggested that GPs with training (increasing knowledge level or familiarity) in a particular condition would refer less. 164–172 The first164 reported that frequency of GPs attending formal meetings (training) and the GPs’ level of expertise were associated with lower referral rate in the Netherlands. A Canadian paper165 reported that GPs with extra training in geriatrics and those in practice longer were likely to refer regardless of tumour stage. Zielinski et al. 166 reported that being a specialist in family medicine, training and experience correlated with lower referral rates in Lithuania. A study in Israel167 found that GPs without any postgraduate training or specialty designation were likely to refer 2.5 times more often than primary paediatricians or family physicians. Kvaerner et al. 168 found that GPs in Norway who had received specialty training in general medicine made 6% fewer referrals than those who did not. Naccarella et al. 169 found that informing and training Australian GPs was the most popular demand management strategy to reduce referrals in a survey of project officers who had carried out demand management projects. The first of two US studies170 reported that GPs who were ‘board certified’ (trained) were associated with lower factor referral scores. The second171 found that GPs who preferred to manage patients without specialty involvement had higher knowledge scores than primary care physicians who preferred to refer to a specialist (p < 0.001). The final paper in this group172 reported a study on referral for juvenile rheumatoid arthritis and reported that 61% of GPs referred only to confirm diagnosis and guide initial therapy.
Another five studies suggested a link between training (or obtaining knowledge) and referral, but the direction of effect was unclear. Tzaribachev et al. 173 reported that a statistically significant predictor of delayed referral was the primary physician’s subspecialty training (p = 0.016). Dodds et al. 174 reported that GPs described that training for the 2-week wait guidance for cancer referrals created a rigid and inflexible system which did not offer scope for GP own judgement and experience. A UK study175 found that 64% of the GPs they surveyed would welcome teaching on epilepsy. Pomeroy and Cant176 reported that GP previous experience and knowledge of service were associated with referral. A second UK paper177 reported that GP expertise was one of 12 ‘doctor-related factors’ which could influence referral decisions.
There were a further six studies which reported that training in a particular condition (and the increased knowledge level, or familiarity with symptoms as a result of this) was not associated with referral. Wassenaar et al. 178 reported no difference in referral patterns related to those who had more or fewer patients with cancer (differing levels of familiarity with condition) in their US practice. Another North American study179 found that the more chronic pain patients a physician saw, the less he or she tended to refer them to pain clinics, but the relationship was not significant. A third UK paper in this group180 reported that referral rate did not differ by experience with renal patients. Jorgensen et al. 181 reported that the GP having frequent contact with a physiotherapist explained only a very small amount of referral variation, leaving the greatest majority of variation unexplained. Rowlands et al. 182 reported no alteration of practice referral rate following a UK education intervention. Rushton et al. carried out a survey across three countries183 and found that providing training in behaviour management did not change rate of referral for child psychosocial services.
Increased knowledge of services or systems for referral
Increased knowledge of services or systems for referral was reported in four studies (Table 26). 184–187 Although three of the four studies showed a positive association between increased knowledge of services or systems and better referral outcomes, three of the studies in the group were at higher risk of bias. 184–186 The evidence for this association was, therefore, graded as weaker.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Angstman 2009 184 | Survey | USA | Viral specialist | 56 GPs | NR |
| Coulston 2008187 | Survey | UK | Hernia surgery | 86 GPs | 72% |
| Kisely 2002 185 | Survey | Australia | Mental health | 74 GPs | 45% |
| Mitchell 2012 186 | Survey/interviews | Australia | Dietitian | 90 survey | 20–22% |
Angstman et al. 184 found that GPs reported that they often forgot that viral specialist consultations were an option, suggesting that increased knowledge would increase referral to the service. The first of two Australian papers185 reported that 80% of participants found the intervention duty officer useful as a point of first contact for the consultation-liaison service. The second186 reported that GP relationships with dieticians were believed to be the primary influencing factor on referral. The fourth study in this group187 found that only 17% of GPs were aware of any specialist consultant surgeons in South Wales performing laparoscopic groin hernia repair. Of those who were aware, 80% had at some time referred to this service.
Greater use or awareness of referral guidelines
Greater use or awareness of referral guidelines was reported in 12 studies (Table 27). 20,188–198 Of these, only four showed an association with better referral outcomes188–191 (one of which was at higher risk of bias190). A further eight studies showed no association between these factors (all at lower risk of bias). 20,192–198 The evidence for this association was, therefore, graded as conflicting.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where available | Response |
|---|---|---|---|---|---|
| Abel 201120 | Interviews | New Zealand | Colorectal cancer | 15 GPs | NA |
| 11 specialists | |||||
| Baker 2006192 | Interviews | UK | Lumbar spine X-ray | 29 GPs | NA |
| 24 male | |||||
| Bederman 2010196 | Delphi | Canada | Lumbar spine disease | 10 GPs/specialists | NA |
| Belgamwar 2011197 | Audit | UK | Anxiety/depression | Seven GPs | NA |
| 204 referrals | |||||
| Blundell 2011189 | Survey | UK | Elective surgery | 310 GPs | 41.6% |
| Clarke 2010 190 | Survey | UK | Elective surgery | 324 GPs | 40% |
| Jiwa 2008193 | Survey | UK | Lower bowel symptoms | 260 GPs | 52% |
| 50% male | |||||
| Aged 40 + years | |||||
| Kasje 2004191 | Survey | the Netherlands | All specialties | 197 GPs | GPs 75% |
| 34 general internists | Internists 50% | ||||
| Ramanathan 2011188 | Survey | Australia | Gynaecology/oncology | 140 GPs | 45.5% |
| Ruston 2004194 | Interviews | UK | All specialties | 85 GPs | NA |
| 49 male | |||||
| Tucker 2003198 | Interview | UK | Paediatric rheumatology | 171 GPs | 68% GP |
| Survey | 158 midwives | 77% midwives | |||
| Watson 2001195 | Survey | UK | Regional genetics service | 50 GPs | 94% |
Ramanathan et al. 188 reported greater variation in referral practice for endometrial cancer for which there are no Australian guidelines: 68% of vignettes with high probability of cancer were referred compared with 83% for ovarian cancer and 80% for cervical cancer for which guidelines are available. Blundell et al. 189 reported that most responding GPs indicated support for UK referral guidelines but 18% reported that they had never used them and < 3% reported use for most or all referral decisions. The odds of using guidelines decreased with increasing GP age, with a 10-year increase in age associated with halving odds of use (OR 0.53, 95% CI 0.29 to 0.90). Another UK study190 similarly found that although there was overall support from GPs for referral guidelines, these were rarely used in practice. Kasje et al. 191 reported that in the Netherlands most hospital specialists relied for their prescribing on international guidelines and agreements within their own department, whereas GPs relied more on national and regional guidelines.
Another UK study192 reported that both high and low referrers were aware of the X-ray guidelines for lumbar spine. Jiwa et al. 193 concluded that the application of guidelines by UK GPs is moderated by the influence of the characteristics of the patients only. Ruston et al194 echoed this lack of influence of UK guidelines, finding that none of responding GPs reported using referral guidelines as they considered them to be of theoretical rather than practical relevance. A Canadian study196 similarly found poor concordance of both predicted GP preferences and guideline recommendations with actual referral. Watson et al. 195 supported these limitations in their finding that, despite UK guidelines, many GPs did not know which patients warranted referral to a genetics service. Belgamwar et al. 197 reported that exactly half of all referrals (32/64) did not follow guidelines. Another study found that for paediatric rheumatology referrals intended management was most often referral or admission to a specialist hospital (59%, 132/224), both courses of action beyond guideline recommendations. 198 Abel and Thompson explored possible reasons underpinning this limited use of guidelines. 20 They reported that GPs perceived that rigid adherence to guidelines was inappropriate when working for the benefit of the patient.
Awareness of quality indicators
The use or awareness of quality indicators was reported in only one study200 at lower risk of bias (Table 28) in which publicly available quality measures were found to be ‘not at all important’ to referral decisions. The evidence from this study was, therefore, graded as no evidence of an association between awareness of quality indicators and referral outcomes.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where available | Response |
|---|---|---|---|---|---|
| Morsi 2012200 | Survey | USA | All specialties | 10 GPs | NR |
Knowledge of patient responsiveness to treatment
Increased knowledge of patient responsiveness to treatment/suitability for treatment was reported in two studies,201,202 one at higher risk of bias (Table 29). 202 The evidence for this association was graded as weaker.
The first study201 reported that referral decisions were made when GPs perceive that they have reached the limits of their capabilities for treating a problem, taking account of patient suitability for therapy and access to services. The second paper202 suggested that the most frequently identified reason for referral was patient unresponsiveness to treatment.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size | Response |
|---|---|---|---|---|---|
| Philichi 2010 202 | Survey | USA | Paediatric gastroenterology | 237 primary care | 38% |
| Paediatricians and nurse practitioners | |||||
| Sigel 2004201 | Interviews | UK | Psychological problems | 10 GPs | 40% |
| Seven male | |||||
| Age 38–60 years |
General practitioner attitudes and beliefs
The following elements were identified within the category of GP attitudes and beliefs which influenced referral decision-making.
Confidence in management of the patient
Increased confidence in management of the patient, or own perceived expertise, was reported in 14 studies (Table 30). 177,203–215 Of these, 12 showed a positive association between increased confidence and better referral outcomes203–213 (three of these were higher risk of bias203,205,208), and two showed no association between the factors214,215 (one of which was at higher risk of bias214). Therefore, the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size | Response |
|---|---|---|---|---|---|
| Ahluwalia 2009 214 | Survey | USA | Palliative care | 145 GPs | 85% |
| 58% female | |||||
| Anthony 2010 208 | Interviews/survey | USA | Depression care | 40 physicians, 15 GPs, 10 nurse practitioners | NA |
| 27 male | |||||
| Bruynincksx 2009209 | Survey | Belgium | All specialties | 163 GPs | NA |
| Calnan 2007207 | Interviews | UK | Immediate care | 15 GPs | NA |
| 10 male | |||||
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Eight male | |||||
| Moore 2000 205 | Survey | USA | Neurology | 504 GPs | NR |
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Nandy 2001204 | Interviews | UK | Mental health | 23 GPs | 67% |
| Olson 2012206 | Survey | Canada | Palliative radiotherapy | NR | 33% |
| Pryor 2001215 | Survey | Australia | Psychology | 105 GPs | 66% |
| 69% female | |||||
| Rosemann 2005211 | Survey | Germany | All specialties | 26 GPs | NR |
| Steele 2012 203 | Survey | Canada | Psychiatry | 847 GPs | 24.9% |
| Remote areas | |||||
| Male aged 41–60 years | |||||
| Van der Weijden 2002212 | Interviews | the Netherlands | Unexplained symptoms | 21 GPs | NA |
| Wilkes 2009213 | Interviews | UK | Infertility | 12 GPs, five specialists | NA |
| 13 patients |
Steele et al. 203 reported that lower GP confidence in managing mental health patients was associated with referral. Nandy et al. 204 associated lower referral rates with GPs having an interest in mental health and having confidence in dealing with mental health. Moore et al. 205 found that GPs who rated themselves as comfortable with seizure patients tended to refer fewer of these patients. Olson et al. 206 similarly found a strong relationship between family physician referral and self-assessed or tested knowledge and confidence (p < 0.001 and p < 0.010). One of five UK studies in this group207 reported that low referrers were more confident in their decisions and less often worried afterwards. Anthony et al. 208 reported that a clinician’s comfort in treating depression was identified by 80% as a very important factor for referral. Bruynincksx et al. 209 reported that whether or not the GP was uncertain of the diagnosis was associated with referral.
A second UK study177 reported that GPs needing advice affected referral. A further UK paper210 reported that GP clinical confidence in identifying risks of brain tumour affected referral. Rosemann et al. 211 reported that GPs’ experiences were more positive if their purpose was to reduce diagnostic uncertainty (p < 0.001) or if the purpose was to exclude serious illness (p < 0.010). Van der Weijden et al. 212 reported that GP uncertainty affected referral in the Netherlands. Wilkes et al. 213 found that UK GPs often reported a lack of skills or lack of confidence over infertility referrals. Ahluwalia et al. 214 found that having personal experience with palliative care was not statistically related to the likelihood of referral (OR 2.13, 95% CI 0.95 to 4.98). The final paper in this group215 reported that perception of professional competency was not a barrier to referral in Australia.
Tolerance of uncertainty and risk
Tolerance of uncertainty and risk in diagnosis and referral was reported in 11 studies (Table 31). 20,183,207,209–212,216–219 Of these, 10 reported a positive association between risk tolerance and better referral outcomes20,183,207,209–212,216–218 (with only one of these being at higher risk of bias20). The remaining study showed no association (and was at lower risk of bias). The evidence for this association was, therefore, graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details were reported | Response |
|---|---|---|---|---|---|
| Abel 2011 20 | Interviews | New Zealand | Colorectal cancer | 15 GPs | NA |
| 11 specialists | |||||
| Bruynincksx 2009209 | Survey | Belgium | All specialties | 163 GPs | NA |
| Calnan 2007207 | Interviews | UK | Immediate care | 15 GPs | NA |
| 10 male | |||||
| Cornford 2004218 | Interviews | UK | Breast cancer | 20 GP/other | NA |
| Surgeons, nurses | |||||
| Espeland 2003217 | Focus groups | Norway | All specialties | 14 GPs | NA |
| Forrest 2003219 | Audit | USA | All specialties | 139 GPs | NA |
| 14,709 visits | |||||
| Franks 2000216 | Survey | USA | All specialties | 173 GPs | 66% |
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Rosemann 2005211 | Survey | Germany | All specialties | 26 GPs | NR |
| Rushton 2002183 | Survey | USA, Canada, Puerto Rico | Psychosocial services | 4012 patients | NR |
| Children | |||||
| van der Weijden 2002212 | Interviews | the Netherlands | Unexplained symptoms | 21 GPs | NA |
Franks et al. 216 found that greater malpractice fear was associated with greater likelihood of referral in the USA. Bruynincksx et al. 209 reported that referral in Belgium was affected by GP uncertainty or anxiety. A UK paper207 reported that high referring GPs tended to express anxiety about the consequences of a decision. A paper reporting a survey across three countries183 described defensive GP referral strategies where there was risk to the woman of not referring when breast cancer was a serious disease and risk of the patient resorting to litigation if not referred and a problem was found later. Morgan et al. 210 found that in the UK referral was related to personal tolerance of uncertainty. This was echoed by a paper from the Netherlands,212 which also reported that GPs’ handling of uncertainty or error tolerance influenced referral. Rosemann et al. 211 reported that GPs’ experiences of referral were more positive if the GP’s purpose was to reduce diagnostic uncertainty (p < 0.001). Abel and Thompson20 found that GPs considered emotional or subjective concerns for the patient more relevant than subjective measures of risk. Espeland et al. ,217 similarly to the above studies, found that GP uncertainty influenced referral. Cornford et al. 218 reported that UK GPs varied in the extent to which they could accept the uncertainty about diagnosis. However, Forrest et al. ,219 in a US study, reported that anxiety as a result of to clinical uncertainty did not influence referral.
Peer opinion
An association between beliefs regarding peer opinion and referral was reported in three studies (Table 32). 209,212,220 All three showed a positive association between positive beliefs regarding peer opinion and referral outcomes (all were at lower risk of bias). The evidence for this association was, therefore, graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where available | Response |
|---|---|---|---|---|---|
| Bruynincksx 2009209 | Survey | Belgium | All specialties | 163 GPs | NA |
| 55% female | |||||
| Green 2008220 | Survey | UK | Eating disorders | 88 GPs | 33% |
| van der Weijden 2002212 | Interviews | the Netherlands | Unexplained symptoms | 21 GPs | NA |
A UK study220 found that intention to refer was significantly related to subjective norms (believing that a referral would be recommended by colleagues) and cognitive attitudes (r = 0.917 and 0.0896, p < 0.001). Bruynincksx et al. 209 found that GP referral was influenced by a perceived negative attitude towards the GP by specialists they had previously referred to. Van der Weijden et al. 212 also highlighted the influence of social norms on referral.
Role perception
The influence of the GP having a specific perception of their own role (perception of role as gatekeeper, responsibility for the patient, or referring for patient reassurance) was highlighted in five studies (Table 33), all of which reported an association between role perception and referral outcomes. As only one study was graded at higher risk of bias,20 the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Abel 2011 20 | Interviews | New Zealand | Colorectal cancer | 15 GPs | NA |
| 11 specialists | |||||
| Calnan 2007207 | Interviews | UK | Immediate care | 15 GPs, 10 male | NA |
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Eight male | |||||
| Nandy 2001204 | Interviews | UK | Mental health | 23 GPs | 67% |
| Young 2010221 | Interviews | Australia | All specialists | 10 GPs | NA |
One of three UK papers201 reported that some GPs saw their role as preventing burden on other agencies and thus tended not to refer, whereas others perceived that their role was diagnostic and patients were best managed by others (and thus tended to refer). A second UK study207 explored GP role perception and reported that low referrers saw hospitals as places to be avoided and that their role was to prevent admission. The other UK paper177 found that low referrers to mental health services might take more responsibility for patients and have more interest in treating psychological problems. Young et al. 221 found that processes of referral were influenced considerably by the degree to which GPs had taken on broader chronic care models rather than a more traditional care approach. Abel and Thomspon20 found that GPs perceived that referral and getting patients seen was part of their duty to do the best for the patient.
Views of a specialist service
The potential influence of a GP having specific views of a specialist service (as a result of increased familiarity with service or a better referral relationship, including communication with the specialist) was reported in 29 studies (Table 34). 108,161,164,176,177,181,186,200,201,214,222–239,292 Of these, 28 studies reported an association between a better GP view of a service and positive referral outcomes (one study reported no association). 214 Four studies were reported as having a higher risk of bias. 224,227,233 Despite this, the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Ahluwalia 2009 214 | Survey | USA | Palliative care | 145 GP | 85% |
| 58% female | |||||
| Allareddy 2007232 | Focus groups | USA | Chiropractic | NR | NA |
| Barnett 2011222 | Survey | USA | All specialties | 386 GPs | 63% |
| 64% male | |||||
| Beel 2008226 | Interviews | Australia | Psychology | 12 GPs | NA |
| Eight male | |||||
| Berendsen 2007237 | Interviews | the Netherlands | All specialists | 21 GPs | NA |
| Chew-Graham 2008229 | Interviews | UK | Mental health | GPs (no number) | NA |
| Clemence 2003228 | Interviews | UK | Musculoskeletal conditions | 22 GPs | NR |
| Dagneaux 2012230 | Focus groups | Belgium | Geriatricians | NR | NA |
| Dale 2000 224 | Survey | UK | Paediatric neurology | 50 GPs | NR |
| Delva 2011161 | Survey | France | Oncology | 436 GPs | NR |
| 75% male | |||||
| Forrest 2002223 | Survey | USA | All specialties | 141 GPs | NR |
| Gandhi 2000108 | Survey | USA | Orthopaedics, cardiology and gastro | 48 GPs | 53–56% |
| 400 specialists | |||||
| Grace 2008292 | Survey | Canada | Cardiology | 510 GPs/specialists | 36% |
| Harlan 2009231 | Survey | USA | Paediatrics | 10 paediatricians | NR |
| 12 GPs | |||||
| Jorgensen 2001181 | Survey | Denmark | Physiotherapy | 38,231 referrals | 90% |
| 410 GPs | |||||
| Kinchen 2004238 | Survey | USA | All specialists | 1252 GPs | 59.1% |
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Eight male, one female | |||||
| Massey 2004236 | Survey | UK | Physiotherapy | 50 GPs | 65% |
| McKenna 2005225 | Survey | USA | All specialties | 460 GPs | 46% |
| Mean age 48 years | |||||
| Mitchell 2012186 | Survey/interviews | Australia | Dietitian | 90 surveys | 20–22% |
| 52 interviews | |||||
| Morsi 2012200 | Survey | USA | All specialties | 10 GPs | NR |
| Pomeroy 2010176 | Interviews/survey | Australia | Dietitian | 248 GPs (survey) | 30% |
| 30 GPs interviewed: 14 male | |||||
| Ringard 2010164 | Survey | Norway | All specialties | 3483 GPs | 48–50% |
| Mean age 48 years | |||||
| Samant 2007234 | Survey | Canada | Radiotherapy | 400 GPs | 50% |
| Sigel 2004201 | Interviews | UK | Psychological problems | 10 GPs | 40% |
| Seven male | |||||
| Age 38–60 years | |||||
| Taggarshe 2006 233 | Focus group/survey | UK | All specialists | NR | 99% |
| Thorsen 2012239 | Focus groups | Norway | All specialists | 31 GPs | NA |
| 17 female | |||||
| Age 29–61 years | |||||
| Wakefield 2012 227 | Survey | Canada | Cardiology | 91 GPs | 19.9% |
| Xu 2002235 | Audit | USA | All specialties | 2572 GPs | NA |
| 79% male |
Barnett et al. 222 reported that GPs initiated referrals to 66% of their professional network colleagues. Delva et al. 161 associated referral with whether or not the GP was used to collaborating with the oncologist. Morsi et al. 200 reported that 70% of GPs said that familiarity with the hospital influenced referral. Ringard164 reported that referral was affected by having a formal arena for co-operation and exchange of information. Forrest et al. 223 reported that personal knowledge of the specialist was the most important reason for selecting a specific specialist. Jorgensen et al. 181 found that having frequent contact with a physiotherapist explained a small variation in referral rates (6.7% to 9.2%). Dale and Goodman224 reported that reasons for referral were having prior knowledge of the service and having previously referred to the service. McKenna225 found that GPs with greater understanding of the practice of the specialists were more likely to refer (p = 0.003). Sigel and Leiper201 found that referral decisions were influenced by professional interactions with psychologists. Knight177 found that previous experience with service influenced referral. Pomeroy and Cant176 found that GP knowledge of local services affected referral. Beel et al. 226 found that GP dissatisfaction with professional communications from psychologists affected referral. Wakefield et al. 227 reported that previous experience with a facility affected referral. Clemence et al. 228 found that GPs’ past experience of physiotherapy significantly affected referral. Chew-Graham et al. 229 found that lack of direct doctor-to-doctor communication was perceived to contribute to referral issues. Dagneaux et al. ,230 in areas with few geriatric services, found that doctors knew little of other professionals and reported suspicion and even conflicts. Harlan et al. 231 found that specialists and GPs acknowledge that significant barriers to optimal communication currently exist. Mitchell et al. 186 found that GPs’ relationships with dieticians were believed to be the primary influencing factor on referral by 81% of dieticians. Allareddy et al. 232 reported that GPs expressed a lack of understanding of chiropractic care and did not have any relationship with practitioners. Gandhi et al. 108 found that 28% of GPs and 43% of specialists were dissatisfied with information received from the other group. Grace et al. 292 found that GP lack of familiarity with cardiology site locations negatively impacted referral (p < 0.001). Taggarshe et al. 233 found almost four out of five GPs made referrals specifically to a named surgeon and valued personal rapport with the consultant. Samant et al. 234 reported that physicians who referred patients for radiotherapy were more likely to have sought advice from a radiation oncologist in the past. Xu et al. 235 found that the most significant determiner of perceived ability to refer was GP satisfaction in their communication with specialists. Massey et al. 236 found that those GPs not previously aware of a physiotherapy service would refer in the future. Berendsen et al. 237 found that ‘developing personal relationships’ and ‘gaining mutual respect’ dominated when the motivational factors for referral were considered. Kinchen et al. 238 found that previous experience with the specialist affected referral. Thorsen et al. 239 reported that GPs wished for improved dialogue with the hospital specialists. However, Ahluwalia et al. 214 found that having personal experience with palliative care was not statistically significantly associated with referral (OR 2.13, 95% CI 0.95 to 4.98).
General practitioner referral behaviour
A number of behaviour effects were reported following interventions which may be associated with changed referral pathways. Factors that were categorised as elements of GP referral behaviour associated with referral included the following subfactors.
Optimal timing of referral
One study was found that considered the potential significance of this aspect of the referral process. 240 This US paper reported that the enhanced use of optimal tests for kidney function by GPs could be associated with timely referral (Table 35). 240 The study was at low risk of bias and this evidence was graded as no evidence (evidence from only one study).
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Greer 2011240 | Survey | USA | Nephrology | 178 GPs and specialists | NR |
Optimal content of referral
The impact of optimal referral content was reported in eight studies (Table 36). 23,29,105,107,108,157,241,242 Of these, six showed an association between referral content and outcome23,29,104,107,108,241 (two studies reported no association156,242). All of the studies were at lower risk of bias and the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Ferriter 2006157 | BA | UK | Psychiatry | 20 referrals | NR |
| Gandhi 2000108 | Survey | NR | All specialties | 430 referrals | NR |
| Harvey 2005242 | Survey | UK | Psychiatry | 107 GPs | 94% |
| Jiwa 200423 | nRCT | UK | All specialties | 26 GPs | 100% |
| Jiwa 2009241 | Audit | UK | Gastroenterology | 207 referrals | NA |
| Jiwa 2012105 | BA | Australia | All specialties | NR | NR |
| Kousgaard 200329 | Survey | Denmark | Oncology | 199 GPs | 88.3% |
| McGowan 2008107 | RCT | Canada | All specialties | 82 GPs | 93.2% |
In the first of three papers by the same author, Jiwa et al. 241 reported that the cases that could be triaged from the letter were those where the letter contained more information (mean 66.38 vs. 49.86, mean difference 16, 95% CI 1.3 to 31.7; p < 0.001). The second paper23 reported that feedback improves the content of GP referral letters and may also impact on the type of patients referred for investigation by specialists. The third104 found that standardising and using electronic communications to refer facilitates the scheduling of specialist appointments. Kousgaard et al. 29 reported that better information provision before and after referral improved co-operation between the specialist department and the GP. McGowan et al. 107 found that providing timely information to clinical questions had a highly positive impact on decision-making and a high approval rating from participants. Gandhi et al. 108 echoed these other authors, highlighting that electronic referral can improve referral content and communication.
However, Harvey et al. ,242 in contrast, found no difference between higher- or lower-quality referral letters and referrals to psychiatric services. Ferriter et al. 156 suggested that the introduction of a single assessment process impaired clinical communication between GPs and psychiatrists.
Use of pre-referral testing
One paper243 reported that in 72% of cases an alteration to the diagnostic investigations thought to be necessary by GPs was required when the patient was seen by a specialist (Table 37). The paper highlighted the importance of accurate referral information in order to select tests prior to consultation. The study was at lower risk of bias and this evidence was graded as no evidence.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size | Response |
|---|---|---|---|---|---|
| O’Byrne 2010243 | Audit | UK | Respiratory consultants | 50 referrals | NA |
Doctor–patient interaction
Outcomes relating to changing the doctor–patient interaction and the association between this and referral practice were described in a large body of work. Elements of the doctor–patient interaction included the following subfactors.
Optimal relationship
Having a positive doctor–patient relationship (optimal relationship) was reported to be positively associated with referral outcomes in 10 studies (Table 38). 177,192,204,211,237,244–248 As only two were considered to be at higher risk of bias, the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Baker 2006192 | Interviews | UK | Lumbar spine X-ray | 29 GP | NA |
| 24 male | |||||
| Berendsen 2007237 | Interviews | the Netherlands | All specialists | 21 GPs | NA |
| Carlsen 2008 244 | Survey | Norway | All specialties | 41 GPs | 46% |
| 66% male | |||||
| Forrest 2007246 | Survey | USA | All specialties | 776 patients | NR |
| 133 GPs | |||||
| Hyman 2001248 | Survey | Canada | Mammography | 64 GPs | NA |
| 40% female, age range 29–71 (42.16) years | |||||
| Johnson 2011 245 | Interviews | Australia | Oncology | 40 GPs | NA |
| Mean age 47 (30–60) years | |||||
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Eight male, one female | |||||
| Nandy 2001204 | Interviews | UK | Mental health | 23 GPs | 67% |
| Ramchandiani 2002247 | Survey | UK | Ophthalmology | 50 GPs | 64% |
| 776 specialists | |||||
| 85 patients, 55 female. Mean age 75.5 years | |||||
| Rosemann 2005211 | Survey | Germany | All specialties | 26 GPs | NR |
Baker et al. 192 found a greater emphasis on the fragility of the doctor–patient relationship in higher referrers, and reported the use of referral for radiography as a method of attempting to preserve this relationship. Nandy et al. 204 reported that poor rapport with a patient was a reason for referral. Carlsen et al. 244 found that the more the doctor and patient differ in attitude towards patient involvement, the more often the GP refers to specialist care (p = 0.001). Knight177 also found that the quality of the doctor–patient relationship influenced referral decisions. Johnson et al. 245 similarly reported that communication and interpersonal issues affected referral. Forrest et al. 246 found that longer duration of the doctor–patient relationship was a positive predictor of referral completion. Rosemann et al. 211 found that experiences with the referral were more positive if the initiative for the referral came from the physician (beta = 0.365, p < 0.001). Ramchandiani et al. 247 reported that pooled lists were unpopular as they devalued the doctor–patient relationship. Berendsen et al. 237 reported that 81% of patients thought it was important that the GP gave them advice on which hospital or specialist to go to. Hyman et al. 248 found that physicians who spent more time on patient education were more likely to refer.
Shared decision-making
Shared decision-making between the GP and the patient was reported to be positively associated with referral outcome in four studies (Table 39). 177,190,204,244 As only two were at higher risk of bias,190,244 the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Carlsen 2008 244 | Survey | Norway | All specialties | 41 GPs | 46% |
| 66% male | |||||
| Clarke 2010 190 | Survey | UK | Elective surgery | 324 GPs | 40% |
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Eight male, one female | |||||
| Nandy 2001204 | Interviews | UK | Mental health | 23 GPs | 67% |
Clarke et al. 190 found that the view that patients should be involved in referral decision-making was strongly supported by UK GPs. Another study from the UK177 found that patient wishes and preferences influenced referral decisions. Carlsen et al. 244 reported a significant negative correlation between GP score and referral rate (–0.46, p = 0.002), indicating that GPs with a preference for patient involvement in Norway are less likely to refer. Nandy et al. , in a third UK study in this group,204 reported that the patient desire to be referred was important.
Appropriate response to patient pressure
Response to patient pressure was reported to be associated with referral outcomes in six studies (Table 40). 207,210,249,250,251,252 All studies were at lower risk of bias and the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample/treatment | Response |
|---|---|---|---|---|---|
| Calnan 2007207 | Interviews | UK | Immediate care | 15 GPs | NA |
| 10 male | |||||
| Little 2004251 | Survey | UK | Depression | 30 GPs | NA |
| 847 patients aged 16–80 years | |||||
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Rosen 2007252 | Interviews, focus groups | UK | All referrals | GPs (no number) | NA |
| Stavrou 2009249 | Interviews | UK | Mental health | 14 GPs | 47% |
| Seven male. Mean age 39 years | |||||
| Vulto 2009250 | Survey | the Netherlands | Palliative radiotherapy | 489 GPs | 45.5% |
| 65% male |
Calnan et al. 207 found that low referrers described themselves as more able to resist pressure from family or carers. Stavrou et al. 249 found that no GP refused if a patient asked to be referred. The one non-UK study in this group250 found that most GPs in the Netherlands reported that they reacted to the wishes of the patient regarding referral. Little et al. 251 found that doctor’s perception of moderate or definite patient pressure was a predictor of referral behaviour: perceived slight patient pressure to be referred – 19% referred, 5% not referred (OR 8.99, 95% CI 4.91 to 16.46; p = 0.994); perceived moderate or definite pressure – 44% referred, 1% not referred (OR 125.3, 95% CI 51.3 to 306.5; p = 0.005). Morgan et al. 210 reported that GPs showed variations in an individual’s willingness or ‘resistance’ to refer, reflecting differences in clinical confidence and views of patients’ ‘right’ to referral. Rosen et al. 252 reported that most GPs make choices on the patient’s behalf (with or without Choose and Book) unless the patient expresses a preference.
Patient factors
Although we found a large body of evidence regarding the potential influence of doctor–patient interaction on referral, we found no studies that reported patient knowledge outcomes and associated these with referral outcomes. Although patient knowledge outcomes were not reported, literature describing an association between patient attitude/belief elements and referral were found. Factors that were categorised as relating to patient attitude or beliefs included two subfactors: patient pressure and service use.
Patient pressure
The association between strong patient wishes (or the amount of patient pressure imposed on the GP) was reported as being associated with referral outcomes in 24 studies (Table 41),165,176,177,179,200,202,208,210,217,223,224,237,249,251,253–261,270 of which five were at higher risk of bias. 165,202,208,224,255 Therefore, the evidence for this association was graded as stronger.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Albertson 2000270 | Survey | USA | All specialists | 12 GPs | NR |
| 822 patients | |||||
| Anthony 2010 208 | Survey | USA | Depression | 40 physicians; 15 general internists, 15 GPs, 10 nurse practitioners; 27 female, 13 male | NR |
| Bekkelund 2001259 | Survey | Norway | Neurology | 105 patients | 75% |
| Berendsen 2007237 | Interviews | the Netherlands | All specialists | 21 GPs | NA |
| Blundell 2010253 | Interviews | UK | Surgical | 22 GPs | 96% |
| Brien 2008258 | Interviews | UK | CAM | 10 GPs | 30% |
| Dale 2000 224 | Survey | UK | Paediatric neurology | 50 GPs | NR |
| Davies 2007256 | Audit | UK | Endoscopy | 33 referrals | NA |
| Edwards 2002257 | Focus group | UK | All specialists | 86 GPs/nurses | 51–90% |
| Espeland 2003217 | Focus groups | Norway | Radiography | 14 GPs | NA |
| Forrest 2002223 | Cohort | USA | All specialists | 141 GPs | NA |
| Glozier 2007254 | Cohort | UK | Orthopaedics | 188 referrals | NA |
| Gross 2000261 | Survey | Israel | All specialists | 1084 patients | 81% |
| Knight 2003177 | Interviews | UK | Mental health | Nine GPs | NA |
| Two practices | |||||
| Lakha 2011179 | Survey | Canada | Pain clinic | 47 GPs | 32% |
| Lewis 2000260 | Interviews | USA | All specialists | 314 patients | NR |
| Little 2004251 | Survey | UK | Depression | 30 GPs | NA |
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Morsi 2012200 | Survey | USA | All specialties | 10 GPs | NR |
| Musila 2011 255 | Referral audit | UK | Chronic knee pain | 12 members including patients, GPs, orthopaedic surgeons and other health-care professionals | NA |
| Philichi 2010 202 | Survey | USA | Paediatric gastro | 237 GPs | 38% |
| Pomeroy 2010176 | Interviews/survey | Australia | Dietitian | 248 GPs (survey) | 30% |
| Stavrou 2009249 | Interviews | UK | Mental health | 14 GPs | 47% |
| Townsley 2003 165 | Survey | Canada | Oncology | 2089 GPs | 24% |
Blundell et al. 189 reported that the extent of patient involvement in the referral decision affected referral. Morsi et al. 200 found that patient preference was considered important in referral decisions by 62% of respondents. Forrest et al. 223 reported that patient request was the reason for 13.6% of referrals. Townsley et al. 165 found that a patient’s desire to be referred influenced GPs’ decision to refer. Stavrou et al. 249 found that referral was influenced by patient request and interest in referral; no GP refused if a patient asked to be referred. Dale and Goodsman224 reported that 78% of GPs, in making a referral, were responding to parental concerns. Little et al. 251 found that patient pressure affected referral; where patient wish to be referred was slight, 16% were referred and 8% were not referred (OR 3.34, CI 1.88 to 5.93; p = 0.796), and where patient pressure was moderate or definite, 28% were referred and 5% were not referred (OR 8.51, CI 4.97 to 14.6; p = 0.028). Glozier et al. 254 found that greater personal control (patient) was associated with referral; assertive patients better able to influence and control their lives were more successful at obtaining an urgent referral. Anthony et al. 208 found that patient preference and resources (willingness to see a mental health specialist, and ability to pay) affected referral. Knight177 found that patient wishes and preferences influenced referral decisions. Lakha et al. 179 reported patient preference for other treatments influenced referral decisions. Philichi and Yuwono202 described that the second most frequently identified reason for referral to paediatric gastroenterology was parents wanting a second opinion (15%). Pomeroy and Cant176 found that patient choice of treatment and willingness to attend affected referral. Musila et al. 255 found that ratings of referral appropriateness were strongly influenced by patients’ referral preferences. Morgan et al. 210 found that readiness to refer in response to pressure was influenced by characteristics of the consultation, including frequent attendance, communication problems and time constraints. Davies et al. 256 found that patients also identified problems with communication, information and support about diagnosis when being referred for endoscopy. Edwards et al. 257 reported the importance of patient psychosocial factors in referral. Brien et al. 258 found that a match between the doctor’s attitude and treatment preferences and patient views was important. Espeland et al. 217 found that patient wishes for radiography and the GP’s response affected referral. Albertson et al. 270 found that continuity of care and familiarity with their GP are associated with patients initiating a referral discussion with their GP.
There were a few international studies where applicability in the UK was questionable: Bekkelund et al. 259 found less Norwegian patient satisfaction (52% dissatisfied) with self-referral than with doctor referral (42% dissatisfied). Lewis et al. 260 found that, in the USA, patients valued the freedom to choose their doctor and have unencumbered access to specialists. Gross et al. 261 reported that one-third of Israeli respondents preferred self-referral to a specialist. Forty per cent preferred their family physician to act as a gatekeeper, and 19% preferred the physician to co-ordinate care but to refer themselves to a specialist. Berendsen et al. 262 reported that 81% of patients in Norway thought that it was important that the GP gave them advice on which hospital or specialist to go to when they referred the patient.
Service use
Appropriate patient behaviour in terms of appropriate service use (number of patient visits to GP and previous referral) was reported as being associated with referral outcomes in eight studies, all at lower risk of bias (Table 42). 210,263–268,270 A further two lower-risk studies showed no association between these factors. 138,269 Overall, the strength of evidence was graded as weaker.
| Study (first author and year) | Design | Country | Specialty/treatment | Sample size and details where reported | Response |
|---|---|---|---|---|---|
| Albertson 2000270 | Survey | USA | All specialists | 12 GPs | NR |
| 822 patients | |||||
| Bertakis 2001264 | Survey | USA | All specialties | 509 patients | NR |
| 26 GPs | |||||
| 79 specialists | |||||
| 38% male patients | |||||
| Cohen 2013267 | Audit | USA | Otolaryngology | 149,653 patients | NA |
| Dearman 2006265 | Audit | UK | Psychiatry | 1089 patients | NA |
| Elderly | |||||
| Harris 2011268 | Survey | Australia | Heart disease/hypertension | 26 practices | NR |
| Patient mean age 61.6 (19 to 90) years; 55% female | |||||
| Morgan 2007210 | Interviews | UK | Headache | 20 GPs | 50% |
| Pfeiffer 2011138 | Audit | USA | Mental health | 49,957 patients | NA |
| Mean age 55.7 years | |||||
| 93% male | |||||
| Ridsdale 2007266 | Cohort | UK | Neurology | 488 patients | NA |
| Shadd 2011263 | Audit | Canada | All specialties | 33,998 patients, 10 GPs | NA |
| Vinker 2007269 | Survey | Israel | Ophthalmology, orthopaedics, ENT, dermatology | 257 referrals | NR |
Shadd et al. 263 found that 92% of the variance in referral rates was attributable to the patient (rather than to the practice). Bertakis et al. 264 reported that after controlling for physical status, patient sex and age, more visits to the GP was associated with more specialist referrals. Dearman et al. 265 reported that patients referred to psychiatry had consulted their GP more frequently in the past year. Morgan et al. 210 found that GP readiness to refer for headache was influenced by frequent attendance. Ridsdale et al. 266 found that, for migraine patients, referred patients consulted more frequently than those not referred in the 3 months before referral (p = 0.003). Albertson et al. 157 found that patients were significantly more likely to have initiated the referral discussion when they had seen the GP previously; there was a trend for patient initiation of the referral discussion when the patient had known the GP for more than 1 year (p = 0.08) Cohen et al. 267 reported that a greater number of GP visits was related to a lower hazard ratio for referral and more days to referral. Harris et al. 268 found that previous referral was associated with the likelihood of subsequent referral.
However, in contrast to these papers reporting an association, Pfeiffer et al. 138 found that attendance at a primary care service for mental health was not a predictor of total number of specialist mental health clinic visits; and Vinker et al. 269 found that the length of time the patient was with the GP did not affect referral.
Non-intervention papers: predictors of changed practice
The second group of factors described in the non-intervention literature were elements which may moderate or mediate the outcomes described above, and act as predictors of whether of not an intervention will lead to long-term change in referral practice. Here, we examined evidence regarding the potential barriers or facilitators to the interventions changing practice at a local level and/or a health-care system level.
Moderating and mediating factors described in the literature related to the GP, the patient or the service in which the referral was taking place (Table 43). The complexity of the evidence here is further increased by many of the identified factors as operating in both directions, for example older age increases referral or older age decreases referral.
| Factor | Studies reporting association (first author and year) | Studies reporting no association (first author and year) | Strength |
|---|---|---|---|
| GP factors | |||
| Years in practice | Longer = fewer: Calnan 2007,207 Fucito 2003,162 Townsley 2003165 | Albertson 2000,270 Delva 2011,161 Johnson 2008,271 Jorgensen 2001,181 Lakha 2011,179 Vulto 2009,250 Wakefield 2012227 | iii |
| Longer = more: Elhayany 2000,167 Franks 2000,216 Ramanathan 2011,188 Harvey 2005242 | |||
| Age | Younger = more: Hugo 2000,272 Jiwa 2008,193 Pryor 2001,215 Balduf 2008274 | Albertson 2000,270 Bolanos-Carmona 2002,276 Delva 2011,161 Elhayany 2000,167 Johnson 2008,271 Jorgensen 2001,181 Lakha 2011,179 Pomeroy 2010,176 Ringard 2010,164 Rushton 2002,183 Wakefield 2012,227 Wassenaar 2007178 | iii |
| Older = more: Bowling 2006,273 Chan 2003,275 Franks 2000,216 Fucito 2003,162 O’Neill 2005,170 Swarzrauber 2002171 | |||
| Ethnicity | Ache 2011,277 Kinchen 2004,238 Navaneethan 2010278 | Lakha 2011179 | iii |
| UK-qualified | Hugo 2000272 (more), O’Neill 2005170 (fewer) | iii | |
| Sex | Females refer more: Bowling 2006,273 Calnan 2007,207 Chan 2003,275 Cooper 2001,279 Coyle 2011,280 Feeney 2007,282 Franks 2000,216 Gruen 2002,281 Hugo 2000,272 Jorgensen 2001,181 McKenna 2005225 | Albertson 2000,270 Bolanos-Carmona 2002,276 Delva 2011,161 Elhayany 2000,167 Forrest 2006,283 Johnson 2008,271 Lakha 2011,179 Montgomery 2006,180 Ringard 2010,164 Rushton 2002,183 Wakefield 2012,227 Wassenaar 2007178 | iii |
| Males perceive barriers: Hyman 2001248 | |||
| Previous experience/familiarity with service | Ahluwalia 2009,214 Allareddy 2007,232 Balduf 2008,274 Barnett 2011,222 Beel 2008,226 Berendsen 2007,237 Brien 2008,258 Chew-Graham 2008,229 Clemence 2003,228 Cornford 2004,218 Dagneaux 2012,230 Dale 2000,224 Delva 2011,161 Dodds 2004,174 Forrest 2002,223 Gandhi 2000,108 Grace 2008,292 Harlan 2009,231 Holley 2010,293 Jorgensen 2001,181 Kier 2012,294 Kinchen 2004,238 Knight 2003,177 Massey 2004,236 McKenna 2005,225 Mitchell 2012,186 Morsi 2012,200 Pomeroy 2010,176 Ringard 2010,164 Samant 2007,234 Sigel 2004,201 Taggarshe 2006,233 Thorsen 2012,239 Wakefield 2012,227 Watson 2001,195 Xu 2002235 | Chan 2003,275 Harris 2011268 | i |
| Satisfaction with specialist service | Beel 2008,226 Johnson 2011,289 Knight 2003,177 Nandy 2001,204 Pryor 2001,215 Ringard 2010,164 Sigel 2004201 | Guevara 2009290 | i |
| Emotional response | Bowling 2000,291 Espeland 2003,217 Nandy 2001204 | i | |
| Ability to judge own referral | Baker 2006192 | iii | |
| Patient factors | |||
| Ethnicity | Chen 2005,286 Greer 2011,240 Navaneethan 2010,278 Chauhan 2012284 | Johnson 2011289 | ii |
| Age | Older referred more: Bertakis 2001,264 Chan 2003,275 Chauhan 2012,284 Cohen 2013,267 Forrest 2006,283 Gruen 2002,281 Harris 2011,268 Jorgensen 2001,181 Sullivan 2005,285 Ramanathan 2011,188 Ringard 2010,164 Shadd 2011,263 Van der Weijden 2002,212 Zielinski 2008166 | Bruynincksx 2009,209 Delva 2011,161 Glozier 2007,254 Johnson 2008,271 Montgomery 2006,180 Pomeroy 2010,176 Townsley 2003,165 Vulto 2009250 | iii |
| Older referred less: Chen 2005,286 McBride 2010,287 Navaneethan 2010,278 Robinson 2010,288 Samant 2007,234 Todman 2011295 | |||
| Children more: Chan 2003275 | |||
| Urgent referral younger: Vinker 2007269 | |||
| Sex | Females referred more: Bertakis 2001,264 Chauhan 2012,284 Jorgensen 2001,181 Sullivan 2005,285 Shadd 2011,263 Zielinski 2008166 | Vinker 2007269 | iii |
| Males referred more: Bruynincksx 2009,209 Chen 2005,286 Cohen 2013,267 Forrest 2006,283 Gruen 2002,281 McBride 2010,287 Navaneethan 2010278 | |||
| Level of education | Berendsen 2010,262 Ringard 2010164 | Johnson 2008 271 | iii |
| General patient-related social/clinical factors | General: Bolanos-Carmona 2002,276 Delva 2011,161 Forrest 2006,283 Harris 2011,268 Johnson 2011,245 Knight 2003,177 Rushton 2002,183 Shadd 2011,263 Vulto 2009,250 Wakefield 2012227 | Glozier 2007254 | i |
| Socioeconomic: Baker 2006,192 Soomro 2000,296 McBride 2010,287 Mulvaney 2005,297 Soerensen 2009,298 Van der Weijden 2002212 | |||
| Clinical specialty/condition | Anthony 2010,208 Bertakis 2001,264 Chan 2003,275 Chen 2005,286 Harris 2011,268 Johnson 2011,289 Johnson 2011,245 Knight 2003,177 Little 2004,251 Musila 2011,255 Sullivan 2005,285 Rushton 2002,183 Shadd 2011263 | Calnan 2007207 | i |
| Comorbidity/complexity of condition | Anthony 2010,208 Bertakis 2001,264 Cohen 2013,267 Dearman 2006,265 Forrest 2006,283 Gruen 2002,281 Harris 2011,268 McBride 2010,287 Navaneethan 2010,278 Pomeroy 2010,176 Ridsdale 2007266 Rushton 2002,183 Zielinski 2008166 | Glozier 2007254 | i |
| Responsiveness to treatment/suitability/likely benefit | Anthony 2010,208 Baker 2006,192 Blundell 2010,189 Green 2008,220 Johnson 2011,245 Knight 2003,177 Nandy 2001,204 Philichi 2010,202 Pomeroy 2010,176 Samant 2007,234 Sigel 2004,201 Stavrou 2009,249 Steele 2012203 | Ahluwalia 2009 214 | i |
| Self-reported health | Harris 2011268 | iii | |
| Service and organisational factors | |||
| Practice location | Greater distance to specialist: Jorgensen 2001,181 Swarzrauber 2002,171 Tzaribachev 2009173 | Delva 2011,161 Gruen 2002,281 Johnson 2011,289 Love 2005,302 Pryor 2001,215 Rushton 2002183 | iii |
| Local more: Franz 2010,307 Hugo 2000,272 Johnson 2011,289 Jorgensen 2001,181 Lakha 2011,179 Todman 2011,295 Wakefield 2012227 | |||
| More deprived location: Chan 2003,275 Rosen 2007252 | |||
| Rural more: Shadd 2011,263 Tucker 2003198 | |||
| Rural less: Jiwa 2008,193 Ramanathan 2011,188 Townsley 2003,165 Zielinski 2008166 | |||
| Size of practice | Large practice more: Chauhan 2012,284 Forrest 2006,283 Harris 2011,268 Navaneethan 2010,278 Trude 2003199 | Ashworth 2002,303 Johnson 2008,271 Johnson 2011,245 Jorgensen 2001,181 Montgomery 2006,180 Rushton 2002,183 Xu 2002235 | iii |
| Single GP more: O’Neill 2005170 | |||
| Other practice characteristics | Managed care higher: Forrest 2006,283 Navaneethan 2010,278 Sullivan 2005,285 Walders 2003300 | Ownership, managed care: Burns 2000,301 Shadd 2011263 | iii |
| Private higher: Hugo 2000,272 Zielinski 2008166 | Fundholding: Ashworth 2002,303 Soomro 2000296 | ||
| Admin resources higher: Boulware 2006,304 Walders 2003300 | Having onsite service: Greenaway 2006306 | ||
| Assistants/nurses: Chung 2010305 | |||
| Financial arrangements in smaller practices only: Xu 2002235 | |||
| Care group/role in practice: Bolanos-Carmona 2002276 | |||
| Gatekeeping: Forest 2003 | |||
| Physician burden/time pressure | Anthony 2010,208 Franz 2010,307 Guevara 2009,290 Kim 2009,98 Knight 2003,177 Kvaerner 2007,168 Morgan 2007,210 Nandy 2001,204 Philichi 2010,202 Trude 2003,199 Van der Weijden 2002212 | Albertson 2000,270 Bolanos-Carmona 2002,276 Hyman 2001248 | i |
| Waiting time | Barnett 2011,222 Bowling 2006,273 Knight 2003,177 Lakha 2011,179 Ramchandiani 2002,247 Ringard 2010,164 Samant 2007,234 Stavrou 2009,249 Steele 2012,203 Taggarshe 2006,233 Todman 2011295 | i | |
| Availability of specialist | Alexander 2008,308 Anthony 2010,208 Franz 2010,307 Guevara 2009,290 Holley 2010,293 Johnson 2011,289 Johnson 2011,245 Kvaerner 2007,168, Morgan 2007,210 Ramanathan 2011,188 Trude 2003,199 Wakefield 2012227 | Forrest 2006,283 Malcolm 2008299 | i |
General practitioner factors
Moderating factors which were categorised as GP factors include the following subfactors.
The number of years a GP had been in practice was reported in 14 studies. 161,162,165,167,179,181,188,207,216,242,250,270,271,277 Of these, three studies reported that a GP who had been in practice longer referred less frequently. 162,165,207 Conversely, three studies reported that GPs who had been in practice longer referred more frequently. 167,188,216 One further study reported better-quality referral letters in those GP who had recently qualified. 242 An additional seven studies reported no association between number of years in practice and referral rate. 161,179,181,227,250,270,271 Overall, three of these studies162,227,271 were at higher risk of bias and the evidence was graded as conflicting.
General practitioner age as a factor associated with referral outcomes was reported in 21 studies. 161,162,164,167,169,171,176,179,181,183,193,215,216,227,270–276 Again, the picture was very mixed, with four studies reporting higher rates of referral for younger GPs193,215,272,274 and six studies reporting higher rates of referral for older GPs. 162,169,171,216,273,275 Thirteen studies reported no association. 161,164,167,176,178,179,181,183,227,270,271,272,276 Three of these studies162,227,271 were at higher risk of bias and the evidence was graded as conflicting.
The ethnicity of the referring GP, or the country of their medical training, was reported by four studies. 179,238,277,278 Of these, three showed an association with referral outcomes. 238,277,278 The fourth study showed no association between ethnicity/country of training and referral. 179 One study was considered to be at higher risk of bias277 and the evidence overall was graded as conflicting.
The sex of the GP was reported by 24 studies. 161,164,167,178–181,183,207,216,225,227,248,270–273,275,276,279–283 Eleven studies suggested that females refer more frequently,181,207,216,225,272,273,275,279–282 with one further study248 discussing perceived male barriers to referral. However, 12 studies reported no association between sex and referral outcomes. 161,164,167,178–180,183,227,270,271,276,283 Two were at higher risk of bias227,271 and overall the evidence was graded as conflicting.
Previous experience or familiarity with the service referring to was reported as a factor associated with increased likelihood of referral outcomes in 38 studies. 108,161,164,174,176,177,181,186,195,200,201,214,218,222–239,258,268,274,275,292–294 Of these, 36 studies reported that previous experience of familiarity with a service was associated with an increase likelihood of referral. 108,161,164,174,176,177,181,186,195,200,201,214,218,222–239,258,274,292–294 Six of these studies186,214,224,227,233,236 were considered at higher risk of bias. Only two studies reported no association268,275 and so this evidence was graded as stronger.
Previous satisfaction with specialists, reported in eight studies (all at lower risk of bias), was shown to be associated with increased likelihood of referral in all but one. 164,177,201,204,215,226,289 The final study showed no association. 290 This evidence was, therefore, also graded as stronger.
The GP’s emotional response to the patient was reported to be associated with referral in three studies of lower risk of bias. 204,217,291 A GP who had greater awareness of their own referral rate and who was able to judge their referral level as lower or higher was reported to be associated with likelihood of referral in one study. 192 As only one study reported this outcome, the evidence for this factor was graded as no evidence.
Patient factors
Moderating factors which were categorised as patient factors include the following subfactors.
Patient age as a factor associated with referral outcomes was reported in 30 studies. Twenty-five studies showed an association between age and referral rate, of which 14 studies reported higher rates of referral for older patients164,166,181,188,212,263,264,267,268,275,281,283–285 and six studies reporting higher rates of referral for younger patients. 234,278,286–288,295 Chan et al. 275 also reported that children were referred more often than adults and Vinker et al. 269 reported more urgent referral for younger patients. A further eight studies reported no association between age and referral. 161,165,176,180,209,250,254,271 Three of these studies165,271,295 were at higher risk of bias and the evidence was graded as conflicting.
An association between the ethnicity of a patient and referral was considered by five studies. 240,278,284,286,289 Of these, four showed an association with referral outcomes. 240,278,284,286 Three studies showed lower referral rates for non-white patients,278,284,286 with one further study reporting improvement in timing of referrals for white patients compared with African Americans. 240 Johnson et al. 289 reported no association between ethnicity and referral. The studies were all considered to be at lower risk of bias and the evidence overall was graded as weaker.
The sex of the patient was reported to be associated with referral rate in 15 studies. 166,181,209,263,264,267,278,281,283–287 Six studies suggested that females were referred more frequently,166,181,263,264,284,285 and seven further studies reported that males were referred more frequently. 209,267,278,281,283,286,287 One further study suggested no association between patient sex and referral. 269 All studies were at lower risk of bias and overall the evidence was graded as conflicting.
Patient level of education was reported in three studies. 164,262,271 Two studies reported an association between being more educated and being more likely to be referred. 164,262 and one study reported no association. 271 The third of these studies271 was at higher risk of bias and the evidence was graded as conflicting.
A further 17 studies reported an association with referral outcomes (including one at higher risk of bias). General patient characteristics were reported to be associated with referral in 11 studies. 161,177,181,183,227,245,250,263,268,276,283 Socioeconomic characteristics of the patient were reported to be associated with referral decisions in a further six studies,192,212,287,296–298 with lower deprivation leading to more referral. One further study reported no association between sociodemographic characteristics and urgent referral requests. 254 Overall, for this association, the evidence was graded as stronger.
Fourteen studies considered whether the clinical specialty being referred to, or the particular condition which the patient presented with, were associated with referral outcomes. 177,183,207,208,245,251,255,263,264,268,275,285,286,289 Thirteen studies reported that referral was moderated by clinical specialty177,183,208,245,251,255,263,264,268,275,285,286,289 and only one did not. 207 In this group only one study was at higher risk of bias,208 and the evidence was graded as stronger.
The presence of comorbidity or the complexity of the presenting condition was further reported as being associated with referral outcomes in 14 studies. Thirteen studies reported that referral was moderated by the complexity of the clinical presentation166,176,183,208,264–268,278,281,283,287 and only one did not. 254 This evidence was graded as stronger.
Related to this, patient responsiveness to treatment, suitability for treatment or likely benefit of referral (perceived by the referring doctor) was reported in 14 studies176,177,182,189,201–204,208,214,220,234,245,249 (four at higher risk of bias202,203,208,214), and suggested as being associated with referral outcomes in 13 studies. 176,177,182,189,201–204,208,220,234,245,249 Only one study (at higher risk of bias) showed no association. 214 The evidence for this association was graded as stronger.
Patient self-reported health was reported in one (lower risk of bias) study. Harris et al. 268 reported that patients with lower self-reported health were more likely to be referred. The evidence was, therefore, graded as no evidence.
Service factors
A number of elements were identified within the category of service factors, as follows.
The location of the GP practice (including the distance to service being referred to and whether urban or rural) was reported in 24 studies (three papers at higher risk of bias). 161,165,166,171,173,179,181,183,188,193,198,215,227,252,263,272,275,281,289,295,302,307 Of these, 18 studies165,166,171,173,179,181,188,193,198,227,252,263,272,275,289,295,307 reported an association with referral outcomes but the directions of association were very mixed. Greater distance to the specialist was reported to be associated with a reduced likelihood of referral in three studies171,173,181 and greater likelihood of referral to more local services was reported in a further seven studies. 179,181,227,272,289,295,307 A more deprived location was also associated with a reduced likelihood of referral. 252,275 Rural practices were associated with more referral in two studies198,263 but less referral in a further four studies. 165,166,188,193 Five further studies reported no association between location of GP practice or distance to the specialist service and referral outcomes. 161,183,215,281,302 The strength of this evidence was graded as inconsistent.
An association between size of the GP practice and referral outcome was reported in six studies,170,199,268,278,283,284 with no association reported by a further seven studies180,181,183,235,245,271,303 (including one study at higher risk of bias271). Of those reporting association, five reported that larger practices were associated with higher referral rates199,268,278,283,284 but one paper reported that single GP practices were associated with higher referral. 169 This evidence was graded as inconsistent.
A further 17 studies (at lower risk of bias) reported on other GP practice characteristics associated with referral outcomes, mostly relating to the fundholding or ownership of the practice. Thirteen studies reported an association166,219,235,272,276,278,283,285,300,304–307 and four studies reported no association263,296,301,303 for the following factors. Four studies reported that managed care practices were associated with higher rates of referral278,283,285,300 but two studies reported no association with referral outcomes. 263,301 Two studies reported no association between fundholding practices and rates of referral. 296,303 The other factors associated with referral outcomes were private practice associated with higher referral,166,272 greater administration resources associated with higher referral,300,304 and practice nurses or assistants associated with higher referral. 305 In addition, financial arrangements in smaller practices were associated with referral outcomes (no direction reported);235 links with a care group associated with referral outcomes (no direction reported);276 and gatekeeping role associated with referral outcomes (no direction reported). 219 One final study reported no association with referral outcomes for having an on-site service to refer to. 306 Given this complexity, the association between additional practice characteristics and referral outcomes was graded as inconsistent.
General practitioners perceived to be under greater burden or time pressure were associated with referral outcomes in 11 studies98,168,177,199,202,204,208,210,212,290,307 (including two studies at higher risk of bias202,208). Three studies reported no association. 248,270,276 This evidence was graded as stronger.
A perceived longer waiting time for the referral was associated with lower referral rates in 11 studies164,177,179,203,222,233,234,247,249,273,295 (including three at higher risk of bias203,233,295). There were no studies reporting no association and so this evidence was graded as stronger.
Greater perceived availability of the specialist was associated with more frequent referral in 12 studies168,188,199,208,210,227,290,245,289,293,307,308 (including three at higher risk of bias208,227,308). Only two studies reported no association between availability of the specialist and referral decisions. 283,299 This evidence was, therefore, graded as stronger.
Chapter 4 Summary of the evidence
What can be learned from the evidence on interventions to manage referral from primary to specialist care?
We firstly examined the overall evidence regarding referral management interventions by typology, and the overall rating of evidence of effectiveness for each group of studies which were described in the earlier sections (Figure 2).
FIGURE 2.
Summary of strength of evidence for referral management interventions. Green text, stronger evidence; blue text, weaker evidence; bold green text, conflicting evidence; standard black text, no evidence. a, Evidence in both directions (i.e. that these interventions may reduce referral but also that they may increase referral).
In the first group (practitioner education interventions), the peer-review and feedback interventions were all shown to be effective to some degree in reducing referrals, although the appropriateness of that reduction was not always considered. Although there was a higher risk of bias for one study (Cooper19), the other three studies were considered to be at lower risk of bias. 21–23 The strength of evidence for effectiveness of this type of intervention was graded as stronger. The evidence indicated that this type of intervention with individual staff had the most potential to effect change. The evidence of effectiveness for the other approaches was more mixed, perhaps owing to the variation in the training provided in terms of aim, duration and intensity. It was not possible overall to draw patterns from the data in terms of a particular type of training which may be more effective than another. Nor is it clear whether longer training programmes have greater impact than short-term or one-off interventions. However, although the evidence of effectiveness was not strong, there was some evidence that GP training could be effective in moderating referral outcomes in some contexts.
The review suggests that only in some limited situations does dissemination of guidelines have any positive effect on referral outcomes, and this is only seen over the short term. Dissemination of referral guidelines with further training, support or feedback seemed to have an inconsistent effect, with no clear patterns in terms of method of guideline development (e.g. local vs. national guidelines), or type of support provided or duration of study. There seemed to be some relationship between outcome and type of specialty; for example, low-back pain guidelines were ineffective,66,67 but those interventions that focused on cardiology (two studies56,63), endoscopy (two studies52,55), and radiology (four studies42,50,54,57) were all effective, possibly suggesting that referral guidelines may be more effective in specialties where referral criteria are clearer and more consistent between patients.
With regard to the second group of interventions (process change), there were three types where evidence of effectiveness appeared to be stronger: firstly, improving the referral information provided to specialists; secondly, enabling a community practitioner to have contact with a specialist prior to the referral; and thirdly, the introduction of electronic referral. All interventions that were focused on improving referral information were shown to be effective in improving referral-related outcomes. It was particularly apparent (given the volume of studies) that pre-referral consultation via teledermatology (where images of the skin condition were sent) was effective in moderating referral and ensuring that those referrals which were made were appropriate. Although four of the five studies here were in the specialty of dermatology,85,87,89,90 a cancer referral intervention using images was also effective,96 suggesting that the use of sending images pre referral could be used more widely than in dermatology. All interventions that reported the introduction of consultation with a specialist seemed to be effective in improving referral-related outcomes. Similarly, in nearly all cases, electronic referral systems were shown to be effective in moderating referral-related outcomes. In the ineffective study in this group,106 uptake and use of the new referral system was very low, which will have impacted on its effectiveness. These interventions seem to share a common purpose: all are designed to provide better-quality information to the specialist (either before or as part of a formal referral process).
From analysis of the studies with less clear evidence, it seemed that designated appointment slots and fast-track clinics may be effective in improving referral outcomes in some cases. The two interventions that were not effective were both focused on oncology referrals to meet the 2-week wait guidelines (although a third system for colorectal cancer referrals, in contrast, was shown to be effective). The evidence indicated that direct access to testing also might be effective in moderating referral outcomes in some cases. Of the three interventions here that did not show a clear effect, it was not possible to distinguish them from the effective interventions in terms of diagnostic test, specialty or length of the study. Decision support tools appeared to be somewhat effective in improving referral outcomes in around half of all the studies identified. We were unable to make distinctions between those studies that were effective and those that were not in terms of the content of the intervention or the specialty and/or location of the study (e.g. of three cardiology studies, one was effective109 and the other two were not114,116). Only one of the waiting list interventions was shown to have a positive effect on referral outcomes. 118 The effectiveness of this study may be due to the fact that patients had been waiting considerably longer than in the other two studies (more than 2 years).
The overall picture for interventions which aim to moderate referral outcomes by wider change at the level of the health-care system is mixed. The evidence was strongest for two types of interventions: first, community practitioners being trained to carry out additional procedures, and, second, outreach clinics. Training GPs to provide a specific procedure in the community (such as LEEP training, ECG monitoring, minor surgery or spirometry) seemed to be effective, but the GPwSI programme was shown to be more variable. Community provision by specialists in outreach clinics was generally shown to have positive effects on referral outcome measures. Two further types of intervention are highlighted in the figure as having stronger evidence of effect; however, these effects were not in a positive direction. The addition of extra nurses or counsellors in primary care did not show either clear positive effects on referral outcomes (with referral rates being no different from controls) or referral rates increasing (although it was not clear whether this increase was due to appropriate or inappropriate referrals). All four studies that evaluated the removal of gatekeeping or compared gatekeeping with an open-access system showed no (or very little) effect on referral outcome or an increase in community physician visits. 144–147 These studies highlight the potential impact on other elements of a system resulting from change to referral practice.
Other intervention types in this group had more mixed or limited evidence underpinning their use. Two interventions were identified which consisted of the return of inappropriate referrals; both showed a positive effect on referral outcomes by reducing further inappropriate referrals. 18,140 Further studies would be needed to understand whether or not this type of intervention could be effective more widely. The evidence for the effectiveness of referral management centres was very mixed, as the studies were divided in terms of whether or not they showed a positive effect on referral outcomes. In addition, of the studies showing a positive effect, two were considered to be at higher risk of bias, which may affect the reliability of their findings. The final category of interventions was patient-focused interventions. The available evidence here was limited, with the role of patients in the referral process seemingly under-researched. Of the three studies we identified, two had a significant effect on referral outcomes.
In terms of the outcomes that may result from these interventions, we found a wide range of measures of effectiveness used by studies. The outcomes divided into those earlier in the referral process, which could be considered to be at the level of the primary-care referrer, and those that were intended to have an impact at a whole-service or system-wide level. Figure 3 provides a summary of the measures and the strength of evidence underpinning interventions having an effect on that outcome. As can be seen, the areas where there was stronger evidence that interventions may have an effect were mostly in the first group: improving the provision of referral information; reducing waiting time; and increasing practitioner and patient satisfaction. There was stronger evidence of an impact on waiting times; however, there was conflicting or weaker evidence of any interventions impacting at a system-wide level on referral rates, attendance rate, cost or appropriateness.
FIGURE 3.
Reported outcomes and system impacts. Green text (uppercase and underlined), stronger evidence; blue text (uppercase), weaker evidence; bold green text (lowercase), conflicting evidence.
What are the pathways from interventions to outcomes?
We used all the elements identified and described in the preceding sections to compile an evidence-based logic model which illustrates the pathway from interventions to system-wide demand management outcomes (Figure 4). The model was constructed by listing the typology of interventions in the first column and the immediate/short-term effects that may result from interventions in the second column, and describing predictors of change (barriers or facilitators) in the third column. The final two columns detail the outcomes for demand management described in the literature at an individual level and then finally a system-level impact.
FIGURE 4.
The pathway between interventions and intended impact. Note: the arrows indicate the direction of the pathway and do not imply causality. Green text (uppercase and underlined), stronger evidence; blue text (uppercase), weaker evidence; bold green text (lowercase), conflicting evidence; no evidence, standard black text.
The model provides a detailed summary of the evidence found in the review relating to the effectiveness of interventions, the key outcomes resulting from interventions, potential obstacles to interventions effecting changed practice, and where there is stronger or weaker evidence of effect on demand management outcomes. The model highlights the complexity of the intervention change pathway, with the influence of individual-, context- and system-level factors acting as barriers and facilitators to any intervention achieving its intended impact in a particular health-care context.
The diagram also illustrates the broad scope of demand management interventions, and the range of outcomes that may result. In particular, it indicates the central role of intermediate factors in the pathway to broad system-wide impact. Currently, the majority of the intervention literature tends to use measures of system impact, whereas analysis of the pathway suggests that these intermediate elements are key important markers of change that should be evaluated in any assessment of intervention effectiveness. The model also highlights the challenges of identifying simple cause–effect relationships between individual interventions and a referral management impact.
How can evidence on interventions to manage referral from primary to specialist care be applied in a UK context, and what factors affect the applicability of international evidence in the UK?
We considered how the findings of the review were applicable and transferable to the UK NHS context. Of the 141 intervention papers we included, the largest group (n = 83) were from the UK, with an additional 19 from countries with similar systems (the Netherlands, Australia). There were a comparatively small number of North American papers (n = 24), with those countries’ very different health-care systems suggesting that the findings of the review regarding the effectiveness of interventions are applicable to the UK without special consideration. The outcomes measured and reported by studies similarly have no particular issues of applicability.
Examination of the spread of country of origin across the intervention types indicates that there is representation of UK evidence for all but two interventions approaches (none of the four gatekeeping or four payment system papers was from the UK). The following papers originated in the UK: all of the papers regarding GP peer review, 9 of 17 papers regarding GP education, all 12 papers regarding guideline dissemination, 14 of 18 papers regarding guidelines plus training/feedback, four of six papers regarding fast-track interventions, all but one of nine papers on direct access interventions, 3 of 11 papers on specialist consultation, 4 of 10 papers on electronic referral, 4 of 10 papers on decision support, one of three papers on waiting lists, six of nine papers on GP provision of specialist services, 3 of 10 papers on specialists in the community, both papers on the return of referrals, two of three papers on additional staff, all but one of six papers on referral management centre, and two of three patient-focused papers. There is UK-based evidence, therefore, across individual, process and system typologies. The review finding that, with regard to individual-level interventions, the evidence was stronger only for peer-review and feedback approaches is significant given the dominance of UK papers evaluating training or guideline provision for individual practitioners.
The non-intervention papers were more spread in terms of country of origin, with around one-third (53 of 154) originating from the UK and almost the same number (n = 52) from North America. Examination of the spread of UK studies across the factors, however, showed that there was representation in all but two categories (the one awareness of quality study was from the USA, and the one optimal timing of referral study was from the USA). This underpinning of the evidence by UK data suggests that the findings regarding these influential factors in the pathway from intervention to outcomes is applicable in the UK context.
In order to further assess how the findings of the review may be applied in the UK context, and the extent to which the diagrammatic representation of the evidence resonated with the real-life experiences of practitioners and commissioners of services, we carried out a phase of evaluation and feedback. We sought the views of key stakeholders from primary care, specialist services and NHS commissioners via a series of presentations and one-to-one meetings. In total, 44 individuals contributed to the validation stage of the work, including 15 GPs, five commissioners, seven members of the public and 17 hospital specialists. The specialties represented included infectious diseases (n = 1), gynaecology (n = 1), neurology (n = 5), palliative care (n = 2), haematology (n = 1), cardiology (n = 3), speech and language therapy (n = 1), orthopaedics (n = 1), oncology (n = 1) and respiratory medicine (n = 1).
Overall, most participants reported that they had clearly understood the logic model, with 38 respondents giving a positive first response (100% of GPs, 100% of commissioners, 76% of specialists and 71% of public respondents). Of those who felt they did not understand it (n = 6), four specialists described the model as too complex and two members of the public found it confusing.
All GPs reported that the model was a good fit with their experience of the way in which referrals are managed. In particular, they discussed how successfully the model was able to portray the ‘chaos’ of general practice. GPs also described how the model had highlighted the role of both the GPs’ and the patients’ attitudes and beliefs, the doctor–patient interaction, and especially the emotional response to the patient, which resonated very much with their experiences as a doctor. Most specialists also reported that the model was a good fit with their experience of the way referrals are managed. However, three specialists criticised the model as being a model of the literature and that this was not the same as the referral process. These participants wished to highlight that referral is often a non-linear process and also that it may be necessary to have a different model for different conditions.
The commissioners reported that the model would be useful when analysing the demand management pathway when commissioning, and for comparing what was being commissioned with what was evidence-based. GPs and commissioners also highlighted that it would be useful for people who educate GPs, and for GPs undergoing training. One GP also was positive regarding the potential of the model as a teaching aid for undergraduates. Patient and public representatives described it as useful for directing research in poorly evidenced areas, and in discussion with GP practices. However, three patient and public representatives reported that they could not see any obvious use for the model.
Many respondents mentioned factors which they thought were missing from the model but which were in fact embedded within the terms used. Although the model was able to convey a vast amount of information, some grouping and categorising had inevitably masked individual subelements. There were a small number of factors mentioned which were not included in the reviewed literature (and, therefore, not represented in the model). One specialist noted that the presence of a locum GP might affect the likelihood of referral. A patient and public representative mentioned that the influence of carers (both family members and nursing home staff) might be important in determining whether or not an elderly person was referred. A number of GPs in the focus group discussed the impact of disease burden, although it was ultimately agreed that this would fall outside the model remit, probably sitting to the left of the intervention column. A number of specialists commented on the amount of information that was sent back to the GPs after a referral, echoing comments about the referral process not always being linear. Several respondents suggested that there should be ‘feedback loops’ included in the model.
Overall, as a result of this feedback process, several changes were made to the draft model including categorising ‘conflicting evidence’ and ‘no evidence’ separately, ensuring consistency of terms throughout the model, and alterations to the arrows between the boxes.
In addition, during this validation and evaluation phase of the work, we sought to compare our findings with other international systematic reviews of evidence. A separate analysis of systematic review papers was undertaken to compare our review findings with previous reviews of the area to further explore generalisability of the current findings. Further detail on the individual systematic review papers is provided in Appendix 6.
The review of reviews focused on systematic and narrative reviews of interventions to manage referral from primary to specialist care. In total, 30 unique reviews met the inclusion criteria and were included in this review of reviews. The review papers identified both factors that influence the referral process and interventions to manage demand. Seventeen of the reviews were judged to be at lower risk of bias, with the remaining 13 at higher risk of bias, largely because of the inadequate description of the search methodologies adopted, or the primary studies included in the review.
Although the number of reviews identified and included suggests a large body of evidence of interventions to manage referrals from primary to secondary care, there was considerable duplication among the reviews identified. A number of the identified reviews drew their findings solely from previous reviews, resulting in the duplication of primary studies, and therefore of findings, across the reviews. Another consequence of this was that a relatively small number of unique primary studies included in the reviews were of relevance here.
Overall, we found that many of the other reviews had been unable to reach firm conclusions about what interventions were effective or not effective. Many reviews were unable to draw robust conclusions because of the inconsistent findings between the primary studies considered. Therefore, their findings echoed our review in concluding that interventions with similar approaches could result in different outcomes in different contexts. Other reviews also concurred with our finding that the passive dissemination of guidelines is unlikely to change referral behaviour (although there was some evidence to suggest that guidelines with education/structured checklists, and feedback and training for GPs, may improve the pre-referral management of patients). Other reviews we examined also highlighted that there was very little review-level evidence on the effectiveness of referral management centres or evidence of interventions aimed at patients and public around changing behaviour, self-care or self-appraisal, together with a lack of evidence around cost-effectiveness of interventions and their sustainability. Some of the other studies highlighted the risk of stimulating demand with interventions that provide an alternative way of accessing a service, for example outreach or attaching specialists to primary care. Many of the other reviews similar to the current work highlighted the need for a whole-systems approach to referral management.
Limitations of the study
Our systematic review and logic model synthesis has added to the existing literature by moving beyond the assumptions about outcomes, to detail fully the pathway between interventions and system-wide impact. The review was also inclusive in terms of study design and considered a large volume of literature. The potential limitations of the work, however, relate firstly to our population inclusion criteria, with the review focusing on demand management within community medical services. We recognise that other services such as community dental practices make referral to specialist services, and therefore our exclusion of studies examining these services may have omitted potentially useful data.
A key potential area of debate concerns our adopted system of quality appraisal and assessment of strength of evidence. There are many available systems for critical appraisal of studies. Our selection of a tool was based on the requirement that it would be applicable to multiple study designs; also, given the substantial volume of literature that we included, that the tool would be assess the range of potential sources of bias while not being overly long to complete. Although there were many quality appraisal systems to select from, we found considerably fewer options for examining overall strength of evidence. Although a simple tally of numbers of studies has been used by some authors, the system we selected, while we recognise its limitations, was able to consider not only quantity but also consistency in evidence and quality of evidence.
Our grouping of interventions may also be a source of limitation in interpretation of the findings. Although the typology was able to distinguish different forms of content successfully, it should be recognised that there may be overlap between categories and, although we identified the core constituents, some interventions may have included several elements.
In terms of evaluation of outcomes, at times it was difficult to establish which outcome measures should be considered as positive, for example where interventions encourage referrals irrespective of their appropriateness, or where quicker referral processes are created and are, therefore, positive; however, this has a cost implication for the system. The concept of appropriateness as an outcome was particularly challenging to interpret as views may differ between community and specialist practitioners, and also, for example, patient satisfaction and/or mental well-being may be increased by a referral; however, the referral may be considered unnecessary.
Although the evidence identified here is international in nature and some of it originates from countries with different health-care systems and processes from the UK, as we have outlined in other sections, the vast majority of studies have relevance in the UK within a NHS setting. It is likely that differences between specialties, UK demographic variation and the impact of individual patients and practitioners will have a stronger impact on the effectiveness of the interventions in a given location than will their country of origin.
We chose to use logic model methods to synthesise the review findings as they have been suggested as useful explanatory tools. The process of evaluation that we undertook following completion of the synthesis indicated that the method was able to provide a detailed illustration of multiple elements of interventions and outcomes which was viewed positively by the majority of stakeholders. The model was able to summarise a complex set of data in a single diagram; however, for some this complexity was viewed as being confusing and overcomplicated. Some practitioners pointed out that this ‘messiness’ represented the reality of endeavouring to manage referral demand, and, although the method may have limitations, it perhaps serves to confirm the challenges inherent in designing and implementing effective complex interventions.
Implications for health care
Our systematic review of the literature and logic model synthesis suggests that no one level of intervention (GP training, process change, system change or patient intervention) stands out as being much more successful than any other in producing successful referral outcomes. However, some groups of interventions may have greater potential for development, given the existing evidence that they can be effective in specific contexts.
The findings suggest that, although individual-level interventions may be popular, the stronger evidence relates to only the effectiveness of GP peer-review and feedback interventions, with evidence underpinning the implementation of formal training and referral guidelines less clear. Providing training (or reinforcement) of guidelines may aid their use.
Process change interventions appear to be most effective when the changes result in the specialist being provided with more or better-quality information about the patient – whether that is provided electronically (electronic referral) as part of the referral process, or via specialist consultation prior to the formal referral being made. The evidence is less strong for the effectiveness of process interventions which do not result in earlier interaction with a specialist (designated appointment slots/fast-track clinics, direct access to screening, and decision support tools).
With regard to system changes, the community provision of specialist services by GPs (having been previously trained by specialists), outreach or community provision by specialists, and the return of inappropriate referrals, all engage the specialist and show the stronger evidence of effect on referral outcomes. However, the evidence suggests that the addition of other primary care staff (e.g. nurses, counsellors) into a GP practice can have a negative effect on referral outcomes including referral rate and appropriateness of referral (although the amount of evidence here was limited).
The evidence for gatekeeping systems overall was very inconsistent and appeared to suggest that adding or removing a gatekeeping system had no positive impact on referral (although there were possibly small negative effects). The evidence here was weaker and originated from countries with different health-care systems from that of the UK.
Despite additional targeted searches, we found a significant lack of an evidence base to support referral management centres or other large triage systems. We were also surprised to find an almost complete lack of patient-focused interventions. This is particularly relevant given the evidence highlighting the impact that the doctor–patient relationship, and the role of patient factors, may have on the referral decision.
A key contribution of this review has been the highlighting of elements that act as mediators and moderators to intervention outcomes. We found a considerable volume of literature which endeavoured to link particular practitioner demographics to referral patterns; however, no clear associations were apparent. Instead, the factors that appeared to be important related to practitioner views and knowledge of the service which was being referred to (previous experience or familiarity with service, and satisfaction with service), and their emotional response to the patient. The importance of understanding that the GP is an individual and that each referral decision is unique was voiced strongly in feedback on the review findings during our validation work. This is further underpinned by evidence regarding the influence of individual patient factors relating to clinical condition (clinical specialty/condition, comorbidity/complexity of condition, and suitability for referral/likely benefit from referral). These factors were important in predicting whether or not referrals would be made.
In terms of service factors, the particular characteristics of the GP practice (location, size and ownership) seemed less important than factors associated with the service referred to (waiting time and availability of specialist). The burden imposed on GPs’ time by the service they were working in was also important in influencing the referral process. These local factors will influence the success and applicability of any interventions.
In interpreting the findings of this review it is important to recognise that a number of the interventions we have identified are condition specific (such as sending photographs with dermatology referrals) and that the same intervention may not be transferable across different conditions or diseases. What may be less clear but equally important is that the same also applies to the moderating and mediating factors. For example, the effect of one patient demographic factor such as age may be a strong predictor of referral in certain conditions.
This review has highlighted the value of overall consideration of the entire referral system rather than examined individual components. To tackle demand management of primary care services, the focus cannot be on primary care alone – a whole-systems approach is needed as the introduction of interventions in primary care is often just the starting point of the referral process. Patton309 has emphasised a ‘systems perspective as becoming increasingly important in dealing with and understanding real-world complexities’ (p. 120). With the introduction of interventions in primary care there are likely to be implications for secondary care. Furthermore, in a climate of ‘payment by results’, any intervention that reduces secondary care activity means a loss of income to secondary care and the implications of this would require consideration. When considering potential interventions to influence referral management, too little regard may be given to the whole referral system, including ensuring that people are referred to the most appropriate destination, that referrals are timely, that all necessary pre-referral tests have been done and that referral letters include all pertinent information. Authors such as Anderson310 argue that health-care organisations should be seen as ‘unpredictable and disorderly’, seen as ‘complex, adaptive systems’ rather than ‘a well-oiled machine’. Complexity theory suggests that it is the interaction and interdependency among elements as well as the unity as a whole that needs to be studied, with the key to understanding a health-care system being ‘patterns of relationships and interactions’. 311
In addition, many of the most complex interventions require culture change as well as a change in individual attitudes. However, often the interventions and strategies have been implemented without due regard to the challenges of changing culture or engaging individuals. This review and model detailing the pathway of change should help to emphasise the role of individuals in the change process.
Recommendations for research
-
More research is needed to develop and evaluate interventions that acknowledge the role of the patient in the referral decision. We found a lack of interventions aimed at providing health information or education to patients or to moderate their concerns or satisfaction regarding a referral. This is an important aspect to tackle, as our model showed that both patient pressure and their relationship with their GP can affect whether or not a referral occurs.
-
Research is also required to better understand the relationship between GP knowledge and GP attitudes and beliefs in terms of how an intervention is framed and how responsive practitioners are to change. The review findings indicate that attitudes and beliefs of the patient and the GP, as well as the doctor–patient interaction, are potentially important mechanisms of change which interventions should seek to impact and should measure in outcome evaluations. It is suggested that interventions focusing on these have a greater potential for impacting on referral demand outcomes. This suggests that interventions which act only to change knowledge may not be as effective as those acting on attitudes and beliefs. This is particularly important for future intervention study design as knowledge is relatively easy to measure and therefore more likely to be included, whereas change in attitudes and beliefs may be more challenging to evaluate.
-
This work highlights that intermediate outcomes such as the content of the referral provided to the specialist are important in the referral pathway. It is only by recognising and evaluating these individual outcomes that the intervention change pathway can be understood. It is recommended that researchers include measures of these intermediate outcomes in their evaluation of intervention effectiveness in order to determine where blocks or facilitators to system-wide impact may be occurring.
Acknowledgements
We would like to acknowledge the members of our steering group for their invaluable contribution to this research: Dr Martin McShane, NHS England; Professor Danuta Kasprzyk (Department of Global Health, University of Washington); Professor Helena Britt (Family Medicine Research Centre, University of Sydney); Ellen Nolte (RAND Corporation Europe/international); Jon Karnon (School of Population Health and Clinical Practice, University of Adelaide); Christine Allmark [patient and public involvement (PPI)]; and Brian Hodges (PPI). Thanks also to all of those who contributed to the validation stage of the work.
Contributions of authors
Dr Lindsay Blank (Research Fellow) was the principal investigator and study manager for this project. She acted as lead reviewer and drafted the final report.
Dr Susan Baxter (Research Fellow) led the logic model development aspect of the work, contributed as a reviewer to all aspects and revised the report following reviewer feedback.
Helen Buckley Woods (Information Specialist) took the lead in the searching aspects of the work.
Professor Elizabeth Goyder (Professor of Public Health) provided oversight and guidance to the project.
Dr Andrew Lee (Senior Clinical University Teacher in Public Health/GP) provided a strategic overview to the project from a NHS perspective and provided expertise in translation of the findings to the NHS context.
Professor Nick Payne (Honorary Professor of Public Health) provided oversight and guidance to the project in terms of review methods and the NHS context.
Melanie Rimmer (Research Assistant) contributed to data collecting during the validation stage and proofreading the report.
All authors have read and approved the final report.
Publications
Baxter SK, Blank L, Woods HB, Payne N, Rimmer M, Goyder E. Using logic model methods in systematic review synthesis: describing complex pathways in referral management interventions. BMC Med Res Methodol 2014;14:62.
Blank L, Baxter S, Woods BH, Goyder E, Lee A, Payne N. Referral interventions from primary to specialist care: a systematic review of international evidence. Br J Gen Pract 2014;64:e765–74.
Disclaimers
This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health.
References
- Akbari A, Mayhew A, Al-Alawi MA, Grimshaw J, Winkens R, Glidewell E, et al. Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev 2008;4. http://dx.doi.org/10.1002/14651858.CD005471.pub2.
- Faulkner A, Mills N, Bainton D, Baxter K, Kinnersley P, Peters TJ, et al. A systematic review of the effect of primary care-based service innovations on quality and patterns of referral to specialist secondary care. Br J Gen Pract 2003;53:878-84.
- Dunst CJ, Gorman E. Practices for increasing referrals from primary care physicians. Cornerstones 2006;2:1-10.
- Grimshaw JM, Winkens RA, Shirran L, Cunningham C, Mayhew A, Thomas R, et al. Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev 2005;3.
- Imison C, Naylor C. Referral Management – Lessons for Success 2010. www.kingsfund.org.uk/sites/files/kf/Referral-management-lessons-for-success-Candace-Imison-Chris-Naylor-Kings-Fund-August2010.pdf (accessed 24 January 2014).
- Rees K, Bennett P, West R, Davey Smith G, Ebrahim S. Psychological interventions for coronary heart disease. Cochrane Database Syst Rev 2004;2. http://dx.doi.org/10.1002/14651858.CD002902.pub2.
- Weiss C, Connell J, Kubisch A, Schorr L, Weiss C. New Approaches to Evaluating Community Initiatives Vol 1: Concepts, Methods and Contexts. Washington, DC: Aspen Institute; 1995.
- Weiss C. Theory-based evaluation: past, present and future. New Dir Eval 2007;76:68-81.
- Dyson A, Todd L. Dealing with complexity: theory of change evaluation and the full service extended schools initiative. Int J Res Method Educ 2010;33:119-34. http://dx.doi.org/10.1080/1743727X.2010.484606.
- Blamey A, Mackenzie M. Theories of change and realistic evaluation: peas in a pod or apples and oranges. Evaluation 2007;13:439-55. http://dx.doi.org/10.1177/1356389007082129.
- Connell J, Kubisch A, Fullbright-Anderson K, Kubisch A, Connell J. New Approaches to Evaluating Community Initiatives, Vol 2: Theory, Measurement and Analysis. Queenstown: The Aspen Institute; 1998.
- EPPI-Centre Methods for Conducting Systematic Reviews. 2010.
- Grant MJ, Brettle A, Long AF. Developing a Review Question: A Spiral Approach to Literature Searching. Poster Presentation n.d.
- Baxter S, Killoran A, Kelly M, Goyder E. Synthesising diverse evidence: the use of primary qualitative data analysis methods and logic models in public health reviews. Public Health 2010;124:99-106. http://dx.doi.org/10.1016/j.puhe.2010.01.002.
- Allmark P, Baxter S, Goyder E, Guillaume L, Crofton-Martin G. Assessing the health benefits of advice services: using research evidence and logic model methods to explore complex pathways. Health Soc Care Comm 2013;21:59-68. http://dx.doi.org/10.1111/j.1365-2524.2012.01087.x.
- Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8. http://dx.doi.org/10.1186/1471-2288-8-45.
- Hoogendoorn WE, van Poppel MNM, Bongers PM, Koes BW, Bouter LM. Physical load during work and leisure time as risk factors for back pain. Scand J Work Environ Health 1999;25:387-403. http://dx.doi.org/10.5271/sjweh.451.
- Wylie KR, Davies-South D. Returning a referral for erectile dysfunction to the referrer by two different routes. BJU Int 2001;87:846-8. http://dx.doi.org/10.1046/j.1464-410x.2001.02188.x.
- Cooper R. How peer review reduced GP referrals by 25% in two months. Pulse 2012;72.
- Abel G, Thompson L. General practitioners, specialists and surveillance guidelines: Interpreting the socio-clinical context of decision-making. Health Risk Soc 2011;13:547-59. http://dx.doi.org/10.1080/13698575.2011.615826.
- Evans E. The Torfaen referral evaluation project. Qual Prim Care 2009;17:423-9.
- Evans E, Aiking H, Edwards A. Reducing variation in general practitioner referral rates through clinical engagement and peer review of referrals: a service improvement project. Qual Prim Care 2011;19:263-72.
- Jiwa M, Walters S, Mathers N. Referral letters to colorectal surgeons: the impact of peer-mediated feedback. Br J Gen Pract 2004;54:123-6.
- Hilty DM, Yellowlees PM, Nesbitt TS. Evolution of telepsychiatry to rural sites: changes over time in types of referral and in primary care providers’ knowledge, skills and satisfaction. Gen Hosp Psychiatry 2006;28:367-73. http://dx.doi.org/10.1016/j.genhosppsych.2006.05.009.
- Lam TP, Chow RWM, Lam KF, Lennox IM, Chan FHW, Tsoi SLT. Evaluation of the learning outcomes of a year-long postgraduate training course in community geriatrics for primary care doctors. Arch Gerontol Geriatr 2011;52:350-6. http://dx.doi.org/10.1016/j.archger.2010.05.019.
- Rowlands G, Sims J, Kerry S, Keene D, Hilton S. Within-practice educational meetings and GP referrals to secondary care: an aid to reflection and review of clinical practice. Educ 2003;14:449-62.
- Ramsay CR, Eccles M, Grimshaw JM, Steen N. Assessing the long-term effect of educational reminder messages on primary care radiology referrals. Clin Radiol 2003;58:319-21. http://dx.doi.org/10.1016/S0009-9260(02)00524-X.
- Shariff ZR, Roshan A, Williams AM, Platt AJ. 2-week wait referrals in suspected skin cancer: does an instructional module for general practitioners improve diagnostic accuracy?. Surgeon 2010;8:247-51. http://dx.doi.org/10.1016/j.surge.2010.03.004.
- Kousgaard KR, Nielsen JD, Olesen F, Jensen AB. General practitioner assessment of structured oncological information accompanying newly referred cancer patients. Scand J Prim Health Care 2003;21:110-14. http://dx.doi.org/10.1080/02813430310001725.
- Bennett K, Haggard M, Churchill R, Wood S. Improving referrals for glue ear from primary care: are multiple interventions better than one alone?. J Health Serv Res Policy 2001;6:139-44. http://dx.doi.org/10.1258/1355819011927387.
- Donohoe ME, Fletton JA, Hook A, Powell R, Robinson I, Stead JW, et al. Improving foot care for people with diabetes mellitus – a randomised controlled trial of an integrated care approach. Diabet Med 2000;17:581-7. http://dx.doi.org/10.1046/j.1464-5491.2000.00336.x.
- Watson E, Clements A, Yudkin P, Rose P, Bukach C, Mackay J, et al. Evaluation of the impact of two educational interventions on GP management of familial breast/ovarian cancer cases: a cluster randomised controlled trial. Br J Gen Pract 2001;51:817-21.
- Adams SGP, Pitts J, Wynne J, Yawn BP, Diamond EJ, Lee S, et al. Effect of a primary care continuing education program on clinical practice of chronic obstructive pulmonary disease: translating theory into practice. Mayo Clin Proc 2012;87:862-70. http://dx.doi.org/10.1016/j.mayocp.2012.02.028.
- Hands S, Holbrook J. Are one-to-one outpatient teaching sessions a useful way for GPs to undertake their continuing education?. Educ 2001;12:392-400.
- Suris X, Cerda D, Ortiz-Santamaria V, Ponce A, Simon JL, Calvo E, et al. A rheumatology consultancy program with general practitioners in Catalonia, Spain. J Rheumatol 2007;34:1328-31.
- Wolters R, Wensing M, Klomp M, Lagro-Jansen T, van Weel C, Grol R. Effects of distance learning on clinical management of LUTS in primary care: a randomised trial. Patient Educ Couns 2005;59:212-18. http://dx.doi.org/10.1016/j.pec.2004.11.009.
- Bhalla RK, Unwin D, Jones TM, Lesser T. Does clinical assistant experience in ENT influence general practitioner referral rates to hospital?. J Laryngol Otol 2002;116:586-8. http://dx.doi.org/10.1258/00222150260171542.
- Ellard R, Gulati A, Hubbard V, Bull R, Pozo-Garcia L, Goldsmith P. The impact of physician education on primary care referrals in dermatology. Br J Dermatol 2012;167.
- Lester H, Birchwood M, Freemantle N, Michail M, Tait L. REDIRECT: cluster randomised controlled trial of GP training in first-episode psychosis. Br J Gen Pract 2009;59:e183-90. http://dx.doi.org/10.3399/bjgp09X420851.
- Emmerson B, Frost A, Powell J, Ward W, Barnes M, Frank RE-MA, et al. Evaluating a GP consultative psychiatric service in an Australian metropolitan hospital district. Australas 2003;11:195-8. http://dx.doi.org/10.1046/j.1039-8562.2003.00547.x.
- Malik A, Wigney L, Murray S, Gerrand CH. The effectiveness of ‘two-week’ referrals for suspected bone and soft tissue sarcoma. Sarcoma 2007;11:1-3. http://dx.doi.org/10.1155/2007/23870.
- Twomey P. Making the best use of a radiology department: an example of implementation of a referral guideline within a primary care organisation. Qual Prim Care 2003;11:53-9.
- Cusack CM, Buckley CC. Efficacy of issuing guidelines on acne management to general practitioners. Br J Dermatol 2005;152:1392-3. http://dx.doi.org/10.1111/j.1365-2133.2005.06641.x.
- Idiculla JM, Perros P, Frier BM. Do diabetes guidelines influence the content of referral letters by general practitioners to a diabetes specialist clinic?. Health Bull (Edinb) 2000;58:322-7.
- Lucassen AW, Watson E, Harcourt J, Rose P, O’Grady J. Guidelines for referral to a regional genetics service: GPs respond by referring more appropriate cases. Fam Pract 2001;18:135-40. http://dx.doi.org/10.1093/fampra/18.2.135.
- Potter S, Govindarajulu S, Shere M, Braddon F, Curran G, Greenwood R, et al. Referral patterns, cancer diagnoses, and waiting times after introduction of two week wait rule for breast cancer: prospective cohort study. BMJ 2007;335:288-90. http://dx.doi.org/10.1136/bmj.39258.688553.55.
- Imkampe A, Bendall S, Chianakwalam C. Two-week rule: has prioritisation of breast referrals by general practitioners improved?. Breast 2006;15:654-8. http://dx.doi.org/10.1016/j.breast.2006.02.002.
- Fearn S, Larner J. Have Quality and Outcomes Framework Depression Indicators changed referrals from primary care to a dedicated memory clinic?. Ment Health Fam Med 2009;6:129-32.
- Hill VAW, Wong E, Hart CJ. General practitioner referral guidelines for dermatology: do they improve the quality of referrals?. Clin Exp Dermatol 2000;25:371-6. http://dx.doi.org/10.1046/j.1365-2230.2000.00665.x.
- Matowe L, Ramsay CR, Grimshaw JM, Gilbert FJ, Macleod MJ, Needham G, et al. Effects of mailed dissemination of the Royal College of Radiologists’ guidelines on general practitioner referrals for radiography: a time series analysis. Clin Radiol 2002;57:575-8. http://dx.doi.org/10.1053/crad.2001.0894.
- Melia J, Coulson P, Coleman D, Moss S. Urological referral of asymptomatic men in general practice in England. Br J Cancer 2008;98:1176-81. http://dx.doi.org/10.1038/sj.bjc.6604291.
- West YRK, Kendrick BL, Williamson DM. Evaluation of the impact of orthopaedic guidelines on referrals from primary care to a specialist department. Qual Prim Care 2007;15:27-31.
- Banait G, Sibbald B, Thompson D, Summerton C, Hann M, Talbot S. Modifying dyspepsia management in primary care: a cluster randomised controlled trial of educational outreach compared with passive guideline dissemination. Br J Gen Pract 2003;53:94-100.
- Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet 2001;357:1406-9. http://dx.doi.org/10.1016/S0140-6736(00)04564-5.
- Elwyn G, Owen D, Roberts L, Wareham K, Duane P, Allison M, et al. Influencing referral practice using feedback of adherence to NICE guidelines: a quality improvement report for dyspepsia. Qual Saf Health Care 2007;16:67-70. http://dx.doi.org/10.1136/qshc.2006.019992.
- Wright J, Harrison S, McGeorge M, Patterson C, Russell I, Russell D, et al. Improving the management and referral of patients with transient ischaemic attacks: a change strategy for a health community. Qual Saf Health Care 2006;15:9-12. http://dx.doi.org/10.1136/qshc.2005.014704.
- Glaves J. The use of radiological guidelines to achieve a sustained reduction in the number of radiographic examinations of the cervical spine, lumbar spine and knees performed for GPs. Clin Radiol 2005;60:914-20. http://dx.doi.org/10.1016/j.crad.2005.02.013.
- Griffiths C, Taylor H, Collins SI, Hobson JE, Collier PA, Chalmers RJ, et al. The impact of psoriasis guidelines on appropriateness of referral from primary to secondary care: a randomised controlled trial. British Association of Dermatologists 86th Annual Meeting, abstract O-2. Br J Dermatol 2006;155. http://dx.doi.org/10.1111/j.1365-2133.2006.07343.x.
- Kerry S, Oakeshott P, Dundas D, Williams J. Influence of postal distribution of the Royal College of Radiologists’ guidelines, together with feedback on radiological referral rates, on X-ray referrals from general practice: a randomised controlled trial. Fam Pract 2000;17:46-52. http://dx.doi.org/10.1093/fampra/17.1.46.
- Robling MR, Houston HL, Kinnersley P, Hourihan MD, Cohen DR, Hale J, et al. General practitioners’ use of magnetic resonance imaging: an open randomised trial comparing telephone and written requests and an open randomised controlled trial of different methods of local guideline dissemination. Clin Radiol 2002;57:402-7. http://dx.doi.org/10.1053/crad.2001.0864.
- White T, Marriott S. Using evidence-based dissemination and implementation strategies to improve routine communication between general practitioners and community mental health teams. Psychiatr Bull 2004;28:8-11. http://dx.doi.org/10.1192/pb.28.1.8.
- Julian S, Naftalin NJ, Clark M, Szczepura A, Rashid A, Baker R, et al. An integrated care pathway for menorrhagia across the primary–secondary interface: patients’ experience, clinical outcomes, and service utilisation. Qual Saf Health Care 2007;16. http://dx.doi.org/10.1136/qshc.2005.016782.
- Walkowski K, Peel C, Sandy L. Effect of academic detailing on primary care referral patterns to high performing cardiac specialists. J Gen Intern Med 2007;22:196-7.
- Morrison J, Carroll L, Twaddle S, Cameron I, Grimshaw J, Leyland A, et al. Pragmatic randomised controlled trial to evaluate guidelines for the management of infertility across the primary care–secondary care interface. BMJ 2001;322:1282-4. http://dx.doi.org/10.1136/bmj.322.7297.1282.
- Wilson BJ, Torrance N, Mollison J, Watson MS, Douglas A, Miedzybrodzka Z, et al. Cluster randomised trial of a multifaceted primary care decision-support intervention for inherited breast cancer risk. Fam Pract 2006;23:537-44. http://dx.doi.org/10.1093/fampra/cml026.
- Dey P, Simpson CW, Collins SI, Hodgson G, Dowrick CF, Simison AJ, et al. Implementation of RCGP guidelines for acute low back pain: a cluster randomised controlled trial. Br J Gen Pract 2004;54:33-7.
- Engers AJ, Wensing M, van Tulder MW, Timmermans A, Oostendorp RAB, Koes BW, et al. Implementation of the Dutch low back pain guideline for general practitioners: a cluster randomised controlled trial. Spine 2005;30:595-600. http://dx.doi.org/10.1097/01.brs.0000155406.79479.3a.
- Jiwa MS, Skinner P, Coker AO, Shaw L, Campbell MJ, Thompson J. Implementing referral guidelines: lessons from a negative outcome cluster randomised factorial trial in general practice. BMC Fam Pract 2006;7. http://dx.doi.org/10.1186/1471-2296-7-65.
- Spatafora S, Canepa G, Migliari R, Rotondo S, Mandressi A, Puppo P, et al. Effects of a shared protocol between urologists and general practitioners on referral patterns and initial diagnostic management of men with lower urinary tract symptoms in Italy: the Prostate Destination study. BJU Int 2005;95:563-70. http://dx.doi.org/10.1111/j.1464-410X.2005.05340.x.
- Bridgman S, Li X, Mackenzie G, Dawes P. Does the North Staffordshire slot system control demand of orthopaedic referrals from primary care? A population-based survey in general practice. Br J Gen Pract 2005;55:704-9.
- Khan A, Smith DA, Whittaker J, Williams A, Khan D, Harvey JE, et al. Effectiveness of direct GP referrals to hospital specialist respiratory teams in avoiding acute admissions. Thorax 2008;63.
- Sved-Williams A, Poulton JE. Primary care mental health consultation-liaison: a connecting system for private psychiatrists and general practitioners. Australas Psychiatry 2010;18:125-9. http://dx.doi.org/10.3109/10398560903469783.
- Hemingway DM, Jameson J, Kelly MJ. Leicester Colorectal Specialist Interest Group Project Steering Committee. Straight to test: introduction of a city-wide protocol driven investigation of suspected colorectal cancer. Colorectal Dis 2006;8:289-95. http://dx.doi.org/10.1111/j.1463-1318.2005.00935.x.
- McNally OM, Wareham V, Flemming DJ, Cruickshank ME, Parkin DE, McNally OM, et al. The impact of the introduction of a fast track clinic on ovarian cancer referral patterns. Eur J Cancer Care (Engl) 2003;12:327-30. http://dx.doi.org/10.1046/j.1365-2354.2003.00419.x.
- Prades J, Espinas JA, Font R, Argimon JM, Borras JM. Implementing a cancer fast-track programme between primary and specialised care in Catalonia (Spain): a mixed methods study. Br J Cancer 2011;105:753-9. http://dx.doi.org/10.1038/bjc.2011.308.
- DAMASK (Direct Access to Magnetic Resonance Imaging: Assessment for Suspect Knees) Trial Team . Cost-effectiveness of magnetic resonance imaging of the knee for patients presenting in primary care. Br J Gen Pract 2008;58:e10-16. http://dx.doi.org/10.3399/bjgp08X342660.
- Shaw IS, Valori RM, Charlett A, McNulty CA. Limited impact on endoscopy demand from a primary care based ‘test and treat’ dyspepsia management strategy: the results of a randomised controlled trial. Br J Gen Pract 2006;56:369-74.
- Simpson GC, Forbes K, Teasdale E, Tyagi A, Santosh C. Impact of GP direct-access computerised tomography for the investigation of chronic daily headache. Br J Gen Pract 2010;60:897-901. http://dx.doi.org/10.3399/bjgp10X544069.
- Thomas RE, Grimshaw JM, Mollison J, McClinton S, McIntosh E, Deans H, et al. Cluster randomised trial of a guideline-based open access urological investigation service. Fam Pract 2003;20:646-54. http://dx.doi.org/10.1093/fampra/cmg605.
- Thomas R, Cook A, Main G, Taylor T, Galizia CE, Swingler R, et al. Primary care access to computed tomography for chronic headache. Br J Gen Pract 2010;60:426-30. http://dx.doi.org/10.3399/bjgp10X502146.
- Wong BC, Chan CK, Wong KW, Wong WM, Yuen MF, Lai KC, et al. Evaluation of a new referral system for the management of dyspepsia in Hong Kong: role of open-access upper endoscopy. J Gastroenterol Hepatol 2000;15:1251-6. http://dx.doi.org/10.1046/j.1440-1746.2000.2353.x.
- Dhillon V, Creiger J, Hannan J, Hurst N, Nuki G. The effect of DXA scanning on clinical decision making by general practitioners: a randomised, prospective trial of direct access versus referral to a hospital consultant. Osteoporos Int 2003;14:326-33. http://dx.doi.org/10.1007/s00198-002-1371-2.
- Eley KA, Fitzgerald JE. Direct general practitioner referrals to audiology for the provision of hearing aids: a single centre review. Qual Prim Care 2010;18:201-6.
- Gough-Palmer AL, Burnett C, Gedroyc WM. Open access to MRI for general practitioners: 12 years’ experience at one institution – a retrospective analysis. Br J Radiol 2009;82:687-90. http://dx.doi.org/10.1259/bjr/88267089.
- Leggett P, Gilliland AE, Cupples ME, McGlade K, Corbett R, Stevenson M, et al. A randomised controlled trial using instant photography to diagnose and manage dermatology referrals. Fam Pract 2004;21:54-6. http://dx.doi.org/10.1093/fampra/cmh112.
- Eminovic N, de Keizer NF, Wyatt JC, ter Riet G, Peek N, van Weert HC, et al. Teledermatologic consultation and reduction in referrals to dermatologists: a cluster randomised controlled trial. Arch Dermatol 2009;145:558-64. http://dx.doi.org/10.1001/archdermatol.2009.44.
- Whited JD, Hall RP, Foy ME, Marbrey LE, Grambow SC, Dudley TK, et al. Teledermatology’s impact on time to intervention among referrals to a dermatology consult service. Telemed J E Health 2002;8:313-21. http://dx.doi.org/10.1089/15305620260353207.
- Whited JD, Hall RP, Foy ME, Marbrey LE, Grambow SC, Dudley TK, et al. Patient and clinician satisfaction with a store-and-forward teledermatology consult system. Telemed J E Health 2004;10:422-31. http://dx.doi.org/10.1089/tmj.2004.10.422.
- McKoy KC, DiGregorio S, Stira L. Asynchronous teledermatology in an urban primary care practice. Telemed J E Health 2004;10:S70-80. http://dx.doi.org/10.1089/1530562042632001.
- Knol A, van den Akker TW, Damstra RJ, de Haan J. Teledermatology reduces the number of patient referrals to a dermatologist. J Telemed Telecare 2006;12:75-8. http://dx.doi.org/10.1258/135763306776084365.
- Hockey AD, Wootton R, Casey T. Trial of low-cost teledermatology in primary care. J Telemed Telecare 2004;10:44-7. http://dx.doi.org/10.1258/1357633042614221.
- Nielsen JDP, Palshof T, Mainz J, Jensen AB, Olesen F. Randomised controlled trial of a shared care programme for newly referred cancer patients: bridging the gap between general practice and hospital. Qual Saf Health Care 2003;12:263-72. http://dx.doi.org/10.1136/qhc.12.4.263.
- Harrington JT, Dopf CA, Chalgren CS. Implementing guidelines for interdisciplinary care of low back pain: a critical role for pre-appointment management of specialty referrals. Jt Comm J Qual Improv 2001;27:651-63.
- Wallace P, Barber J, Clayton W, Currell R, Fleming K, Garner P, et al. Virtual outreach: a randomised controlled trial and economic evaluation of joint teleconferenced medical consultations. Health Technol Assess 2004;8. http://dx.doi.org/10.3310/hta8500.
- Jaatinen T, Aarnio P, Remes J, Hannukainen J, Koymari S. Teleconsultation as a replacement for referral to an outpatient clinic. J Telemed Telecare 2002;8:102-6. http://dx.doi.org/10.1258/1357633021937550.
- Tadros A, Murdoch R, Stevenson JH. Digital image referral for suspected skin malignancy – a pilot study of 300 patients. J Plast Reconstr Aesthet Surg 2009;62:1048-53. http://dx.doi.org/10.1016/j.bjps.2008.02.005.
- Nicholson C, Jackson CL, Wright B, Mainwaring P, Holliday D, Lankowski A, et al. Online referral and OPD booking from the GP desktop. Aust Health Rev 2006;30:397-404. http://dx.doi.org/10.1071/AH060397.
- Kim Y, Chen AH, Keith E, Yee HF, Kushel MB. Not perfect, but better: primary care providers’ experiences with electronic referrals in a safety net health system. J Gen Intern Med 2009;24:614-19. http://dx.doi.org/10.1007/s11606-009-0955-3.
- Dennison J, Eisen S, Towers M, Ingham CC. An effective electronic surgical referral system. Ann R Coll Surg Engl 2006;88:554-6. http://dx.doi.org/10.1308/003588406X130642.
- Chen AH, Kushel MB, Grumbach K, Yee HF. A safety-net system gains efficiencies through ‘eReferrals’ to specialists. Health Affairs 2010;29:969-71. http://dx.doi.org/10.1377/hlthaff.2010.0027.
- Gandhi TK, Sittig DF, Franklin M, Sussman AJ, Fairchild DG, Bates DW, et al. Communication breakdown in the outpatient referral process. J Gen Intern Med 2000;15:626-31. http://dx.doi.org/10.1046/j.1525-1497.2000.91119.x.
- Kim-Hwang JE, Chen AH, Bell DS, Guzman D, Yee HF, Kushel MB. Evaluating electronic referrals for specialty care at a public hospital. J Gen Intern Med 2010;25:1123-8. http://dx.doi.org/10.1007/s11606-010-1402-1.
- Stoves J, Connolly J, Cheung C. Electronic consultation as an alternative to hospital referral for patients with chronic kidney disease: a novel application for networked electronic health records to improve the accessibility and efficiency of healthcare. BMJ Quality and Safety 2010;19:1-4. http://dx.doi.org/10.1136/qshc.2009.038984.
- Patterson V, Humphreys J, Chua R. E-mail triage of new neurological outpatient referrals from general practice. J Neurol Neurosurg Psychiatry 2004;75:617-20. http://dx.doi.org/10.1136/jnnp.2003.024489.
- Jiwa MD, Dhaliwal S. Referral Writer: preliminary evidence for the value of comprehensive referral letters. Qual Prim Care 2012;20:39-45.
- Kennedy AM, Aziz A, Khalid S, Hurman D. Do GP referral guidelines really work? Audit of an electronic urgent referral system for suspected head and neck cancer. Eur Arch Otorhinolaryngol 2012;269:1509-12. http://dx.doi.org/10.1007/s00405-011-1788-3.
- McGowan J, Hogg W, Campbell C, Rowan M. Just-in-time information improved decision-making in primary care: a randomised controlled trial. PLOS ONE 2008;3. http://dx.doi.org/10.1371/journal.pone.0003785.
- Gandhi TK, Keating NL, Ditmore M, Kiernan D, Johnson R, Burdick E, et al. Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 3: Performance and Tools). Rockville, MD: Agency for Healthcare Research and Quality; 2008.
- Junghans C, Feder G, Timmis AD, Eldridge S, Sekhri N, Black N, et al. Effect of patient-specific ratings vs conventional guidelines on investigation decisions in angina – appropriateness of referral and investigation in angina (ARIA) trial. Arch Intern Med 2007;167:195-202. http://dx.doi.org/10.1001/archinte.167.2.195.
- Akbari A, Grimshaw J, Stacey D, Hogg W, Ramsay T, Cheng-Fitzpatrick M, et al. Change in appropriate referrals to nephrologists after the introduction of automatic reporting of the estimated glomerular filtration rate. CMAJ 2012;184:E269-76. http://dx.doi.org/10.1503/cmaj.110678.
- Emery J, Morris H, Goodchild R, Fanshawe T, Prevost AT, Bobrow M, et al. The GRAIDS trial: a cluster randomised controlled trial of computer decision support for the management of familial cancer risk in primary care. Br J Cancer 2007;97:486-93. http://dx.doi.org/10.1038/sj.bjc.6603897.
- Knab JH, Wallace MS, Wagner RL, Tsoukatos J, Weinger MB. The use of a computer-based decision support system facilitates primary care physicians’ management of chronic pain. Anaesth Analg 2001;93:712-20. http://dx.doi.org/10.1097/00000539-200109000-00035.
- Mariotti G, Meggio A, de Pretis G, Gentilini M. Improving the appropriateness of referrals and waiting times for endoscopic procedures. J Health Serv Res Policy 2008;13:146-51. http://dx.doi.org/10.1258/jhsrp.2008.007170.
- Greiver M, Drummond N, White D, Weshler J, Moineddin R. Angina on the Palm: randomised controlled pilot trial of Palm PDA software for referrals for cardiac testing. Can Fam Physician 2005;51:382-3.
- Magill MK, Day J, Mervis A, Donnelly SM, Parsons M, Baker AN, et al. Improving colonoscopy referral rates through computer-supported, primary care practice redesign. J Healthc Qual 2009;31:43-52. http://dx.doi.org/10.1111/j.1945-1474.2009.00037.x.
- Tierney WMO, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, et al. Effects of computerised guidelines for managing heart disease in primary care: a randomised, controlled trial. J Gen Intern Med 2003;18:967-76. http://dx.doi.org/10.1111/j.1525-1497.2003.30635.x.
- Slade M, Gask L, Leese M, McCrone P, Montana C, Powell R, et al. Failure to improve appropriateness of referrals to adult community mental health services – lessons from a multi-site cluster randomised controlled trial. Fam Pract 2008;25:181-90. http://dx.doi.org/10.1093/fampra/cmn025.
- Stainkey LA, Seidl IA, Johnson AJ, Tulloch GE, Pain T. The challenge of long waiting lists: how we implemented a GP referral system for non-urgent specialist’ appointments at an Australian public hospital. BMC Health Serv Res 2010;10. http://dx.doi.org/10.1186/1472-6963-10-303.
- King RHB, Bateman H. Piloting an approach to the identification of avoidable referrals in a general practice with a high referral rate. J Clin Excel 2001;2:209-13.
- van Bokhoven MA, Koch H, van der Weijden T, Weekers-Muyres AH, Bindels PJ, Grol RP, et al. The effect of watchful waiting compared to immediate test ordering instructions on general practitioners’ blood test ordering behaviour for patients with unexplained complaints; a randomised clinical trial (ISRCTN55755886). Implement Sci 2012;7. http://dx.doi.org/10.1186/1748-5908-7-29.
- Callaway P, Frisch L. Does a family physician who offers colposcopy and LEEP need to refer patients to a gynaecologist?. J Fam Pract 2000;49:534-6.
- Standing PD. Changes in referral patterns to cardiac out-patient clinics with ambulatory ECG monitoring in general practice. Br J Cardiol 2001;8:394-9.
- van Dijk CE, Verheij RA, Spreeuwenberg P, Groenewegen PP, de Bakker DH, van Dijk CE, et al. Minor surgery in general practice and effects on referrals to hospital care: observational study. BMC Health Serv Res 2011;11. http://dx.doi.org/10.1186/1472-6963-11-2.
- Ridsdale L, Doherty J, McCrone P, Seed P. Headache Innovation and Evaluation Group. A new GP with special interest headache service: observational study. Br J Gen Pract 2008;58:478-83. http://dx.doi.org/10.3399/bjgp08X319440.
- Salisbury C, Noble A, Horrocks S, Crosby Z, Harrison V, Coast J, et al. Evaluation of a general practitioner with special interest service for dermatology: randomised controlled trial. BMJ 2005;331:1441-6. http://dx.doi.org/10.1136/bmj.38670.494734.7C.
- Sanderson D. Evaluation of the GPs with Special Interests (GPwSIs) Pilot Projects within the Action On ENT Programme: Final Report. Department of Health Modernisation Agency, York Health Economics Consortium; 2002.
- Sauro AS, Scalzitti F, Buono N, Siringano R, Petrazzuoli F, Diodati G, et al. Spirometry is really useful and feasible in the GPs’ daily practice but guidelines alone are not. Eur J Gen Pract 2005;11:29-31. http://dx.doi.org/10.3109/13814780509178015.
- Rosen R, Jones R, Tomlin Z, Cavanagh M. Evaluation of General Practitioners with Special Interests: Access, Cost Evaluation and Satisfaction with Services. NIHR Service Delivery and Organisation; 2006.
- Levell NJ, Penart-Lanau AM, Garioch JJ. Introduction of intermediate care dermatology services in Norfolk, England was followed by a 67% increase in referrals to the local secondary care dermatology department. Br J Dermatol 2012;167:443-5. http://dx.doi.org/10.1111/j.1365-2133.2012.10850.x.
- Leiba AM, Martonovits G, Magnezi R, Goldberg A, Carroll J, Benedek P, et al. Evaluation of a specialist outreach clinic in a primary healthcare setting: the effect of easy access to specialists. Clinician Manag 2002;11:131-6.
- Campbell H, Holloway S, Cetnarskyj R, Anderson E, Rush R, Fry A, et al. Referrals of women with a family history of breast cancer from primary care to cancer genetics services in South East Scotland. Br J Cancer 2003;89:1650-6. http://dx.doi.org/10.1038/sj.bjc.6601348.
- Felker BL, Barnes RF, Greenberg DM, Chaney EF, Shores MM, Gillespie-Gateley L, et al. Preliminary outcomes from an integrated mental health primary care team. Psychiatr Serv 2004;55:442-4. http://dx.doi.org/10.1176/appi.ps.55.4.442.
- Gurden M, Morelli M, Sharp G, Baker K, Betts N, Bolton J, et al. Evaluation of a general practitioner referral service for manual treatment of back and neck pain. Prim Health Care Res Dev 2012;13:204-10. http://dx.doi.org/10.1017/S1463423611000648.
- Schulpen GJ, Vierhout WP, van der Heijde DM, Landewe RB, Winkens RA, van der Linden S, et al. Joint consultation of general practitioner and rheumatologist: does it matter?. Ann Rheum Dis 2003;62:159-61. http://dx.doi.org/10.1136/ard.62.2.159.
- Vlek JF, Vierhout WP, Knottnerus JA, Schmitz JJ, Winter J, Wesselingh-Megens AM, et al. A randomised controlled trial of joint consultations with general practitioners and cardiologists in primary care. Br J Gen Pract 2003;53:108-12.
- Hughes-Anderson W, Rankin SL, House J, Aitken J, Heath D, House AK, et al. Open access endoscopy in rural and remote Western Australia: does it work?. ANZ J Surg 2002;72:699-703. http://dx.doi.org/10.1046/j.1445-2197.2002.02535.x.
- Hermush V, Daliot D, Weiss A, Brill S, Beloosesky Y. The impact of geriatric consultation on the care of the elders in community clinics. Arch Gerontol Geriatr 2009;49:260-2. http://dx.doi.org/10.1016/j.archger.2008.09.007.
- Pfeiffer PN, Szymanski BR, Zivin K, Post EP, Valenstein M, McCarthy JF, et al. Are primary care mental health services associated with differences in specialty mental health clinic use?. Psychiatr Serv 2011;62:422-5. http://dx.doi.org/10.1176/ps.62.4.pss6204_0422.
- Johnson G, White A, Livingstone R. Do general practices which provide an acupuncture service have low referral rates and prescription costs? A pilot survey. Acupunct Med 2008;26:205-13. http://dx.doi.org/10.1136/aim.26.4.205.
- Tan E, Levell NJ, Garioch JJ. The effect of a dermatology restricted-referral list upon the volume of referrals. Clin Exp Dermatol 2007;32:114-15.
- van Dijk CE, Verheij RA, Hansen J, van d V, Nijpels G, Groenewegen PP, et al. Primary care nurses: effects on secondary care referrals for diabetes. BMC Health Serv Res 2010;10. http://dx.doi.org/10.1186/1472-6963-10-230.
- White M, Bijlani N, Bale R, Burns T. Impact of counsellors in primary care on referrals to secondary mental health services. Psychiatr Bull 2000;24:418-20. http://dx.doi.org/10.1192/pb.24.11.418.
- Simpson S, Corney R, Fitzgerald P. Counselling provision, prescribing and referral rates in a general practice setting. Prim Care Psychiatry 2003;8:115-19. http://dx.doi.org/10.1185/135525703125001857.
- Schillinger D, Bibbins-Domingo K, Vranizan K, Bacchetti P, Luce JM, Bindman AB, et al. Effects of primary care coordination on public hospital patients. J Gen Intern Med 2000;15:329-36. http://dx.doi.org/10.1046/j.1525-1497.2000.07010.x.
- Ferris TG, Chang Y, Blumenthal D, Pearson SD. Leaving gatekeeping behind – effects of opening access to specialists for adults in a health maintenance organisation. N Engl J Med 2001;345:1312-17. http://dx.doi.org/10.1056/NEJMsa010097.
- Ferris TG, Chang Y, Perrin JM, Blumenthal D, Pearson SD. Effects of removing gatekeeping on specialist utilisation by children in a Health Maintenance Organisation. Arch Pediatr Adolesc Med 2002;156:574-9. http://dx.doi.org/10.1001/archpedi.156.6.574.
- Joyce GF, Kapur K, Van Vorst KA, Escarce JJ. Visits to primary care physicians and to specialists under gatekeeper and point-of-service arrangements. Am J Manag Care 2000;6:1189-96.
- McGarry HH, Hegarty K, Johnson C, Gunn J, Blashki G. Managing depression in a changing primary mental healthcare system: comparison of two snapshots of Australian GPs’ treatment and referral patterns. Ment Health Fam Med 2009;6:75-83.
- van Dijk CE, Verheij RA, Spreeuwenberg P, Van den Berg MJ, Groenewegen PP, Braspenning J, et al. Impact of remuneration on guideline adherence: empirical evidence in general practice. Scand J Prim Health Care 2013;31:56-63. http://dx.doi.org/10.3109/02813432.2012.757078.
- Vardy DA, Freud T, Sherf M, Spilberg O, Goldfarb D, Cohen AD, et al. A co-payment for consultant services: primary care physicians’ referral actualisation. J Med Syst 2008;32:37-41. http://dx.doi.org/10.1007/s10916-007-9105-9.
- Iversen T, Luras H. The effect of capitation on GPs’ referral decisions. Health Econ 2000;9:199-210. http://dx.doi.org/10.1002/(SICI)1099-1050(200004)9:3<199::AID-HEC514>3.0.CO;2-2.
- Watson E, Clements A, Lucassen A, Yudkin P, Mackay J, Austoker J. Education improves general practitioner (GP) management of familial breast/ovarian cancer: findings from a cluster randomised controlled trial. J Med Genet 2002;39:779-81. http://dx.doi.org/10.1136/jmg.39.10.779.
- Whiting M. Introducing a GP-led referral gateway. Pulse 2011;71:29-30.
- Maddison P, Jones J, Breslin A, Carton C, Fleur J. Improved access and targeting of musculoskeletal services in northwest Wales: targeted early access to musculoskeletal services (TEAMS) programme. BMJ 2004;329:1325-7. http://dx.doi.org/10.1136/bmj.329.7478.1325.
- Kim C, Williamson DF, Herman WH, Safford MM, Selby JV, Marrero DG, et al. Referral management and the care of patients with diabetes: the Translating Research into Action for Diabetes (TRIAD) study. Am J Manag Care 2004;10:137-43.
- Cox JMS, Steel N, Clark AB, Kumaravel B, Bachmann MO. Do referrral management schemes reduce hospital outpatient attendances?. Br J Gen Pract 2013;63:299-300. http://dx.doi.org/10.3399/bjgp13X668177.
- Ferriter K, Gangopadhyay P, Nilforooshan R. Quality of referrals to old age psychiatry following introduction of the single assessment process. Psychiatr Bull 2006;30. http://dx.doi.org/10.1192/pb.30.12.452.
- Albertson G, Lin CT, Schilling L, Cyran E, Anderson S, Anderson RJ, et al. Impact of a simple intervention to increase primary care provider recognition of patient referral concerns. Am J Manag Care 2002;8:375-81.
- Heaney D, Wyke S, Wilson P, Elton R, Rutledge P. Assessment of impact of information booklets on use of healthcare services: randomised controlled trial. BMJ 2001;322:1218-21. http://dx.doi.org/10.1136/bmj.322.7296.1218.
- Lyon D, Knowles J, Slater B, Kennedy R. Improving the early presentation of cancer symptoms in disadvantaged communities: putting local people in control. Br J Cancer 2009;101:49-54. http://dx.doi.org/10.1038/sj.bjc.6605390.
- Delva F, Marien E, Fonck M, Rainfray M, Demeaux JL, Moreaud P, et al. Factors influencing general practitioners in the referral of elderly cancer patients. BMC Cancer 2011;11. http://dx.doi.org/10.1186/1471-2407-11-5.
- Fucito LMG, Gomes B, Murnion B, Haber P. General practitioners’ diagnostic skills and referral practices in managing patients with drug and alcohol-related health problems: implications for medical training and education programmes. Drug Alcohol Rev 2003;22:417-24. http://dx.doi.org/10.1080/09595230310001613930.
- Scheeres K, Wensing M, Mes C, Bleijenberg G. The impact of informational interventions about cognitive behavioural therapy for chronic fatigue syndrome on GPs referral behaviour. Patient Educ Couns 2007;68:29-32. http://dx.doi.org/10.1016/j.pec.2007.04.002.
- Ringard A. Why do general practitioners abandon the local hospital? An analysis of referral decisions related to elective treatment. Scand J Public Health 2010;38:597-604. http://dx.doi.org/10.1177/1403494810371019.
- Townsley CA, Naidoo K, Pond GR, Melnick W, Straus SE, Siu LL, et al. Are older cancer patients being referred to oncologists? A mail questionnaire of Ontario primary care practitioners to evaluate their referral patterns. J Clin Oncol 2003;21:4627-35. http://dx.doi.org/10.1200/JCO.2003.06.073.
- Zielinski AH, Håkansson A, Jurgutis A, Ovhed I, Halling A. Differences in referral rates to specialised health care from four primary health care models in Klaipeda, Lithuania. BMC Fam Pract 2008;26. http://dx.doi.org/10.1186/1471-2296-9-63.
- Elhayany A, Shvartzman P, Regev S, Reuveni H, Tabenkin H. Variations in referrals to consultants: a study of general practitioners’ characteristics in southern Israel. J Ambulatory Care Manage 2000;23:45-54. http://dx.doi.org/10.1097/00004479-200001000-00005.
- Kvaerner KJ, Helgaker AB. Otitis media referrals – the general practitioner perspective. Int J Pediatr Otorhinolaryngol 2007;71:1219-24. http://dx.doi.org/10.1016/j.ijporl.2007.04.012.
- Naccarella L, Pirkis J, Morley B, Kohn F, Blashki G, Burgess PE-MA, et al. Managing demand for psychological services within an Australian primary mental healthcare initiative. Primary Care Community Psychiatry 2008;13:126-33.
- O’Neill L, Kuder J. Explaining variation in physician practice patterns and their propensities to recommend services. Med Care Res Rev 2005;62:339-57. http://dx.doi.org/10.1177/1077558705275424.
- Swarztrauber K, Vickrey BG, Mittman BS. Physicians’ preferences for specialty involvement in the care of patients with neurological conditions. Med Care 2002;40:1196-209. http://dx.doi.org/10.1097/00005650-200212000-00007.
- Freed GL, Jee S, Stein L, Spera L, Clark SJ. Comparing the self-reported referral and management preferences of pediatricians and family physicians for children with juvenile rheumatoid arthritis. J Rheumatol 2003;30:2700-4.
- Tzaribachev NB, Benseler SM, Tyrrell PN, Meyer A, Kuemmerle-Deschner JB. Predictors of delayed referral to a pediatric rheumatology center. Arthritis Rheum 2009;61:1367-72. http://dx.doi.org/10.1002/art.24671.
- Dodds W, Morgan M, Wolfe C, Raju KS. Implementing the 2-week wait rule for cancer referral in the UK: general practitioners’ views and practices. Eur J Cancer Care 2004;13:82-7. http://dx.doi.org/10.1111/j.1365-2354.2004.00447.x.
- Lambert MV, Bird JM. The assessment and management of adult patients with epilepsy – the role of general practitioners and the specialist services. Seizure 2001;10:341-6. http://dx.doi.org/10.1053/seiz.2001.0520.
- Pomeroy SE, Cant RP. General practitioners’ decision to refer patients to dietitians: insight into the clinical reasoning process. Aust J Prim Health 2010;16:147-53. http://dx.doi.org/10.1071/PY09024.
- Knight L. How do GPs make referral and treatment decisions when patients present with mental health problems?. Counsell Psychol Q 2003;16:195-221. http://dx.doi.org/10.1080/09515070310001610092.
- Wassenaar TR, Eickhoff JC, Jarzemsky DR, Smith SS, Larson ML, Schiller JH. Differences in primary care clinicians’ approach to non-small cell lung cancer patients compared with breast cancer. J Thorac Oncol 2007;2:722-8. http://dx.doi.org/10.1097/JTO.0b013e3180cc2599.
- Lakha SF, Yegneswaran B, Furlan JC, Legnini V, Nicholson K, Mailis-Gagnon A. Referring patients with chronic noncancer pain to pain clinics: survey of Ontario family physicians. Can Fam Physician 2011;57:e106-12.
- Montgomery AJ, McGee HM, Shannon W, Donohoe J. Factors influencing general practitioner referral of patients developing end-stage renal failure: a standardised case-analysis study. BMC Health Serv Res 2006;6. http://dx.doi.org/10.1186/1472-6963-6-114.
- Jørgensen CK, Olesen F. Predictors for referral to physiotherapy from general practice. Scand J Prim Health Care 2001;19:48-53. http://dx.doi.org/10.1080/028134301300034684.
- Rowlands G, Willis S, Singleton A. Referrals and relationships: in-practice referrals meetings in a general practice. Fam Pract 2001;18:399-406. http://dx.doi.org/10.1093/fampra/18.4.399.
- Rushton JB, Bruckman D, Kelleher K. Primary care referral of children with psychosocial problems. Arch Pediatr Adolesc Med 2002;156:592-8. http://dx.doi.org/10.1001/archpedi.156.6.592.
- Angstman KB, Adamson SC, Furst JW, Houston MS, Rohrer JE. Provider satisfaction with virtual specialist consultations in a family medicine department. Health Care Manag (Frederick) 2009;28:14-8. http://dx.doi.org/10.1097/HCM.0b013e318196def8.
- Kisely S, Horton-Hausknecht J, Miller K, Mascall C, Tait A, Wong P, et al. Increased collaboration between primary care and psychiatric services. A survey of general practitioners’ views and referrals. Aust Fam Physician 2002;31:587-9.
- Mitchell L, Macdonald-Wicks L, Capra S. Increasing dietetic referrals: perceptions of general practitioners, practice nurses and dietitians. Nutrition Dietetics 2012;69:32-8. http://dx.doi.org/10.1111/j.1747-0080.2011.01570.x.
- Coulston JE, Williams GL, Stephenson BM. Audit of referral patterns for hernia repair – are general practitioners aware of the changing face of herniorrhaphy?. Ann R Coll Surg Engl 2008;90:140-1. http://dx.doi.org/10.1308/003588408X261573.
- Ramanathan SA, Baratiny G, Stocks NP, Searles AM, Redford RJ. General practitioner referral patterns for women with gynaecological symptoms: a randomised incomplete block study design. Med J Aust 2011;195:602-6. http://dx.doi.org/10.5694/mja10.10867.
- Blundell N, Taylor-Phillips S, Spitzer D, Martin S, Forde I, Clarke A, et al. Elective surgical referral guidelines--background educational material or essential shared decision making tool? A survey of GPs’ in England. BMC Fam Pract 2011;12. http://dx.doi.org/10.1186/1471-2296-12-92.
- Clarke A, Nusila M, Le M. The REFER Project: Realistic Effective Facilitation of Elective Referral for Elective Surgical Assessment 2010. www.netscc.ac.uk/hsdr/files/project/SDO_FR_08–1310–072_V01.pdf (accessed 14 January 2014).
- Kasje WN, Denig P, De Graeff PA, Haaijer-Ruskamp FM. Physicians’ views on joint treatment guidelines for primary and secondary care. Int J Qual Health Care 2004;16:229-36. http://dx.doi.org/10.1093/intqhc/mzh038.
- Baker R, Lecouturier J, Bond S. Explaining variation in GP referral rates for X-rays for back pain. Implement Sci 2006;1. http://dx.doi.org/10.1186/1748-5908-1-15.
- Jiwa M, Gordon M, Arnet H, Ee H, Bulsara M, Colwell B, et al. Referring patients to specialists: a structured vignette survey of Australian and British GPs. BMC Fam Pract 2008;9. http://dx.doi.org/10.1186/1471-2296-9-2.
- Ruston A. Risk, anxiety and defensive action: general practitioner’s referral decisions for women presenting with breast problems. Health Risk Soc 2004;6. http://dx.doi.org/10.1080/1369857042000193066.
- Watson E, Austoker J, Lucassen. A study of GP referrals to a family cancer clinic for breast/ovarian cancer. Fam Pract 2001;18:131-4. http://dx.doi.org/10.1093/fampra/18.2.131.
- Bederman SS, McIsaac WJ, Coyte PC, Kreder HJ, Mahomed NN, Wright JG, et al. Referral practices for spinal surgery are poorly predicted by clinical guidelines and opinions of primary care physicians. Med Care 2010;48:852-8. http://dx.doi.org/10.1097/MLR.0b013e3181e3588b.
- Belgamwar B, Bates C, Goes C, Talyor N. Appropriateness of GP referrals of patients with anxiety and depression. Progress Neurol Psychiatry 2011;15:27-9. http://dx.doi.org/10.1002/pnp.188.
- Tucker JF, Farmer J, Stimpson P. Guidelines and management of mild hypertensive conditions in pregnancy in rural general practices in Scotland: issues of appropriateness and access. Qual Saf Health Care 2003;12:286-90. http://dx.doi.org/10.1136/qhc.12.4.286.
- Trude S, Stoddard JJ. Referral gridlock: primary care physicians and mental health services. J Gen Intern Med 2003;18:442-9. http://dx.doi.org/10.1046/j.1525-1497.2003.30216.x.
- Morsi E, Lindenauer PK, Rothberg MB. Primary care physicians’ use of publicly reported quality data in hospital referral decisions. J Hosp Med 2012;7:370-5. http://dx.doi.org/10.1002/jhm.1931.
- Sigel PL, Leiper R. GP views of their management and referral of psychological problems: a qualitative study. Psychol Psychother 2004;77:279-95. http://dx.doi.org/10.1348/1476083041839394.
- Philichi L, Yuwono M. Primary care: constipation and encopresis treatment strategies and reasons to refer. Gastroenterol Nurs 2010;33:363-6. http://dx.doi.org/10.1097/SGA.0b013e3181f35020.
- Steele M, Zayed R, Davidson B, Stretch N, Nadeau L, Fleisher W, et al. Referral patterns and training needs in psychiatry among primary care physicians in Canadian rural/remote areas. J Can Acad Child Adolesc Psychiatry 2012;21:111-23.
- Nandy S, Chalmers-Watson C, Gantley M, Underwood M. Referral for minor mental illness: a qualitative study. Br J Gen Pract 2001;51:461-5.
- Moore JL, McAuley JW, Mott D, Reeves AL, Bussa B. Referral characteristics of primary care physicians for seizure patients. Epilepsia 2000;41:744-8. http://dx.doi.org/10.1111/j.1528-1157.2000.tb00237.x.
- Olson R, Lengoc S, Tyldesley S, French J, McGahan C, Soo J. Relationships between family physicians’ referral for palliative radiotherapy, knowledge of indications for radiotherapy, and prior training: a survey of rural and urban family physicians. Radiation Oncol 2012;7. http://dx.doi.org/10.1186/1748-717X-7-73.
- Calnan MP, Payne S, Kemple T, Rossdale M, Ingram J. A qualitative study exploring variations in GPs’ out-of-hours referrals to hospital. Br J Gen Pract 2007;57:706-13.
- Anthony JS, Baik SY, Bowers BJ, Tidjani B, Jacobson CJ, Susman J, et al. Conditions that influence a primary care clinician’s decision to refer patients for depression care. Rehabil Nurs 2010;35:113-22. http://dx.doi.org/10.1002/j.2048-7940.2010.tb00286.x.
- Bruyninckx R, Van den Bruel A, Aertgeerts B, Van CV, Buntinx F. Why does the general practitioner refer patients with chest pain not-urgently to the specialist or urgently to the emergency department? Influence of the certainty of the initial diagnosis. Acta Cardiol 2009;64:259-65. http://dx.doi.org/10.2143/AC.64.2.2036147.
- Morgan M, Jenkins L, Ridsdale L. Patient pressure for referral for headache: a qualitative study of GPs’ referral behaviour. Br J Gen Pract 2007;57:29-35.
- Rosemann T, Wensing M, Rueter G, Szecsenyi J. Referrals from general practice to consultants in Germany: if the GP is the initiator, patients’ experiences are more positive. BMC Health Serv Res 2005;6. http://dx.doi.org/10.1186/1472-6963-6-5.
- van der Weijden T, van Bokhoven M, Dinant GJ, van Hasselt CM, Grol RPTM. Understanding laboratory testing in diagnostic uncertainty: a qualitative study in general practice. Br J Gen Pract 2002;52:974-80.
- Wilkes S, Murdoch A, Steen N, Wilsdon J, Rubin G. Open Access Tubal aSsessment for the initial management of infertility in general practice (the OATS trial): a pragmatic cluster randomised controlled trial. Br J Gen Pract 2009;59:329-35. http://dx.doi.org/10.3399/bjgp09X420590.
- Ahluwalia SC, Fried TR. Physician factors associated with outpatient palliative care referral. Palliat Med 2009;23:608-15. http://dx.doi.org/10.1177/0269216309106315.
- Pryor AMR, Knowles AA. The relationship between general practitioners’ characteristics and the extent to which they refer clients to psychologists. Aust Psychol 2001;36:227-31. http://dx.doi.org/10.1080/00050060108259659.
- Franks P, Williams GC, Zwanziger J, Mooney C, Sorbero M. Why do physicians vary so widely in their referral rates?. J Gen Intern Med 2000;15:163-8. http://dx.doi.org/10.1046/j.1525-1497.2000.04079.x.
- Espeland A, Baerheim A. Factors affecting general practitioners’ decisions about plain radiography for back pain: implications for classification of guideline barriers – a qualitative study. BMC Health Serv Res 2003;3. http://dx.doi.org/10.1186/1472-6963-3-8.
- Cornford CS, Harley J, Oswald N. The ‘2-week rule’ for suspected breast carcinoma: a qualitative study of the views of patients and professionals. Br J Gen Pract 2004;54:584-8.
- Forrest CB, Nutting P, Werner JJ, Starfield B, von Schrader S, Rohde C. Managed health plan effects on the specialty referral process – results from the ambulatory sentinel practice network referral study. Med Care 2003;41:242-53. http://dx.doi.org/10.1097/01.MLR.0000044903.91168.B6.
- Green H, Johnston O, Cabrini S, Fornai G, Kendrick T. General practitioner attitudes towards referral of eating-disordered patients: a vignette study based on the theory of planned behaviour. Ment Health Fam Med 2008;5:213-18.
- Young CE, Mutch AJ, Boyle FM, Dean JH. Investigating referral pathways from primary care to consumer health organisations. Aust J Prim Health 2010;16:260-7. http://dx.doi.org/10.1071/PY09081.
- Barnett ML, Keating NL, Christakis NA, O’Malley AJ, Landon BE. Reasons for choice of referral physician among primary care and specialist physicians. J Gen Intern Med 2012;27:506-12. http://dx.doi.org/10.1007/s11606-011-1861-z.
- Forrest CB, Nutting PA, Starfield B, von SS. Family physicians’ referral decisions: results from the ASPN referral study. J Fam Pract 2002;51:215-22.
- Dale N, Godsman J. Factors influencing general practitioner referrals to a tertiary paediatric neurodisability service. Br J Gen Pract 2000;50:131-2.
- McKenna CF, Farber NJ, Eschbach KS, Collier VU. Primary care practitioners’ understanding of psychiatric practice: effects on intention to refer. Arch Phys Med Rehabil 2005;86:881-8. http://dx.doi.org/10.1016/j.apmr.2004.09.014.
- Beel JV, Gringart E, Edwards MEE-MA, Beel JV. Western Australian general practitioners’ views on psychologists and the determinants of patient referral: an exploratory study. Families Systems Health 2008;26:250-66. http://dx.doi.org/10.1037/a0012913.
- Wakefield P, Randall G, Fiala JM. Competing for referrals for cardiac diagnostic tests: what do family physicians really want?. JMIRS 2012;43:155-60. http://dx.doi.org/10.1016/j.jmir.2012.04.001.
- Clemence ML, Seamark DA. GP referral for physiotherapy to musculoskeletal conditions – a qualitative study. Fam Pract 2003;20:578-82. http://dx.doi.org/10.1093/fampra/cmg515.
- Chew-Graham C, Slade M, Montâna C, Stewart M, Gask L. Loss of doctor-to-doctor communication: lessons from the reconfiguration of mental health services in England. J Health Serv Res Policy 2008;13:6-12. http://dx.doi.org/10.1258/jhsrp.2007.006053.
- Dagneaux I, Gillard I, De Lepeleire J. Care of elderly people by the general practitioner and the geriatrician in Belgium: a qualitative study of their relationship. J Multidiscip Healthc 2012;5:17-25. http://dx.doi.org/10.2147/JMDH.S27617.
- Harlan G, Srivastava R, Harrison L, McBride G, Maloney C. Paediatric hospitalists and primary care providers: a communication needs assessment. J Hosp Med 2009;4:187-93. http://dx.doi.org/10.1002/jhm.456.
- Allareddy V, Greene BR, Smith M, Haas M, Liao J. Facilitators and barriers to improving interprofessional referral relationships between primary care physicians and chiropractors. J Ambulatory Care Manage 2007;30:347-54. http://dx.doi.org/10.1097/01.JAC.0000290404.96907.e3.
- Taggarshe D, Haldipur N, Singh S. Generic outpatient referrals: why don’t GPs make them?. J Public Health (Oxf) 2006;28:218-20. http://dx.doi.org/10.1093/pubmed/fdl027.
- Samant RS, Fitzgibbon E, Meng J, Graham ID. Barriers to palliative radiotherapy referral: a Canadian perspective. Acta Oncol 2007;46:659-63. http://dx.doi.org/10.1080/02841860600979005.
- Xu T, Rohrer J, Borders T. The impact of managed care and practice size on primary care physicians’ perceived ability to refer. J Health Serv Res Policy 2002;7:143-50. http://dx.doi.org/10.1258/135581902760082445.
- Massey J. Referral criteria to improve access for patients with incontinence/pelvic floor dysfunction from primary care to a physiotherapy continence service. J Assoc Chartered Physiother Womens Health 2004;95:29-35.
- Berendsen AJ, Benneker WH, Meyboom-de JB, Klazinga NS, Schuling J. Motives and preferences of general practitioners for new collaboration models with medical specialists: a qualitative study. BMC Health Serv Res 2007;7. http://dx.doi.org/10.1186/1472-6963-7-4.
- Kinchen KS, Cooper LA, Levine D, Wang NY, Powe NR. Referral of patients to specialists: factors affecting choice of specialist by primary care physicians. Ann Fam Med 2004;2:245-52. http://dx.doi.org/10.1370/afm.68.
- Thorsen O, Hartveit M, Baerheim A. General practitioners’ reflections on referring: an asymmetric or non-dialogical process?. Scand J Prim Health Care 2012;30:241-6. http://dx.doi.org/10.3109/02813432.2012.711190.
- Greer RC, Powe NR, Jaar BG, Troll MU, Boulware LE. Effect of primary care physicians’ use of estimated glomerular filtration rate on the timing of their subspecialty referral decisions. BMC Nephrol 2011;12. http://dx.doi.org/10.1186/1471-2369-12-1.
- Jiwa M, Arnet H, Bulsara M, Ee HC, Harwood A. What is the importance of the referral letter in the patient journey? A pilot survey in Western Australia. Qual Prim Care 2009;17:31-6.
- Harvey NSG, Gill PV, Kimlim J. A survey of general practitioners’ preferences, when referring to mental health services, and the implications for electronic outpatient booking. Prim Care Community Psychiatry 2005;10:51-6.
- O’Byrne L, Darlow C, Roberts N, Wilson G, Partridge MR. Smoothing the passage of patients from primary care to specialist respiratory opinion. Prim Care Respir J 2010;19:248-53. http://dx.doi.org/10.4104/pcrj.2010.00028.
- Carlsen B, Aakvik A, Norheim OF. Variation in practice: a questionnaire survey of how congruence in attitudes between doctors and patients influences referral decisions. Med Decis Making 2008;28:262-8. http://dx.doi.org/10.1177/0272989X07311751.
- Johnson C, Paul C, Girgis A, Adams J, Currow D. Australian general practitioners’ and oncology specialists’ perceptions of barriers and facilitators of access to specialist palliative care services. J Palliat Med 2011;14:429-35. http://dx.doi.org/10.1089/jpm.2010.0259.
- Forrest CB, Shadmi E, Nutting PA, Starfield B. Specialty referral completion among primary care patients: results from the ASPN Referral Study. Ann Fam Med 2007;5:361-7. http://dx.doi.org/10.1370/afm.703.
- Ramchandani M, Mirza S, Sharma A, Kirkby G. Pooled cataract waiting lists: views of hospital consultants, general practitioners and patients. J R Soc Med 2003;95:598-600. http://dx.doi.org/10.1258/jrsm.95.12.598.
- Hyman I, Singh M, Ahmad F, Austin L, Meana M, George U, et al. The role of physicians in mammography referral for older Caribbean women in Canada. Medscape Womens Health 2001;6.
- Stavrou S, Cape J, Barker C. Decisions about referrals for psychological therapies: a matched-patient qualitative study. Br J Gen Pract 2009;59. http://dx.doi.org/10.3399/bjgp09X454089.
- Vulto A, van Bommel M, Poortmans P, Lybeert M, Louwman M, Baart R, et al. General practitioners and referral for palliative radiotherapy: a population-based survey. Radiother Oncol 2009;91:267-70. http://dx.doi.org/10.1016/j.radonc.2009.01.009.
- Little P, Dorward M, Warner G, Stephens K, Senior J, Moore M. Importance of patient pressure and perceived pressure and perceived medical need for investigations, referral, and prescribing in primary care: nested observational study. BMJ 2004;328:444-46. http://dx.doi.org/10.1136/bmj.38013.644086.7C.
- Rosen R, Florin D, Hutt R. An Anatomy of GP Referral Decisions: A Qualitative Study of GPs’ Views on Their Role in Supporting Patient Care. London: The King’s Fund; 2007.
- Blundell N, Clarke A, Mays N. Interpretations of referral appropriateness by senior health managers in five PCT areas in England: a qualitative investigation. Qual Saf Health Care 2010;19:182-6. http://dx.doi.org/10.1136/qshc.2007.025684.
- Glozier N, Prince M. Psychological determinants of primary care requests for urgent outpatient appointments in elective referrals. J Eval Clin Pract 2007;13:236-41. http://dx.doi.org/10.1111/j.1365-2753.2006.00685.x.
- Musila N, Underwood M, McCaskie AW, Black N, Clarke A, van der Meulen JH, et al. Referral recommendations for osteoarthritis of the knee incorporating patients’ preferences. Fam Pract 2011;28:68-74. http://dx.doi.org/10.1093/fampra/cmq066.
- Davies E, van der Molen B, Cranston A, Davies E, van der Molen B, Cranston A. Using clinical audit, qualitative data from patients and feedback from general practitioners to decrease delay in the referral of suspected colorectal cancer. J Eval Clin Pract 2007;13:310-17. http://dx.doi.org/10.1111/j.1365-2753.2006.00820.x.
- Edwards M. BRIDGE Study Group . Responses of primary health care professionals to UK national guidelines on the management and referral of women with breast conditions. J Eval Clin Pract 2002;8:319-25. http://dx.doi.org/10.1046/j.1365-2753.2002.00335.x.
- Brien S, Howells E, Leydon G, Brien Ssacu. Why GPs refer patients to complementary medicine via the NHS: a qualitative exploration. Prim Health Care Res Development 2008;9:205-15. http://dx.doi.org/10.1017/S1463423608000789.
- Bekkelund SI, Salvesen R. Are headache patients who initiate their referral to a neurologist satisfied with the consultation? A population study of 927 patients – the North Norway Headace Study (NNHS). Fam Pract 2001;18:524-7. http://dx.doi.org/10.1093/fampra/18.5.524.
- Lewis CL, Wickstrom GC, Kolar MM, Keyserling TC, Bognar BA, Dupre CT, et al. Patient preferences for care by general internists and specialists in the ambulatory setting. J Gen Intern Med 2000;15:75-83. http://dx.doi.org/10.1046/j.1525-1497.2000.05089.x.
- Gross R, Tabenkin H, Brammli-Greenberg S. Who needs a gatekeeper? Patients’ views of the role of the primary care physician. Fam Pract 2000;17:222-9. http://dx.doi.org/10.1093/fampra/17.3.222.
- Berendsen AJ, de Jong GM, Schuling J, Bosveld HE, de Waal MW, Mitchell GK, et al. Patient’s need for choice and information across the interface between primary and secondary care: a survey. Patient Educ Couns 2010;79:100-5. http://dx.doi.org/10.1016/j.pec.2009.07.032.
- Shadd J, Ryan BL, Maddocks H, Thind A. Patterns of referral in a Canadian primary care electronic health record database: retrospective cross-sectional analysis. Inform Prim Care 2011;19:217-23.
- Bertakis KD, Callahan EJ, Azari R, Robbins JA, Bertakis KD. Predictors of patient referrals by primary care residents to specialty care clinics. Fam Med 2001;33:203-9.
- Dearman SP, Waheed W. Management strategies in geriatric depression by primary care physicians and factors associated with the use of psychiatric services: a naturalistic study. Aging Ment Health 2006;10:521-4. http://dx.doi.org/10.1080/13607860600637984.
- Ridsdale LC, Clark LV, Dowson AJ, Goldstein LH, Jenkins L, McCrone P, et al. How do patients referred to neurologists for headache differ from those managed in primary care?. Br J Gen Pract 2007;57:388-95.
- Cohen SM, Jaewhan K, Nelson R, Courey M. Factors influencing referral of patients with voice disorders from primary care to otolaryngology. Laryngoscope 2014;124:214-20. http://dx.doi.org/10.1002/lary.24280.
- Harris M, Jayasinghe U, Chan B. Patient and practice characteristics predict the frequency of general practice multidisciplinary referrals of patients with chronic diseases: a multilevel study. Health Policy 2011;101:140-5. http://dx.doi.org/10.1016/j.healthpol.2010.10.019.
- Vinker S, Kaiserman I, Karni A, Kitai E, Kasinetz LM, Elhayany A, et al. Urgent referrals to a specialist by family physicians – is the ‘urgency’ real: a prospective study. Eur J Gen Pract 2007;13:37-9. http://dx.doi.org/10.1080/13814780600881003.
- Albertson GA, Lin CT, Kutner J, Schilling LM, Anderson SN, Anderson RJ. Recognition of patient referral desires in an academic managed care plan. J Gen Intern Med 2000;15:242-7. http://dx.doi.org/10.1111/j.1525-1497.2000.02208.x.
- Johnson CE, Danhauer JL, Koch LL, Celani KE, Lopez IP, Williams VA, et al. Hearing and balance screening and referrals for Medicare patients: a national survey of primary care physicians. J Am Acad Audiol 2008;19:171-90. http://dx.doi.org/10.3766/jaaa.19.2.7.
- Hugo P, Kendrick T, Reid F, Lacey H. GP referral to an eating disorder service: why the wide variation?. Br J Gen Pract 2000;50:380-3.
- Bowling A, Harries C, Forrest D, Harvey N. Variations in cardiac interventions: doctors’ practices and views. Fam Pract 2006;23:427-36. http://dx.doi.org/10.1093/fampra/cmi125.
- Balduf LM, Farrell TM. Attitudes, beliefs, and referral patterns of PCPs to bariatric surgeons. J Surg Res 2008;144:49-58. http://dx.doi.org/10.1016/j.jss.2007.01.038.
- Chan BT, Austin PC. Patient, physician, and community factors affecting referrals to specialists in Ontario, Canada: a population-based, multi-level modelling approach. Med Care 2003;41:500-11. http://dx.doi.org/10.1097/01.MLR.0000053971.89707.97.
- Bolaños-Carmona V, Ocaña-Riola R, Prados-Torres A, Gutiérrez-Cuadra P. Variations in health services utilisation by primary care patients. Health Serv Manage Res 2002;15:116-25. http://dx.doi.org/10.1258/0951484021912888.
- Ache KA, Shannon RP, Heckman MG, Diehl NN, Willis FB, Ache KA, et al. A preliminary study comparing attitudes towards hospice referral between African American and white American primary care physicians. J Palliat Med 2011;14:542-7. http://dx.doi.org/10.1089/jpm.2010.0426.
- Navaneethan SD, Kandula P, Jeevanantham V, Nally JV, Liebman SE, Navaneethan SD, et al. Referral patterns of primary care physicians for chronic kidney disease in general population and geriatric patients. Clin Nephrol 2010;73:260-7. http://dx.doi.org/10.5414/CNP73260.
- Cooper SM, Wojnarowska F. The influence of the sex of the general practitioner on referral to a vulval clinic. Journal Obstet Gynaecol 2001;21:179-80. http://dx.doi.org/10.1080/01443610020026155.
- Coyle E, Hanley K, Sheerin J. Who goes where? A prospective study of referral patterns within a newly established primary care team. Ir J Med Sci 2011;180:845-9. http://dx.doi.org/10.1007/s11845-011-0724-2.
- Gruen RL, Knox S, Britt H. Where there is no surgeon: the effect of specialist proximity on general practitioners’ referral rates. Med J Aust 2002;177:111-15.
- Feeney E, Noble H, Waller G. Referral patterns to a specialist eating disorder service: the impact of the referrer’s gender. Eur Eat Disord Rev 2007;15:45-9. http://dx.doi.org/10.1002/erv.759.
- Forrest CB, Nutting PA, von SS, Rohde C, Starfield B. Primary care physician specialty referral decision making: patient, physician, and health care system determinants. Med Decis Making 2006;26:76-85. http://dx.doi.org/10.1177/0272989X05284110.
- Chauhan M, Bankart MJ, Labeit A, Baker R. Characteristics of general practices associated with numbers of elective admissions. J Public Health 2012;34:584-90. http://dx.doi.org/10.1093/pubmed/fds024.
- Sullivan CO, Omar RZ, Ambler G, Majeed A. Case-mix and variation in specialist referrals in general practice. Br J Gen Pract 2005;55:529-33.
- Chen FM, Fryer GE, Norris TE. Effects of comorbidity and clustering upon referrals in primary care. J Am Board Fam Pract 2005;18:449-52. http://dx.doi.org/10.3122/jabfm.18.6.449.
- McBride D, Hardoon S, Walters K, Gilmour S, Raine R. Explaining variation in referral from primary to secondary care: cohort study. BMJ 2010;341. http://dx.doi.org/10.1136/bmj.c6267.
- Robinson PC, Taylor WJ. Time to treatment in rheumatoid arthritis: factors associated with time to treatment initiation and urgent triage assessment of general practitioner referrals. J Clin Rheumatol 2010;16:267-73. http://dx.doi.org/10.1097/RHU.0b013e3181eeb499.
- Johnson CE, Girgis A, Paul CL, Currow DC. Palliative care referral practices and perceptions: the divide between metropolitan and non-metropolitan general practitioners. Palliat Support Care 2011;9:181-9. http://dx.doi.org/10.1017/S1478951511000058.
- Guevara JP, Greenbaum PE, Shera D, Bauer L, Schwarz DF. Survey of mental health consultation and referral among primary care pediatricians. Acad Pediatr 2009;9:123-7. http://dx.doi.org/10.1016/j.acap.2008.12.008.
- Bowling A, Redfern J. The process of outpatient referral and care: the experiences and views of patients, their general practitioners, and specialists. Br J Gen Pract 2000;50:116-20.
- Grace SL, Grewal K, Stewart DE. Factors affecting cardiac rehabilitation referral by physician specialty. J Cardiopulm Rehab Prev 2008;28:248-52. http://dx.doi.org/10.1097/01.HCR.0000327182.13875.07.
- Holley CD, Lee PP. Primary care provider views of the current referral-to-eye-care process: focus group results. Invest Ophthalmol Vis Sci 2010;51:1866-72. http://dx.doi.org/10.1167/iovs.09-4512.
- Kier A, George M, McCarthy P. Referral patterns to spinal manipulative therapy by Welsh general practitioners. Clinical Chiropractic 2012;15:91-2. http://dx.doi.org/10.1016/j.clch.2012.06.015.
- Todman JP, Law J, MacDougall A. Attitudes of GPs towards Older Adults Psychology Services in the Scottish Highlands. Rural Remote Health 2011;11.
- Soomro GM, Burns T, Majeed A. Socio-economic deprivation and psychiatric referral and admission rates: an ecological study in one London borough. Psychiatr Bull 2002;26. http://dx.doi.org/10.1192/pb.26.5.175.
- Mulvaney C, Coupland C, Wilson A, Hammersley V, Dyas J, Carlisle R, et al. Does increased use of private health care reduce the demand for NHS care? A prospective survey of general practice referrals. J Public Health (Oxf) 2005;27:182-8. http://dx.doi.org/10.1093/pubmed/fdi013.
- Soerensen TH, Olsen KR, Vedsted P. Association between general practice referral rates and patients’ socioeconomic status and access to specialised health care: a population-based nationwide study. Health Policy 2009;92:180-6. http://dx.doi.org/10.1016/j.healthpol.2009.03.011.
- Malcolm JC, Liddy C, Rowan M, Maranger J, Keely E, Harrison C, et al. Transition of patients with type 2 diabetes from specialist to primary care: a survey of primary care physicians on the usefulness of tools for transition. Can J Diabetes 2008;32:37-45. http://dx.doi.org/10.1016/S1499-2671(08)21009-9.
- Walders N, Childs GE, Comer D, Kelleher KJ, Drotar D, Walders N, et al. Barriers to mental health referral from pediatric primary care settings. Am J Manag Care 2003;9:677-83.
- Burns T. Socio-economic deprivation and psychiatric referral and admission rates: an ecological study in one London borough. Psychiatr Bull 2002;26:175-8. http://dx.doi.org/10.1192/pb.26.5.175.
- Love T, Crampton P, Salmond C, Dowell A. Patterns of medical practice variation: variability in referral for back pain by New Zealand general practitioners. N Z Med J 2005;118.
- Ashworth M, Clement S, Sandhu J, Farley N, Ramsay R, Davies T, et al. Psychiatric referral rates and the influence of on-site mental health workers in general practice. Br J Gen Pract 2002;52:39-41.
- Boulware LE, Troll MU, Jaar BG, Myers DI, Powe NR. Identification and referral of patients with progressive CKD: a national study. Am J Kidney Dis 2006;48:192-204. http://dx.doi.org/10.1053/j.ajkd.2006.04.073.
- Chung K, Yang D, Lee JH. Determinants of primary care physicians’ referral pattern: a structural equation model approach. Int J Public Pol 2010;5:259-71. http://dx.doi.org/10.1504/IJPP.2010.030607.
- Greenaway R, Fortune L. General practitioners’ views of psychology services: a comparison of general practitioners who refer to onsite and offsite services. Prim Care Ment Health 2006;4:245-54.
- Franz CE, Barker JC, Kim K, Flores Y, Jenkins C, Kravitz RL, et al. When help becomes a hindrance: mental health referral systems as barriers to care for primary care physicians treating patients with Alzheimer’s disease. Am J Geriatr Psychiatry 2010;18:576-85. http://dx.doi.org/10.1097/JGP.0b013e3181a76df7.
- Alexander C, Fraser J. General practitioners’ management of patients with mental health conditions: the views of general practitioners working in rural north-western New South Wales. Aust J Rural Health 2008;16:363-9. http://dx.doi.org/10.1111/j.1440-1584.2008.01017.x.
- Patton M. Qualitative Research and Evaluation Methods. Thousand Oaks, CA: Sage; 2002.
- Anderson R, Crabtree B, Steele D, McDaniel R. Case study research: the view from complexity science. Qual Health Res 2005;15:669-85. http://dx.doi.org/10.1177/1049732305275208.
- Capra F. The Web of Life. New York, NY: Anchor-Doubleday; 1996.
- Alvarez MP, Agra Y. Systematic review of educational interventions in palliative care for primary care physicians. Palliat Med 2006;20:673-83. http://dx.doi.org/10.1177/0269216306071794.
- Bazian Ltd . Specialist outreach into primary care: is it better than standard care?. Evid-Based Healthc Public Health 2005;9:294-301.
- Bower P, Sibbald B. Systematic review of the effect of on-site mental health professionals on the clinical behaviour of general practitioners. BMJ 2000;320:614-17. http://dx.doi.org/10.1136/bmj.320.7235.614.
- Bower P, Sibbald B. Do consultation-liaison services change the behaviour of primary care providers? A review. General Hospital Psychiatry 2000;22:84-96. http://dx.doi.org/10.1016/S0163-8343(00)00058-X.
- Brocklehurst PR, Baker SR, Speight PM. Primary care clinicians and the detection and referral of potentially malignant disorders in the mouth: a summary of the current evidence. Prim Dent Care 2010;17:65-71. http://dx.doi.org/10.1308/135576110791013749.
- Evaluation of Referral Management Pilots in Wales. NHS Wales and National Leadership and Innovation Agency for Healthcare; 2006.
- Referral Management Pilots in Wales – Follow Up Review. National Leadership and Innovation Agency for Healthcare; 2007.
- Clarke A, Blundell N, Forde I, Musila N, Spitzer D, Naqvi S, et al. Can guidelines improve referral to elective surgical specialties for adults: a systematic review. Qual Saf Health Care 2008;19:187-94. http://dx.doi.org/10.1136/qshc.2008.029918.
- Delva F, Soubeyran P, Rainfray M, Mathoulin-Pelissier S. Referral of elderly cancer patients to specialists: action proposals for general practitioners. Cancer Treatment Reviews 2012;38:935-41. http://dx.doi.org/10.1016/j.ctrv.2012.03.010.
- Foot C, Naylor C, Imison C. The Quality of GP Diagnosis and Referral. London: The King’s Fund; 2010.
- Forrest CB. Primary care gatekeeping and referrals: effective filter of failed experiment?. BMJ 2003;326:692-5. http://dx.doi.org/10.1136/bmj.326.7391.692.
- Gruen RL, Weeramanthri TS, Knight SS, Bailie RS. Specialist outreach clinics in primary care and rural hospital settings. Cochrane Database Syst Rev 2003;4. http://dx.doi.org/10.1002/14651858.CD003798.pub2.
- Harkness EF, Bower PJ. On site mental health workers delivering psychological therapy and psychosocial interventions to patients in primary care: effects on the professional practice of primary care providers. Cochrane Database Syst Rev 2009;1. http://dx.doi.org/10.1002/14651858.CD000532.pub2.
- Herrington P, Baker R, Gibson SL, Golden S. GP referrals for counselling: a review and model. J Interprof Care 2003;17:263-71. http://dx.doi.org/10.1080/1356182031000122889.
- Jiwa M, Dadich A. Referral letter content: can it affect patient outcomes?. Br J Health Care Manage 2013;19:140-7. http://dx.doi.org/10.12968/bjhc.2013.19.3.140.
- Lin CY. Improving care coordination in the specialty referral process between primary and specialty care. N C Med J 2012;73:61-2.
- Martin A, Macleod C, Naqui SAR. Effectiveness and Cost-Effectiveness of Targeted Interventions to Reduce Unnecessary Referrals and Improve the Quality of Referrals from Primary Care to Secondary Care. NHS Evidence Adoption Centre East of England; 2010.
- Mead N, Bower P. Patient centred consultations and outcomes in primary care: a review of the literature. Patient Educ Couns 2002;48:51-6. http://dx.doi.org/10.1016/S0738-3991(02)00099-X.
- Mehrota A, Forrest CB, Lin CY. Dropping the baton: specialty referrals in the United States. Millbank Q 2011;89:39-68. http://dx.doi.org/10.1111/j.1468-0009.2011.00619.x.
- Navaneethan SD, Aloudat S, Singh S. A systematic review of patient and health system characteristics associated with late referral in chronic kidney disease. BMC Nephrology 2008;9. http://dx.doi.org/10.1186/1471-2369-9-3.
- O’Donnell CA. Variation in GP referral rates: what can we learn from the literature?. Fam Pract 2000;17:462-71. http://dx.doi.org/10.1093/fampra/17.6.462.
- Piterman L, Koritsas S. Part II General practitioner-specialist referral process. Intern Med J 2005;35:491-6. http://dx.doi.org/10.1111/j.1445-5994.2005.00860.x.
- Powell J. Systematic review of outreach clinics in primary care in the UK. J Health Serv Res Policy 2007;7:177-83. http://dx.doi.org/10.1258/135581902760082490.
- Qureshi NA, van der Molen HT, Schmidt HG, Al-Habeeb TA, Magzoub MEM. Criteria for a good referral system for psychiatric patients: the view from Saudi Arabia. Eastern Mediterr Health J 2009;15:1580-95.
- Roland M, McDonald R, Sibbald B, Boyd A, Fotaki M, Gravelle H, et al. Outpatient Services and Primary Care: A Scoping Review of Research into Strategies for Improving Outpatient Effectiveness and Efficiency. National Primary Care Research and Development Centre, and Centre for Public Policy and Management of the University of Manchester; 2006.
Appendix 1 Extraction tables
Intervention papers
| Adams 201233 Country: USA Study design: Before-and-after Data collection method: Audience response system Aim: To describe the development and implementation process and assess the effect on self-reported clinical practice changes of a multidisciplinary, collaborative, interactive CME/continuing education (CE) programme on COPD Detail of participants (number, any reported demographics): 351 participants |
Method: Multidisciplinary subject matter experts and education specialists used a systematic instructional design approach and collaborated with the American College of Chest Physicians and American Academy of Nurse Practitioners to develop, deliver and reproduce a 1-day interactive COPD CME/CE program for 351 primary-care clinicians in 20 US cities from 23 September 2009 to 13 November 2010 Control: NA Length of follow-up: 3- to 6-month follow-up survey Response and/or attrition rate: 132 of 271 participants (48.7%) Context (from what/who to what/who): GP referral for COPD |
Outcome measures: Self-confidence Knowledge and comprehension |
Main results: Clinician self-confidence improved after the course in all areas measured. In addition, clinician knowledge/comprehension significantly improved [mean (SD) pre-test percentage correct, 77.1% (16.4); 95% CI 76.2% to 78.9%; and mean (SD) post-test percentage correct, 94.7% (8.7%); 95% CI 94.2% to 95.2%; p < 0.001], with an absolute percentage change of 17.6% (13.2%) Of the five knowledge (recall) questions, the mean (SD) improvement in pre-test vs. post-test scores was 14% (5.0%) (95% CI 9.6% to 18.4%), from 83.1% to 97.1% (p < 0.001). The mean improvement in the five comprehension/application questions was 2.7% 17.5% (95% CI 7.4% to 38.0%), from 68.8% to 91.5% (p < 0.001) Of the follow-up survey respondents, 92 of 132 (69.7%) reported completely implementing at least one clinical practice change, and only 8 of 132 (6.1%) reported inability to make any clinical practice change after the programme |
Reported associations between elements for logic model: A carefully designed, interactive, flexible, dynamic, and reproducible COPD CME/CE programme tailored to clinicians’ needs that involves diverse instructional strategies and media can have short-term and long-term improvements in clinician self-confidence, knowledge/comprehension, and clinical practice |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Akbari 2012110 Country: Canada Study design: Before-and-after Data collection method Aim: To show whether or not automatic reporting of the eGFR, along with an ad hoc educational component for primary care physicians, would increase the number of appropriate referrals Detail of participants (number, any reported demographics): All referrals to nephrologists received at the centre during the year before and the year after automatic reporting of the eGFR was introduced were included. The area served by the Champlain Local Health Integration Network has a population of 1,176,600, of whom 12.5% are 65 years or older, 17% are immigrants and 13% are from visible minorities |
Method: Automatic reporting of eGFR to inform referral decision Concurrent with the introduction of automatic reporting of the eGFR, the nephrology service mailed an algorithm to all primary care physicians in the Champlain Local Health Integration Network. This algorithm explained the interpretation of the eGFR and appropriate parameters for referrals to nephrology, based on the value. In addition, ad hoc educational sessions (lectures and workshops) were provided to the primary care physicians to discuss interpretation of the eGFR results and parameters for referral to nephrology Control: None Length of follow-up: 1 year Response and/or attrition rate: NR Context (from what/who to what/who): GP to nephrologist |
Outcome measures: Number and appropriateness of referrals to nephrologists |
Main results: A total of 2672 patients were included in the study. In the year after automatic reporting began, the number of referrals from primary care physicians increased by 80.6% (95% CI 74.8% to 86.9%) The number of appropriate referrals increased by 43.2% (95% CI 38.0% to 48.2%) There was no significant change in the proportion of appropriate referrals between the two periods (−2.8%, 95% CI −26.4% to 43.4%) In the year after automatic reporting of the eGFR was introduced, the total number of referrals increased significantly among patients 80 years and older (percentage-point change 8.0; p < 0.001) and among women (percentage-point change 12.6; p < 0.001) |
Reported associations between elements for logic model: The total number of referrals increased after automatic reporting of the eGFR began, especially among women and elderly people The number of appropriate referrals also increased, but the proportion of appropriate referrals did not change significantly |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Albertson 2002158 Country: USA Study design: Sequential prospective study before-and-after Data collection method: Questionnaire Aim: To determine whether or not a brief pre-visit questionnaire about referral concerns can improve primary care provider recognition of patient concerns and satisfaction with care Detail of participants (number, any reported demographics): 12 PCP, 1495 patients |
Intervention: Patients were given a pre-visit questionnaire about referral need and rationale and a post-visit questionnaire about referral concern and visit satisfaction. Providers were given a post-visit questionnaire asking whether a referral was discussed, or made, and about visit satisfaction. In the control phase patient pre-visit questionnaires remained confidential, whereas in the intervention phase PCPs were shown the pre-visit questionnaire at the time of the encounter Control: As above Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to specialist |
Outcome measures: Referral Satisfaction |
Main results: The intervention significantly increased PCP referral recognition from 61% to 81% (p < 0.001) and was associated with increased visit satisfaction (p = 0.05). Satisfaction of PCPs with the referral discussion, overall rate of referral, and visit duration were not affected by the intervention |
Reported associations between elements for logic model: | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Banait 200353 Country: UK Study design: Cluster RCT Data collection method: NR Aim: To test the effectiveness of educational outreach as a strategy for facilitating the uptake of dyspepsia management guidelines in primary care Detail of participants (number, any reported demographics): 114 general practices (233 GPs) in Salford and Trafford |
Method: All groups received the guidelines by post. The intervention groups began to receive education outreach 3 months later Including: Practice-based seminars with hospital specialists at which guidelines recommendations were appraised and implementation plans formulated, reinforcement visits after 12 weeks Control: Guidelines received by post Length of follow-up: 6 months post intervention Response and/or attrition rate: One practice dropped out Context (from what/who to what/who): GP to open-access endocscopy (GI) |
Outcome measures: Appropriateness of referral for open-access endoscopy |
Main results: The proportion of appropriate referrals was higher in the intervention group in the 6 months’ post-intervention period (practice medians: control = 50%, intervention = 63.9%; p < 0.05) The proportion of major findings at endoscopy did not alter significantly, but there was an overall rise in acid suppressing drugs in the intervention group, compared with the control group (+ 8% vs. + 2%, p = 0.005) |
Reported associations between elements for logic model: Outreach may be more effective that passive guideline dissemination in changing clinical behaviour, but unexpected outcomes may emerge (increased drug prescription) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Bennett 200130 Country: UK Study design: cRCT Data collection method: Routine data on ENT referrals and diagnostic results Aim: To evaluate the effect of a risk factor checklist and training video for GPs in reducing inter-practice variation and improving the appropriateness of referrals Detail of participants (number, any reported demographics): 50 practices (177 GPs) from NHS Trent and West Scotland |
Intervention: Practices were cluster randomised to either the control group (n = 12) or to one of three intervention groups (training video n = 16, checklist n = 11, or both n = 11) Data on all paediatric ENT referrals and diagnostic results were collected for 1 year pre and post intervention. Referral rates for otitis media with effusion (glue ear) and for closely related conditions were calculated for children aged 0–15 years based on practice size. Positive predictive value was defined as the proportion of referrals resulting in bilateral hearing loss of > 20 dB at the ENT outpatient department Number of hours: NA Delivered by who? NA Control: No intervention Length of follow-up: 1 year Response and/or attrition rate: NR Context (from what/who to what/who): GP referral to ENT for glue ear |
Outcome measures: Appropriateness of referrals Variation in referral rate |
Main results: There was significant improvement in the positive predictive value, adjusted for patient waiting time between GP referral and appointment at the ENT department. The improvement in positive predictive value pre and post intervention was 15% (95% CI –12.1% to 41.7%) for the practices receiving both interventions, compared with 20% (95% CI –32.9 to –6.4%) for practices receiving only one intervention and a degradation of 34% for those receiving no intervention |
Reported associations between elements for logic model: Disseminating a risk factor checklist and training video improved quality of ENT referral for glue ear by more than administering only one of these interventions |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Bhalla 200237 Country: UK Study design: Case control Data collection method: Annual otolaryngology referral rates from individuals in two demographically matched general practices to the two ENT consultants were gathered prospectively over a 6-year time period Aim: To determine if the provision of clinical ENT training to a GP influenced the subsequent rates of referral to specialist clinics, and if the presence of this practitioner within the practice made any difference to overall referrals by other partners Detail of participants (number, any reported demographics): Two demographically twinned practices of similar size (demographics not reported). Intervention practice had four partners (no other details) |
Intervention: One partner, within practice A, attended three or four clinical ENT sessions over a 2-week period, once a year for 3 years, where he would work alongside the consultant otolaryngologist Number of hours: Three or four clinical ENT sessions over a 2-week period, once a year for 3 years Delivered by who? Consultant otolaryngologist Control: Second practice with no training intervention Length of follow-up: 3 years. Data gathered over 6-year period Response and/or attrition rate: NA Context (from what/who to what/who): Referral from general practice to otolaryngology (ENT) |
Outcome measures: Referral rates |
Main results: There was no statistical difference in referral rates (Kruskal–Wallis: p = 0.63) for the trained partner when compared with the other three partners in the same practice No statistical difference in referral patterns between the intervention and the control practice (Mann–Whitney U-test: p = 0.50) Intervention practice 552 referrals over 6 years, control practice 521 referrals. No difference in each year either Sharp increase in number of referrals from both practices in third year of data collection TABLE I: Numbers of ENT referrals for individual partners within the same practice YearDUJSMLST1994–542461995–615101481996–7403622391997–8383337311998–9332227321999–200026192826Total156122132142 There was also no statistical difference (Mann–Whitney U-test: p = 0.50) in the referral patterns between the two practices TABLE II: Numbers of ENT referrals from all partners within two demographically matched general practices YearPractice APractice B1994–516211995–647381996–71371191997–81391421998–9114901999–200099111 |
Year | DU | JS | ML | ST | 1994–5 | 4 | 2 | 4 | 6 | 1995–6 | 15 | 10 | 14 | 8 | 1996–7 | 40 | 36 | 22 | 39 | 1997–8 | 38 | 33 | 37 | 31 | 1998–9 | 33 | 22 | 27 | 32 | 1999–2000 | 26 | 19 | 28 | 26 | Total | 156 | 122 | 132 | 142 | Year | Practice A | Practice B | 1994–5 | 16 | 21 | 1995–6 | 47 | 38 | 1996–7 | 137 | 119 | 1997–8 | 139 | 142 | 1998–9 | 114 | 90 | 1999–2000 | 99 | 111 | Reported associations between elements for logic model: The presence of a partner in receipt of continued ENT training made no difference to the specialist referral rates over a 6-year period when compared with the other partners within the same general practice. Furthermore, the presence of this partner did not influence the number of referrals from the practice as a whole when compared with a demographically matched general practice |
| Year | DU | JS | ML | ST | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1994–5 | 4 | 2 | 4 | 6 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1995–6 | 15 | 10 | 14 | 8 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1996–7 | 40 | 36 | 22 | 39 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1997–8 | 38 | 33 | 37 | 31 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1998–9 | 33 | 22 | 27 | 32 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1999–2000 | 26 | 19 | 28 | 26 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Total | 156 | 122 | 132 | 142 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Year | Practice A | Practice B | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1994–5 | 16 | 21 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1995–6 | 47 | 38 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1996–7 | 137 | 119 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1997–8 | 139 | 142 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1998–9 | 114 | 90 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 1999–2000 | 99 | 111 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Bridgman 200570 Country: UK Study design: Controlled before-and-after Data collection method: Routine data Aim: To evaluate a slot system Detail of participants (number, any reported demographics): General practices in Staffordshire – 12 intervention, 24 controls, 63 others. Intervention group – mix of practices in area to include large and small, urban and rural, and champions and likely critics. Practices represented 14% of population, size varied from 17,000 to 14,000 and from one to six partners. 33 GPs involved, 30 full-time, three single handed and one job-share, one practice a training practice. Eleven practices were average referrers and one was a high referrer |
Intervention: System developed by MD input. GPs and orthopaedic consultants invited to a meeting to discuss and input to design. Postal survey to GPs regarding need and views Number of slots available based on registered practice population. Not adjusted for age. Quota of slots and their use fed back to practice on monthly basis. If they went beyond their quota they might not be allowed to refer any more patients that month. GPs guaranteed maximum of 8-week assessment for patients and backlog of waiting patients removed. GPs received guidelines on appropriate referrals and routes of referrals for musculoskeletal problems Clerical officer appointed to answer queries, make appointments. Modification to hospital software; referrals made using a special pro forma, which included a prioritisation score. New referrals triaged to most appropriate clinic by clinical director Number of hours: NA Delivered by who? NA Control: 24 control practices and all other practices in area (631) Length of follow-up: Up to second half of second year Response and/or attrition rate: All selected practices agreed to participate Context (from what/who to what/who): GP to orthopaedic outpatient clinic |
Outcome measures: Difference in referral rates per 10,000 population per month by practice |
Main results: 15,439 referrals made, 90% attended appointments Mean monthly referral rate in the intervention group declined 22% in year 1 and was maintained in year 2. From baseline to intervention year 1 [9.4 (SE 0.41) to 7.29 (SE 0.31) and in intervention year 2 [7.31 (SE 0.21)] Rates for two non-intervention groups were stable/slight decrease in year 1 [baseline 10.99 (SE 0.52) and 9.50 (SE 0.29) to year 1 9.9 (SE 0.39) and 9.31 (SE 0.36). Referrals increased in year 2 [11.7 (SE 0.48) and 10.33 (SE 0.36)] Difference in mean referral rate control to intervention = –1.59 intervention, –2.61 control, –4.39 other comparator Relative mean rate in reductions in mean referral rates were 14.5%, –23.7% and –39.5% in period 0, year 1 and year 2, respectively Linear regression indicated that the interaction between practice group and the time period was statistically significant (not reported) |
Reported associations between elements for logic model: A slot system can reduce referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Callaway and Frisch 2000121 Country: USA Study design: Service data review Data collection method: Clinic log review Aim: To evaluate whether LEEP training for family physicians impacts on referral to gynaecology Detail of participants (number, any reported demographics): n = 272 women attending a cervical dysplasia clinic |
Intervention: LEEP for cervical dysplasia carried out by family physicians in a cervical dysplasia clinic Number of hours: NA Delivered by who? Family physicians Control: None Length of follow-up: Evaluation over 6 years Response and/or attrition rate: NA Context (from what/who to what/who): Family physician to gynaecologist |
Outcome measures: Number of women referred to gynaecology |
Main results: During the study period, 283 women were seen in the clinic, and 26 individuals (9%) were referred by the family physician colposcopist to a consulting gynaecologist Of the 9% referred to gynaecologist, all but one subsequently treated with laser or combination of laser and LEEP |
Reported associations between elements for logic model: Family physicians who are well-trained in LEEP can manage cases without referral Physicians need to be thoroughly trained in cognitive and technical aspects of electrosurgery (and presumably have necessary equipment) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Campbell 2003131 Country: UK Study design: Cluster RCT Data collection method: Referral data Aim: To gather referral data as part of RCT evaluating specialist outreach clinics Detail of participants (number, any reported demographics): 203 GP practices in SE Scotland Women with a positive family history of breast cancer |
Intervention: Referral to a clinic held in community setting near to the GP practice Number of hours: NA Delivered by who? NA Control: Existing service – an appointment to see a consultant geneticist and breast surgeon at a regional centre Length of follow-up: NA Response and/or attrition rate: 84% of practices agreed to take part Context (from what/who to what/who): GPs to cancer genetics advisors |
Outcome measures: Referral rates before and during the trial |
Main results: The referral rate rose from 2 years before the trial to during the trial (0.21 to 0.31). A 48% increase in referral rate (p < 0.001) 43% of women asked to be referred. Younger women were more likely to have taken the initiative to request referral (p = 0.001 chi-squared). Substantially greater increase in referral rates to community clinics than to regional centre (64% increase vs. 38% increase), suggesting that providing a service in the community resulted in a change in GP referral behaviour. This was particularly apparent in practices in relatively deprived communities. Higher referral rates from practices with more female partners before and during the trial (p < 0.005 and p < 0.02) |
Reported associations between elements for logic model: Provision of specialist service in community Sex of referrer |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Chen 2010100 Country: USA Study design: Before-and-after Data collection method: Clinical data Aim: To evaluate a new consultation request process, called e-Referral Detail of participants (number, any reported demographics): Primary care clinicians practising in five hospital-based, 11 community-based, and 10 independent non-profit community clinics in San Francisco San Francisco General Hospital, which provides more than 500,000 outpatient visits annually |
Intervention: A new consultation request process, called e-Referral, was integrated into the hospital’s electronic health record. Clinician reviewers screen requests to evaluate urgency, choice of specialties, whether or not sufficient work-up information is provided, and whether or not a specialist needs to see the patient or can guide the primary care clinician through the e-Referral system Control: None Length of follow-up: NR Response and/or attrition rate: NR Context (from what/who to what/who): GP referral to secondary care |
Outcome measures: Waiting times Appropriate referrals |
Main results: Waiting times for non-urgent visits declined in seven of eight medical specialty clinics by up to 90% during the first 6 months of use. Expedited visits accounted for up to one-third of all visits in some specialties. The percentage of referrals deemed inappropriate by medical and surgical specialists was cut by more than half For clinics that had been plagued by long wait times, implementation of e-Referral resulted in dramatic improvements. For example, in rheumatology, the median wait time for a non-urgent appointment initially dropped from 126 days to 29 days Access to a common electronic health record and participation by specialists who are salaried, and thus not financially dependent on generating visits, were critical to this programme’s success, but generally neither is an option in most practice settings. Success also depends on how well reviewers interact with primary care clinicians. The majority of primary care clinicians reported that e-Referral improved patient care, but those with poorer access to the electronic health record found it more time-consuming than the previous paper-based system |
Reported associations between elements for logic model: The percentage of referrals deemed inappropriate by medical and surgical specialists was cut by more than half |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Cooper 201219 Country: UK Study design: Evaluation/audit report Data collection method: NR Aim: To identify the underlying drivers for variance in referral rate and make plans to address them Detail of participants (number, any reported demographics): Not clear |
Intervention: Peer review scheme for referrals. Two guiding principles – the review would benefit the practice and the commissioning group; there was no blame GPs, nurses, advanced health-care practitioners and practice managers attended a workshop event. Each practice bought two or three trauma and orthopaedic referral letters. Participants worked at mixed tables to understand each practice’s referral profile, and share how each practice would handle each situation. Then to identify any gaps or areas of changed needed in terms of: Information needs Training needs Commissioning needs Needs for guidelines This led to a joint health training programme and individual practices were encouraged to tackle specialties where they were outliers Number of hours: Unclear Delivered by who? Unclear Control: None Length of follow-up: NR Response and/or attrition rate: NA Context (from what/who to what/who): GP referrals to trauma/orthopaedics and any specialty |
Outcome measures: Achieving target referral levels (incentivised) Target calculated by working out average rate per 1000 of weighted captitation for all GP referrals. Practices were awarded 20p per registered patient for referring below that level |
Main results: Trauma and orthopaedic expenditure in 2010–11 was 17% lower than in 2006–7 when the first workshop was conducted. One practice cut ENT referrals by 20% in the first year and 40% overall |
Reported associations between elements for logic model: Using clear communication and a professional approach can lead doctors to accept that peer review in not a ‘blame game’ but an opportunity to share and build experience In this context peer review can have a positive effect in reducing referral rates |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Cox 2013156 Country: UK Study design: Retrospective before-and-after Data collection method: Internal peer review of referral data Aim: To establish whether or not the introduction of referral management centres was associated with a reduction in hospital outpatient attendance rates Detail of participants (number, any reported demographics): 376,000 outpatient attendances from 85 practices |
Method: 85 practices formed five groups to manage referrals. Two groups also carried out peer review of referrals The directly age standardised GP referred first outpatient monthly attendance rate was calculated for each group from April 2009 to March 2012 using 5-year age bands. Linear regression tested for association between the introduction of referral management and change in outpatient attendance rate The RM interventions were more complex than internal peer-review interventions, involved a wider range of activities and including activities not directly related to referral management (no info given) Control: None Length of follow-up: Unclear. Study 3 years Response and/or attrition rate: NA Context (from what/who to what/who): GP to hospital outpatients |
Outcome measures: Attendance rate |
Main results: Four groups showed statistically significant increases in attendance rates ranging from 0.41 to 1.20 attendances per 1000 persons per month After correction, only one group (3, referral management centre) remained significant (1.05 attendances per 1000 persons per month, 95% CI 0.64 to 1.64; p < 0.005) There were no decreases in attendance rate |
Reported associations between elements for logic model: The introduction of referral management centres was not associated with a reduction in hospital attendance rates in any group |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Cusack and Buckley 200543 Country: UK Study design: Before-and-after Data collection method: Examination of referral letter quality Aim: To investigate the impact of issuing guidelines on acne referral Detail of participants (number, any reported demographics): 90 referral letters from GPs prior to guidelines and 60 following guideline introduction. 36 GPs |
Intervention: Guidelines and pro forma for future referrals sent to GPs Number of hours: NA Control: None Length of follow-up: 18 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to dermatology |
Outcome measures: Number of referrals |
Main results: The number of referrals in accordance with NICE guidelines increased from 31% to 45% after introduction of guidelines (p = 0.041). The number of inappropriate referrals decreased from 69% to 55% 22% of GPs (8 of 36) fully complied with guidelines. Over 50% of referrals still inappropriate. Pro forma used in only 23% of referrals and provision of data in referral letters remained poor. Number of referrals per month only marginally decreased |
Reported associations between elements for logic model: Guideline provision and limited impact on referral |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| DAMASK 200876 Country: UK Study design: Cost-effectiveness analysis alongside RCT Data collection method: Costs estimated in terms of QALYs, patient responses to EQ-5D questionnaire Aim: To investigate cost-effectiveness of GP referral to early MRI scan Detail of participants (number, any reported demographics): n = 386 patients consulting GP about a knee problem |
Intervention: Referral from GP to local radiology department for MRI. Early access to imaging Number of hours: NA Delivered by who? NA Control: Normal care – referral to local orthopaedic department Length of follow up: NA Response and/or attrition rate: Data available for 70% (386) of sample Context (from what/who to what/who): GP to MRI (knee problems) |
Outcome measures: QALYs Cost Health outcomes Knee-related NHS usage reported by patients |
Main results: Early MRI was associated with higher NHS cost by £294 per patient and a larger number of QALYs by 0.05. Incremental cost per QALY gained of £5840 was below cost threshold of £20,000 per QALY commonly used in the NHS; therefore, is cost-effective use of NHS resources Higher cost in intervention group was partly due to higher number of primary care consultations and more use of physiotherapy but largely due to results from greater use of MRI |
Reported associations between elements for logic model: Early access to imaging cost-effective |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Dennison 200699 Country: UK Study design: Cross-sectional evaluation Data collection method: Electronic data analysis Aim: To assess the effect of the intervention on waiting times and attendance Detail of participants (number, any reported demographics): 54 patients referred electronically and 189 referred on paper; London hospital. 22 GPs in four practices |
Intervention: Electronic surgical referral pro forma system including patient details, symptoms, urgent/routine, provisional diagnosis, free-text box Number of hours: NA Delivered by who? NA Control: Paper referrals Length of follow-up: NA Response and/or attrition rate: All referrals over a 1-year period Context (from what/who to what/who): GP to colorectal and gastroenterology clinics |
Outcome measures: Waiting time referral to appointment booked Waiting time referral to clinic appointment Patient attendance rate |
Main results: Patients were 21% less likely to change appointment when referred electronically. Referral to appointment was 8 weeks for electronic system, 10 weeks for paper system. Referral to booking was 0 days for electronic system, 7 days for paper system – significant difference (Mann–Whitney U-test no other details) 8.5% non-attendance electronic system, 22.5% paper system – significant difference (χ2 no other details) |
Reported associations between elements for logic model: Electronic referral can improve efficiency of service |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Dey 200466 Country: UK Study design: cRCT Data collection method: NR Aim: To investigate the impact on patient management of an educational strategy to promote these guidelines among GPs Detail of participants (number, any reported demographics): 24 health centres. Two thousand, one hundred and eighty-seven eligible patients presented with acute low-back pain during the study period – 1049 in the intervention group and 1138 in the control group. Aged 18–64 years Mean age in years (SD) – 42.2 (12.1) intervention and 41.3 (12.5) control Female sex (%) –568 (54.1) intervention and 618 (54.3) control |
Intervention: Practices in the intervention arm were offered outreach visits to promote national guidelines on acute low-back pain, as well as access to fast-track physiotherapy and to a triage service for patients with persistent symptoms At least two members of the guideline team attended each visit; included senior representatives from the musculoskeletal directorate, physiotherapy services and the health authority. Members of the guideline team facilitated a structured interactive discussion with the GP, which was based on the ‘elaboration likelihood model of persuasion’. This discussion was used to raise awareness of the RCGP guidelines, adapted to the local context; emphasise the key messages in the guidelines; identify potential barriers to implementation; and suggest strategies for overcoming the barriers identified. GPs were given a poster reinforcing guideline recommendations and a copy of a text recommended by the RCGP for patients. Referral forms for access to fast-track physiotherapy were distributed at this session, as were forms for direct access to the back clinic of patients who had failed to respond to conservative management within 6 weeks Number of hours: NA Delivered by who? Guidance team (as above) Control: No intervention Length of follow up: 8 months Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to secondary care for low-back pain |
Outcome measures: Radiography referral, sickness certificate issued, prescribed opioids or muscle relaxants, referred to secondary care, physiotherapy, or educational programmes |
Main results: The estimated annual consultation rate for acute low-back pain was 35 per 1000 adults in the intervention group, compared with 38 per 1000 in the control group. There were no significant differences between study groups with respect to the proportion of patients who were referred for radiography (difference = 1.4%; 95% CI –4.1% to 6.8%), issued with a sickness certificate (difference = –1.5%; 95% CI –10.3% to 7.3%), prescribed opioids or muscle relaxants (difference = –0.03%; 95% CI –5.5% to 5.4%) or referred to secondary care (difference = 1.1%; 95% CI –0.3% to 2.6%). Significantly more patients in the intervention group were first referred to physiotherapy or to educational programmes at the back pain unit than in the control group (26.0% and 13.8%, respectively; difference = 12.2%; ICC = 0.0563; χ2 = 6.49, 1 df; p = 0.01; 95% CI for difference in proportion = 2.8% to 21.6%). A total of 121 (11.5%) patients in the intervention arm were referred to the triage service within the follow-up period. Of the 273 referrals to physiotherapy or the back pain unit by GPs in the intervention group, 110 (40.3%) were directed to these services by the back clinic triage service |
Reported associations between elements for logic model: The management of patients presenting with low-back pain to primary care was mostly unchanged by an outreach educational strategy to promote greater adherence to RCGP guidelines among GPs |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Dhillon 200382 Country: UK Study design: RCT Data collection method: NR Aim: To evaluate the impact of GP direct access to scanning Detail of participants (number, any reported demographics): 330 patients aged 31 to 89 years, Edinburgh. 18 general practices |
Intervention: GP direct access to DXA scanning for patients at risk of osteoporosis, no specific guidelines issued Number of hours: NA Delivered by who? NA Control: Routine rheumatology clinic appointment and scan Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GP to rheumatology specialist |
Outcome measures: Frequency of change of GP management following scan |
Main results: Mostly clinical outcomes, but includes limited referral rate data. Before intervention the range of number of referrals for scanning was 0.01% to 0.6%, median 0.2%. Number of referrals to specialist clinic was 24 in intervention group, compared with 12 in control group Study concludes direct access more economically efficient; not fully explained in the data how this is evaluated |
Reported associations between elements for logic model: Access to scanning and efficiency? |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Donohoe 200031 Country: UK Study design: cRCT Data collection method: Questionnaires Aim: To evaluate a model of integrated diabetic foot care on identification and clinical management Detail of participants (number, any reported demographics): 10 towns from mid and East Devon. Total of 1939 patients with diabetes (aged over 18 years) |
Intervention: The intervention consisted of the integrated care model where the patient’s feet are examined by the GP or practice nurse. It includes complementary educational interventions aimed at clarifying management of the diabetic foot, referral criteria, and the responsibilities of professionals The intervention included practice visits and education of the whole practice team Leaflets outlining patients’ role and responsibility were disseminated to the practices Control: Current foot care arrangements and a practice visit where an alternative education package (on diabetic neuropathy) was given Length of follow-up: 6 months Response and/or attrition rate: Patients – intervention 68%, control 65% Professionals – intervention 80%, control 81% Context (from what/who to what/who): GP referral for diabetic foot care |
Outcome measures: Patient attitudes to value and importance of foot care Foot care knowledge (patient and professional) Use of services |
Main results: Attitudes to foot care improved in both groups (mean change 3.91, 0.68) with SD in change of 3.18 (95% CI 1.29 to 5.07) between groups Patient knowledge about foot problems improved significantly in both groups (mean percentage change 1.09, 1.32, but with no significant difference in change –0.09 (95% CI –1.81 to 18.62) between groups Appropriate referrals from intervention practices to the specialist foot clinic rose significantly (p = 0.05), compared with control practices (p = 0.14) |
Reported associations between elements for logic model: Integrated care arrangements can lead to an increase number of appropriate referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Eccles 200154 Country: UK Study design: RCT Data collection method: A random subset of GP patients’ records were examined for concordance with the guidelines Aim: To assess two methods of reducing GP requests for radiological tests in accordance with the UK Royal College of Radiologists’ guidelines on lumbar spine and knee radiographs Detail of participants (number, any reported demographics): six radiology departments and 244 general practices |
Intervention: A group of GPs and consultant radiologists wrote referral guidelines and educational messages for lumbar spine and knee radiographs (based on the Royal College of Radiologists’ guidelines and the RCGPs’ back-pain guidelines). The referral guidelines were sent by post to all GPs. Each practice was then randomly allocated to: audit and feedback, or control; and educational messages or control Feedback was prepared by the research team from routine data provided by the radiology departments. It covered the previous 6 months and was sent to GPs at the start of the intervention period and 6 months later. Feedback contained the number of requests for lumbar spine and knee radiographs made by the whole practice compared with requests made by all GPs in the study Educational messages were attached to the reports of every knee or lumbar spine radiograph requested during the 12-month intervention (e.g. ‘in adults with knee pain, without serious locking or restriction in movement, radiograph is not routinely indicated’) Number of hours: NA Delivered by who? Radiologists Control: Usual care Length of follow-up: 1 year Response and/or attrition rate: Of 247 practices, three dropped out Context (from what/who to what/who): GP to radiology |
Outcome measures: Number of radiograph requests per 1000 patients per year |
Main results: The effect of educational reminder messages (i.e. the change in request rate after intervention) was an absolute change of 1.53 (95% CI 2.5 to 0.57) for lumbar spine and of 1.61 (2.6 to 0.62) for knee radiographs, both relative reductions of about 20% The effect of audit and feedback was an absolute change of 0.07 (1.3 to 0.9) for lumbar spine of 0.04 (0.95 to1.03) for knee radiograph requests, both relative reductions of about 1% Concordance between groups did not differ significantly. Requests from doctors who had received audit and feedback were no more likely to be appropriate than requests from other doctors. The OR for lumbar spine radiographs was 0.75 (95% CI 0.52 to 1.07) and for knee was 0.82 (0.50 to 1.33). For doctors who had received educational reminder messages, the equivalent values were 0.95 (0.63 to 1.67) and 1.36 (0.86 to 2.23) |
Reported associations between elements for logic model: In this study 6-monthly feedback of audit data was ineffective but the routine attachment of educational reminder messages to radiographs was effective and did not affect quality of referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Eley 201083 Country: UK Study design: Audit Data collection method: Retrospective review of electronic records Aim: To assess effectiveness of direct referral audiology clinics on ENT appointments and appropriate GP use of clinics Detail of participants (number, any reported demographics): 353 patients [178 female, 175 male, mean age 77 (60–96) years] |
Intervention: DRACs for the assessment and provision of hearing aids in those over 60 years were introduced as a means of decreasing outpatient waiting times and demand on ENT appointments Number of hours: NA Delivered by who? Audiologist Control: None Length of follow-up: 4 months Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to audiology clinics or ENT |
Outcome measures: ENT appointments and appropriate GP use of clinics |
Main results: Of the 353 patients seen within the DRAC clinics, 320 were ultimately provided with a hearing aid. 55 patients require review by an otolaryngologist, either by direct referral or via their GP. The greatest lack of adherence to the referral criteria for DRAC appointments related to appropriate treatment of wax within the community |
Reported associations between elements for logic model: DRAC continues to provide a cost benefit to the NHS by reducing demand on ENT appointments |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ellard 201238 Country: UK Study design: Before-and-after study (also control condition) Data collection method: All adult dermatology referrals seen between 1 April and 30 June in 2010 and 2011 (after the teaching programme) were examined using clinic letters or pro formas completed by clinicians after consultations based on NICE guidelines as criterion for appropriateness Aim: To assess the benefit of a consultant-led dermatology training programme for GPs on the number and quality of referrals to a local university teaching hospital Detail of participants (number, any reported demographics): 30 GPs from 26 practices attended the teaching programme |
Intervention: All local GPs were invited to participate in six 2-hour interactive sessions on common skin conditions in early 2011 Number of hours: 6 × 2 hours Delivered by who? NR Control: 36 other local GP practices that did not participate in the training programme Length of follow up: 3 months Response and/or attrition rate: NR Context (from what/who to what/who): GP referrals to dermatology |
Outcome measures: Appropriate referrals |
Main results: During the 3-month study period in 2010, 542 patients were seen, of whom 39% were appropriate referrals. After the teaching programme, 478 patients were seen during the same time period in 2011, of whom 58% were appropriate. More appropriate referrals were seen in all conditions except basal cell carcinoma, where there was a fall from 100% in 2010 to 93.8% in 2011 30 GPs from 26 practices attended the teaching programme. Appropriate referrals from these GPs increased from 37.2% in 2010 to 51.8% after training, accompanied by an increase in the mean number of referrals from 20.7 to 25.7. Furthermore, the overall number of appropriate referrals increased from 37.8% to 49.5% at these 26 surgeries. These results were compared with the 36 other local GP practices that did not participate in the training programme, which also displayed an increase in appropriate referrals from 40.8% to 56.4% from 2010 to 2011 This may reflect local referral priorities, patient factors and underlying differences between the practices, suggested by the observation that the mean number of referrals in 2010 from practices attending teaching was 12.37 (SD 9.02), compared with 4.83 (SD 3.54) in those that abstained. This study has limitations in its design and size, but the results suggest that further investigation into the benefits of GP education in dermatology would be worthwhile |
Reported associations between elements for logic model: During the 3-month study period (post GP training), the number of appropriate referrals increase, but control GPs also saw an increase in appropriate referral |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Elwyn 200755 Country: UK Study design: Before and after Data collection method: NR Aim: To evaluate a system of providing feedback to clinicians following referral requests not adhering to NICE guidelines Detail of participants (number, any reported demographics): 215 GPs in catchment area of three endoscopy units |
Intervention: Letter to GPs stating that two GPs would be employed part-time to assess all endoscopy letters and referrals for dyspepsia and they would be judged against recently issued NICE guidelines. Letter said that where referrals did not meet criteria, the referring doctor would be informed by letter giving reason for non-adherence to guidelines. All Wales Dyspepsia Guidelines based on NICE criteria circulated to all GPs 2 weeks earlier to this letter Referrals were processed in usual way Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 5 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to endoscopy clinic |
Outcome measures: Adherence to NICE guidelines Number of referrals for gastroscopy Time referral to procedure |
Main results: Adherence to NICE guidelines for referral criteria increased significantly among GPs following the intervention (mean 55% to 75%, 95% CI 13.6 to 26.4; p < 0.001). No similar effect for hospital doctors Number of gastroscopy referrals for dyspepsia declined after the intervention; however, not significantly for GPs after inclusion of seasonal effects (p = 0.065) Intervention significantly reduced the referral to procedure time for gastroscopy (mean 52.1 days to mean 39.4 days, p < 0.001, 95% CI 6.6 to 18.6 days) Need to consider demand generated across all health care – hospital doctor referrals accounted for many more referrals than GPs (1720 bed hospital doctors referred more than 215 GPs) |
Reported associations between elements for logic model: Feedback to referrers can improve adherence to referral guidelines |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Emery 2007111 Country: Australia Study design: cRCT Data collection method Aim: To evaluate the effect of an assessment strategy using the computer decision support system (the GRAIDS software), on the management of familial cancer risk in British general practice in comparison with best current practice Detail of participants (number, any reported demographics): 45 general practice teams in East Anglia Randomised to GRAIDS support [intervention (n = 23) or comparison (n = 22)] |
Method: Training in the new assessment strategy and access to the GRAIDS software (GRAIDS arm) was conducted, compared with an educational session, and guidelines about managing familial breast and colorectal cancer risk (comparison) were mailed All GPs and practice nurses attended a 45-minute educational session on cancer genetics, delivered at their general practice. They were also introduced to the principles of the GRAIDS intervention Control: Current practice Length of follow-up: 12 months Response and/or attrition rate: 45/170 participated. All 45 practice teams were in the trial for a minimum of 12 months and none withdrew Context (from what/who to what/who): GP to regional cancer genetics service |
Outcome measures: Proportion of referrals made to the Regional Genetics Clinic for familial breast or colorectal cancer that were consistent with referral guidelines Practitioner confidence in managing familial cancer (GRAIDS arm only) and cancer worry, risk perception and knowledge about familial cancer |
Main results: There were more referrals to the Regional Genetics Clinic from GRAIDS than comparison practices (mean 6.2 and 3.2 referrals per 10,000 registered patients per year; mean difference 3.0 referrals; 95% CI 1.2 to 4.8; p = 0.001); referrals from GRAIDS practices were more likely to be consistent with referral guidelines [(OR) 5.2; 95% CI 1.7 to 15.8; p = 0.006]. Patients referred from GRAIDS practices had lower cancer worry scores at the point of referral (mean difference 1.44; 95% CI 2.64 to 0.23; p = 0.02) There were no differences in patient knowledge about familial cancer. The intervention increased GPs’ confidence in managing familial cancer |
Reported associations between elements for logic model: Compared with education and mailed guidelines, assessment including computer decision support increased the number and quality of referrals to the Regional Genetics Clinic for familial cancer risk, improved practitioner confidence and had no adverse psychological effects in patients |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Eminovic 200986 Country: the Netherlands Study design: cRCT Data collection method Aim: To determine whether or not teledermatologic consultations can reduce referrals to a dermatologist by GPs Detail of participants (number, any reported demographics): 85 GPs from 35 general practices in two regions in the Netherlands (Almere and Zeist); five dermatologists from two non-academic hospitals were also included in the study |
Method: The GPs randomised to the intervention used a teledermatologic consultation system to confer with a dermatologist, whereas those in the control group referred their patients according to usual practice The improved training programme for all intervention GPs included instructions on taking digital images, downloading images to the computer, managing files, and using the website. Dermatologists were taught how to use the website and complete the study forms All patients, regardless of their condition, were seen in the office by a dermatologist after approximately 1 month Control: Usual practice referral. In most cases this involved patients visiting the outpatient clinic with a letter in which the GP described findings pertinent to the case Length of follow up: 1 month Response and/or attrition rate: Of 56 GP practices eligible for participation, 36 (53%), including 110 GPs, agreed to participate Context (from what/who to what/who): GP to dermatology |
Outcome measures: The proportion of office visits prevented by teledermatologic consultation, as determined by dermatologists |
Main results: The 85 study GPs enrolled 631 patients (46 intervention GPs, 327 patients; 39 control GPs, 304 patients). The five dermatologists considered a consultation preventable for 39.0% of patients who received teledermatologic consultation and 18.3% of 169 control patients, a difference of 20.7% (95% CI 8.5% to 32.9%) At the 1-month dermatologist visit, 20.0% of patients who received teledermatologic consultation had recovered, compared with 4.1% of control patients. No significant differences in patient satisfaction were found between groups |
Reported associations between elements for logic model: Teledermatologic consultation can reduce referrals to a dermatologist by 20.7% |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Emmerson 200340 Country: Australia Study design: Evaluation questionnaire Data collection method: Surveys Aim: To develop a psychiatric assessment and advisory service for local GPs Detail of participants (number, any reported demographics): Five psychiatrists. 200 GPs in total |
Intervention: Five full-time psychiatrists dedicated a 1-hour appointment per week in their hospital private practice clinics to assess patients referred by local GPs. The Psych Opinion clinic was advertised through the Division of General Practice Newsletter Number of hours: 1 per week Delivered by who? Psychiatrists Control: NA Length of follow-up: NA – 1-year project Response and/or attrition rate: NA Context (from what/who to what/who): GP to psychiatry |
Outcome measures: Referral to the Psych Opinion Clinic Satisfaction with and awareness of the service |
Main results: After 12 months referrals to the clinic were disappointing (n = 30, with 10 referrals from one GP) Feedback from GPs who had used the service showed high levels of satisfaction with the service (mean score 6.2/7) Feedback from GPs who had not used the service showed a strong endorsement of the concept (94%), but there was poor awareness of the service’s existence (26%) |
Reported associations between elements for logic model: This study highlights the need to market new referral interventions, both initially and on an ongoing basis |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Engers 200567 Country: the Netherlands Study design: cRCT Data collection method: GP registration forms Aim: To assess the effectiveness of tailored interventions (multifaceted implementation strategy) to implement the Dutch low-back pain guideline for GPs with regard to adherence to guideline recommendations Detail of participants (number, any reported demographics): 41 of the 67 randomised GPs reported on a total of 616 consultations for 531 patients with non-specific low-back pain |
Intervention: GPs were randomised to an intervention or a control group. The GPs in the intervention group (n = 21) received tailored interventions consisting of the Dutch low-back-pain guideline for GPs, a 2-hour educational and clinical practice workshop; two scientific articles on low-back-pain management; the guideline for occupational physicians; a tool for patient education; and a tool for reaching agreement on low back care with physical, exercise, and manual therapists The participating GPs were asked to recruit consecutive patients with a new episode of low-back pain as the main reason for consultation Number of hours: 2 Delivered by who? Psychologist-physiotherapist Control: The control group (n = 20) received no intervention Length of follow-up: NR Response and/or attrition rate: NR Context (from what/who to what/who): GP referrals for low-back pain |
Outcome measures: Advice and information, referral to other health-care providers, and prescription of medication |
Main results: 41 of the 67 randomised GPs reported on a total of 616 consultations for 531 patients with non-specific low-back pain. The advice and explanation provided by the GP, the prescription of paracetamol (33% vs. 21%) or non-steroidal anti-inflammatory drugs (54% vs. 62%), and prescription of pain medication on a time-contingent (70% vs. 69%) or a pain-contingent basis (30% vs. 31%), showed no statistically significant differences between the intervention and control groups. There were also no differences in overall referral rate [23% vs. 28%, OR 0.8 (95% CI 0.5 to 1.4)]. However, in follow-up consultations fewer patients were referred to a physical or exercise therapist by the GPs in the intervention group than in the control group [36% vs. 76%, OR 0.2 (95% CI 0.1 to 0.6)] |
Reported associations between elements for logic model: The multifaceted intervention designed to address certain barriers to the implementation of the Dutch guideline for low-back pain for GPs was found to have minimal impact with regard to patient education, referral to a therapist, and prescription of pain medication, although the GPs studied here were already found to adhere to the guidelines to a fair extent |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Evans 200921 Country: UK Study design: Evaluation Data collection method: Referral data fed back to practices (patient attendance statistics) Aim: To engage GPs and consultants in discussions as to the validity, quality and appropriateness of GP referrals to increase the quality of referrals Detail of participants (number, any reported demographics): 9 of 13 interested practices selected by competitive tender |
Intervention: A year-long scheme where GPs were funded for weekly protected time to discuss their referrals retrospectively through peer review, and to attend six weekly cluster meetings where representatives from the practices met with consultants to discuss the appropriateness of the referrals and the use of alternative, community-based services Number of hours: Weekly protected time (1 hour a week) Delivered by who? NA Control: None Length of follow-up: 1-year intervention Response and/or attrition rate: NA Context (from what/who to what/who): GP referrals to hospital specialists (emergency and orthopaedics in all practices, plus one of paediatrics, gastroenterology and cardiology) |
Outcome measures: Quality of referral (appropriateness) |
Main results: There was, on average, a significant drop in referrals between the first and fourth quarters (z = 2.25, p = 0.025) The quality of referrals ad judged by doctors’ peers improved. Referral rates in orthopaedics showed a striking reduction of up to 50%. Variability between practices decreased and referral to local services increased Alternative community-based services were explored and an understanding of the best local pathways for some common conditions was reached |
Reported associations between elements for logic model: The authors suggest that this peer-review intervention was a more sustainable and intuitive method of improving referrals than referral management centres |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Evans 201122 Country: UK Study design: Service development project Data collection method: Data collected in Microsoft® Excel (Microsoft Corporation, Redmond, WA, USA) Aim: To use peer review with consultant engagement to influence GPs to improve the quality and effectiveness of their referrals Detail of participants (number, any reported demographics): 10 GP practices and seven specialties in Gwent 21 of 53 individual GPs were female (median age 44 years for females and 48.5 years for males) |
Intervention: GPs discussed the appropriateness of referrals in selected specialties including referral information, and compatibility with local guidelines, usually on a weekly basis, and were provided with regular feedback of benchmarked referral rates. Six weekly cluster groups, involving GPs, hospital specialists and community health practitioners discussed referral pathways and appropriate management in community-based services Number of hours: Six weekly groups Delivered by who? NR Control: None (some comparison with practices not taking part in the study) Length of follow-up: Study length 2008–9 Response and/or attrition rate: 10 of 13 GP practices. 53 of 58 individual GPs at the end of year 1 Context (from what/who to what/who): GP referral to seven specialties |
Outcome measures: Referral rate |
Main results: Overall, there was a reduction in variation in individual GP referral rates (from 2.7–7.7 to 3.0–6.5 per 1000 patients per quarter), and a related reduction in overall referral rates (from 5.5 to 4.3 per 1000 patients per quarter) However, although the highest individual referrers showed a decrease, the lowest referrers may show an increase in referrals (and a significant negative correlation comparing the first month’s data with the change from first to last month: r = 0.719, p = 0.019) Both reductions appeared sustainable while the intervention continued and referral rates rose in keeping with local trends once the intervention has finished |
Reported associations between elements for logic model: The peer-review intervention was effective and sustainable while the intervention continued. Subsequently, referral rates rose again in line with local trends |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Fearn 200948 Country: UK Study design: Before-and-after Data collection method: Semistructured interviews Aim: To investigate whether or not the introduction of the QOF Depression Indicators changed the pattern of referrals from primary care to a dedicated dementia clinic Detail of participants (number, any reported demographics): All referrals |
Intervention: Examined all referrals from primary care physicians seen in the cognitive function clinic for the 18-month period immediately preceding (November 2004 to April 2006) and following (May 2006 to October 2007) introduction of the QOF in April 2006 Control: NA Length of follow-up: 18-months Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to a dedicated dementia clinic |
Outcome measures: Referral |
Main results: The percentage of all referrals to the cognitive function clinic originating from primary care was about half in both time periods and did not differ significantly between the two time periods (χ2 = 0.88, df = 1, p > 0.1; z = 0.77, p > 0.05) Of the referrals from primary care, about one-third referred in both time periods had dementia. The RR of a diagnosis of dementia in a primary care referral pre and post QOF was 0.55 (95% CI 0.40 to 0.74) and 0.66 (95% CI 0.49 to 0.89), respectively The null hypothesis tested was that the proportion of patients referred from primary care with dementia was the same in cohorts seen both before and after introduction of the QOF Depression Indicator (equivalence hypothesis). The result of the chi-squared test did not permit rejection of the null hypothesis (χ2 = 0.54, df = 1, p > 0.05), a finding corroborated by the z-test (z = 0.60, p > 0.05) |
Reported associations between elements for logic model: No indication that QOF changed referral |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Felker 2004132 Country: USA Study design: Before-and-after Data collection method: Audit Aim: To evaluate a multidisciplinary mental health care team in primary care Detail of participants (number, any reported demographics): 9656 enrolled patients. Average age 53 years. 90% male PCP included 17 internal medicine physicians, 22 nurse practitioners, 10 internal medicine fellows and a variable number of residents |
Method: A multidisciplinary mental health team was created consisting of a psychologist, a psychology intern, psychiatry residents, clinical social workers and a chaplain Control: None Length of follow-up: 1 year before compared with 1 year after inception Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to mental health |
Outcome measures: Number of referrals to specialty mental health care services |
Main results: Before implementation 543 consultations occurred over the year. Of these, 543 (38%) were subsequently referred to specialty mental health care services The following year, 560 consultations occurred, but only 81 (14%) were referred The change in referral rate was significant: χ2 = 77.85, df = 1, p < 0.001 |
Reported associations between elements for logic model: A specialist mental health team in primary care reduced referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ferris 2001145 Country: USA Study design: Before-and-after Data collection method: Record analysis Aim: To evaluate the elimination of a gatekeeping system Detail of participants (number, any reported demographics): Patients aged over 18 years, n = 59,997 at baseline, 29,999 intervention. Mean age 41.7 years; 53% female |
Intervention: Need for referral from a primary care provider removed. Patients able to call and schedule an appointment with any specialist in the group Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 6-month study period Response and/or attrition rate: NA Context (from what/who to what/who): Patient self-referral to specialist service |
Outcome measures: Absolute and relative utilisation of specialty services |
Main results: Rates of visits to specialists were stable during baseline period and during the intervention period First visits to specialists, however, increased slightly from 0.19 to 0.22 per patient per 6-month period (p < 0.001) The average proportion of visits to eligible specialists as a percentage of all visits was 29% during the year before removal of gatekeeping and 29.6% during the year afterwards (p = 0.39) |
Reported associations between elements for logic model: Patient direct access had small impact on initial assessment rates however little evidence of substantial increase in use of specialty services |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ferris 2002146 Country: USA Study design: Before-and-after Data collection method: Routine data Aim: To investigate the impact of removing gatekeeping on specialist utilisation Detail of participants (number, any reported demographics): 59,952 patients |
Intervention: A multispecialty primary care group discontinued a gatekeeping system on 1 April 1998. The system was previously in place for 25 years Control: NA Length of follow-up: 6 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Overall number and distribution of patient visits to primary care and specialist |
Main results: Elimination of gatekeeping was not associated with changes in the mean number of visits to specialists (0.28 visits per 6 months before and after gatekeeping was removed), or the percentage of all children visits to specialists (11.6% vs. 12.1%, 95% CI 29.4 to 31.8 vs. 11.8 to 12.4) The proportion of all specialist visits that were initial consultations increased after gate keeping was removed from 39.6% (95% CI 29.4% to 31.8%) to 34.8% (95% CI 33.6% to 36.1%) Visits to any specialist by children with chronic disease increased from 18.6% (95% CI 17.7% to 19.1%) to 19.8% (95% CI 19.0% to 20.7%) New patient visits to specialists by children with chronic condition, as a percentage of all specialist visits increased from 28.1% (95% CI 25.9% to 30.2%) to 32.2% (95% CI 30.1% to 34.5%) |
Reported associations between elements for logic model: Removal of gatekeeping resulted in only minimal changes to utilisation of specialist care overall, but visits from children with chronic conditions increased |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Ferriter 2006157 Country: UK Study design: Before-and-after Data collection method: Referral audit Aim: To identify changes in the quality of information in referrals to an old age psychiatry service before and after the introduction of the single assessment process Detail of participants (number, any reported demographics): 20 consecutive new referrals from primary care to an old age psychiatry service in north-west London for the year before the new form was introduced (April 2003 to March 2004 – from 15 different general practices) and the following year (17 practices) |
Intervention: The single assessment process, a key element of the National Service Framework for Older People, was introduced to facilitate referrals between agencies and reduce duplication for patients, carers and clinicians. All referrals between agencies are now expected to be made on designated forms. Although there is no uniform national pro forma, many localities undertook rigorous consultation and development of referral forms, the use of which became mandatory for referrals to our service in April 2004. The referral form consists of several free-text sections: identity of patient and carer, identity of referrer; reason for referral; assessment of urgency; risk factors; current services provided to patient; diagnosis and recent history; current medication; signature of referrer Two senior clinicians performed independent and masked rating of each referral, using a five-point Likert scale of ‘strongly agree’ (1) to ‘strongly disagree’ (5). The raters answered the questions ‘I am able to judge the appropriateness of the referral’, ‘I would need to seek further information before processing this referral’ and ‘overall I think this referral is useful’ Number of hours: NA Delivered by who? NA Control: Before intervention Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Referral from primary care to old age psychiatry |
Outcome measures: Referral length, legibility, information and clinical utility |
Main results: The authors report that referrals were worse in all areas of quality of referral information after implementation of the single assessment process Word count decreased from 240 (SD 120) to 129 (SD 39), p = 0.005. Time to read in seconds increased from 96 (SD 40) to 124 (SD 41), p = 0.001. Illegible sections (% of) increased from 2 (10%) to 6 (30%), p = 0.011. The number of raters who strongly agreed or agreed with the following statements before, compared with after the single assessment process are as follows: ‘I am able to judge the appropriateness of the referral’ decreased from 19 to 5, p = 0.001 ‘I would need to seek further information before processing this referral’ increased from 3 to 17, p = 0.001 ‘Overall I think the referral is useful’ decreased from 17 to 3, p = 0.001 |
Reported associations between elements for logic model: The results of this small study suggest that the introduction of this particular single assessment process has impaired clinical communication between GPs and psychiatrists Note: It is clear that the practitioners did not ‘like’ the new process and it is unclear who the senior clinicians performing the ratings were |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Gandhi 2008108 Country: NR Study design: nRCT Data collection method: Survey Aim: Implementation of an electronic referral tool to analyse its impact on communication between primary care and specialists Detail of participants (number, any reported demographics): 430 referrals |
Intervention: Studied one practice site that implemented the referral tool and one that did not and surveyed affiliated specialists, PCPs and patients about referral communication Control: No electronic referral Length of follow-up: 2 years Response and/or attrition rate: Unclear Context (from what/who to what/who): Primary care to specialists |
Outcome measures: Referral communication |
Main results: Specialists more often received information before the referral visit from intervention PCPs vs. non-intervention PCPs (62% vs. 12%, p < 0.001), a finding that persisted after adjustment (OR = 3.3, p = 0.008) Intervention PCPs more often received communication from specialists (69% vs. 50%, p = 0.08) Patients of intervention PCPs were more likely than patients of control PCPs to report that specialists had received information before their visit (70% vs. 43%, p = 0.007) |
Reported associations between elements for logic model: Electronic referral can improve communication |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Glaves 200557 Country: UK Study design: Before-and-after Data collection method: Clinical data Aim: To determine if the use of request guidelines can achieve a sustained reduction in the number of radiographic examinations of the cervical spine, lumbar spine and knee joints performed for GPs Detail of participants (number, any reported demographics): All GPs referring to the three community hospitals |
Method: GPs referring to three community hospitals and a district general hospital were circulated with referral guidelines for radiography of the cervical spine, lumbar spine and knee, and all requests for these three examinations were checked. Requests that did not fit the guidelines were returned to the GP with an explanatory letter and a further copy of the guidelines. Where applicable, a large joint replacement algorithm was also enclosed. If the GP maintained the opinion that the examination was indicated, she or he had the option of supplying further justifying information in writing or speaking to a consultant radiologist Control: None Length of follow-up: 1 year Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for radiographic examinations of the cervical spine, lumbar spine and knee joints |
Outcome measures: Referral |
Main results: Over all sites and for all three examinations, the total number of examinations fell by 68% in the first year (95% CI 67% to 69%), achieving a 79% reduction in the second year (95% CI 78% to 80%). Knee radiographs fell by 64% in the first year (95% CI 62% to 65%), achieving a 77% reduction in the second year (95% CI 75% to 79%). Lumbar spine radiographs fell by 69% in the first year (95% CI 68% to 71%), achieving a 78% reduction in the second year (95% CI 77% to 80%). Cervical spine radiographs fell by 76% in the first year (95% CI 74% to 78%), achieving an 86% reduction in the second year (95% CI 84% to 88%). The p-value for all of these reductions was 0.0001 (highly significant). The largest individual reduction was 92% for cervical spine radiographs at Whitworth Hospital. The lowest reduction was 74% for lumbar spine radiographs at Buxton Hospital. For knee radiographs, the range was 75% to 86%; for lumbar spine radiographs, the range was 74% to 89%; and for cervical spine radiographs, the range was 80% to 92% |
Reported associations between elements for logic model: The use of referral guidelines, reinforced by request checking and clinical management algorithms, can produce a dramatic and sustained reduction in referral |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Gough-Palmer 200984 Country: UK Study design: Retrospective record analysis Data collection method: Analysis of GP requests for MRI scans Aim: To evaluate 12 years of GP open access to MRI scans Detail of participants (number, any reported demographics): 1798 scans requested by 209 GPs |
Intervention: GP access to MRI scans. No protocol, guidance or formal consultant or radiologist vetting Number of hours: NA Delivered by who? NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to MRI scanning |
Outcome measures: Number of referrals Type of scan Severity of reported findings |
Main results: GP-requested scans as percentage of workload of department are low (around 2.6%). While workload of department increased over study period, this percentage remained stable Spine, knee and brain imaging were 86% of requests. 48% of scans requested were normal or minor degenerative changes. 26% demonstrated serious pathology warranting hospital referral Wide range of scans requested per requester; average 8.5, varied from 1 to 240 |
Reported associations between elements for logic model: Marked discrepancy between GPs, suggesting need for referral guidelines While the rate of no identified abnormality was 48%, a normal scan could be beneficial in providing rapid patient reassurance, return to work and a reduction in outpatient referrals Demand for complex areas very low |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Greiver 2005114 Country: UK Study design: cRCT Data collection method: NR Aim: To determine the effectiveness of a PDA software application to help family physicians to diagnose angina among patients with chest pain Detail of participants (number, any reported demographics): 18 family physicians belonging to the North Toronto Primary Care Research Network (Nortren) or recruited from a local hospital |
Intervention: Intervention physicians received a Palm PDA (which included the angina diagnosis software). Physicians prospectively recorded the process of care for patients aged 30 to 75 years presenting with suspected angina, over 7 months Number of hours: NA Delivered by who? NA Control: Continue conventional care Length of follow-up: 7 months Response and/or attrition rate: NR Context (from what/who to what/who): GP to cardiology |
Outcome measures: Frequency of cardiac stress test orders for suspected angina The appropriateness of referral for cardiac stress testing at presentation and for nuclear cardiology testing after cardiac stress testing Secondary outcome was referrals to cardiologists |
Main results: 14 of the 28 patients in the control arm (50%) and 30 of the 37 patients in the PDA arm (81%) were referred for cardiac stress tests (p = 0.007), an absolute difference of 31% (95% CI 8% to 58%) There was a trend towards more appropriate use of stress testing (48.6% with the PDA vs. 28.6% control), an increase of 20% (p = 0.284, 95% CI –11.54% to 51.4%). There was also a trend towards more appropriate use of nuclear cardiology following cardiac stress testing (63.0% vs. 45.5%), an absolute increase of 17.5% (p = 0.400, 95% CI –13.9% to 48.9%) Referrals to cardiologists did not increase (38.2% with the PDA vs. 40.9%, p = 0.869). A referral was more likely to have been made if the final diagnosis was angina (likelihood ratio for referral 15.455, 95% CI 2.124 to 112.431); in other words, family physicians appeared to refer appropriately |
Reported associations between elements for logic model: A PDA-based software application can lead to improved care for patients with suspected angina seen in family practices; this finding requires confirmation in a larger study |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Griffiths 200658 Country: UK Study design: cRCT Data collection method: Dermatologists completed a standardised pro forma on all patients seen in their clinic with a diagnosis of psoriasis who had been referred from primary care Aim: To assess the effectiveness of guidelines and training sessions on the management of psoriasis in reducing inappropriate referrals from primary care Detail of participants (number, any reported demographics): Patients aged 18 years or over with psoriasis (165 health centres). n = 188 |
Intervention: Health centres in the intervention arm received guidelines on the management of psoriasis in primary care, developed by local dermatologists, supplemented by the offer of a practice-based nurse-led training session; those in the control arm received neither guidelines nor training sessions Number of hours: NR Delivered by who? Training delivered by nurse Control: No intervention Length of follow-up: Response and/or attrition rate: Outcome data were available for 188 of the 196 eligible patients Context (from what/who to what/who): Referral from primary care to dermatology for psoriasis |
Outcome measures: Anonymised pro formas were assessed by three members of an expert panel, comprising a dermatologist, a GP and a dermatology specialist nurse Referral was considered appropriate if the patient fulfilled any of the following criteria: extent of disease 20% of body surface area; unstable disease; no improvement following topical treatment (as per guidelines, for 6–8 weeks); or, when following dermatological assessment, the patient was admitted to hospital, was referred to our day treatment centre or to the dermatology nursing service, or received phototherapy or systemic therapy |
Main results: 82 health centres were randomised to the intervention arm and 83 were randomised to the control arm. Outcome data were available for 188 of the 196 eligible patients referred during the study period Patients in the intervention arm (82/105) were significantly more likely to be appropriately referred in comparison with patients in the control arm 49/83) (difference = 19.1%; OR 2.47; 95% CI 1.31 to 4.68; ICC = 0) Only 25 (30%) health centres in the intervention arm took up the offer of training sessions There was no significant difference in outcome between health centres in the intervention arm that received a training session and those that did not (OR 1.28, 95% CI 0.50 to 3.29; ICC = 0) |
Reported associations between elements for logic model: Dissemination of guidelines on the management of psoriasis in primary care improved the appropriateness of referral of patients to secondary care |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Gurden 2012133 Country: UK Study design: Before-and-after Data collection method: Questionnaire Aim: To describe and evaluate a community-based musculoskeletal service Detail of participants (number, any reported demographics): n = 696. Adult patients presenting to their GP with back or neck pain; mean age 52 years; two-thirds female; just over half in paid employment |
Intervention: Patients still having pain after 4–6 weeks’ ‘usual GP care’ offered a course of manual therapy and referred to private provider of their choice. Seen within 14 days Number of hours: Six treatments over 8 weeks. Practitioners worked to agreed protocols Delivered by who? Independent providers of chiropractic, osteopathy and physiotherapy services Control: None Length of follow-up: Until discharge from service (usually 8 weeks) Response and/or attrition rate: 696 of the 2810 seen by the service Context (from what/who to what/who): Community-based musculoskeletal service |
Outcome measures: Bournemouth Questionnaire (for back and neck pain) Bothersome Scale Global Improvement Scale Patient satisfaction with treatment |
Main results: Percentage change in scores baseline to discharge – Bournemouth Questionnaire = 64.6% patients categorised as improved, Bothersome Scale = 69.9% categorised as improved, Global Improvement Scale = 67.8% improved 99.5% satisfied or very satisfied with the treatment, 3% referred back to GP with recommendation for referral to secondary care services 97% given self-management advice and recommended for discharge ‘evaluation by PCT demonstrated reduced primary care consultations, imaging and inappropriate referrals to secondary care’ |
Reported associations between elements for logic model: Referrals to spinal surgeons reduced by more than 25% |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hands 200134 Country: UK Study design: Before-and-after Data collection method: Surveys Aim: To evaluate the effectiveness of an education interaction between consultants and GPs Detail of participants (number, any reported demographics): 22 consultants; 21 GPs |
Intervention: GPs attended outpatient sessions in different clinical specialties of their choice. Completed a questionnaire immediately after the session and at 6 months Control: NA Length of follow-up: 6 months Response and/or attrition rate: 21/150 Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral |
Main results: GPs reported changes in their clinical behaviour which appear to have been maintained at 6 months GPs stated that referral was discussed/taught in 83% of interactions. Immediately after the session, 25% of GPs thought that this would change their referral behaviour. After 6 months, 29% reported behaviour change in reference to referral Behaviour change was also reported with regard to diagnosis (42%), management (79%), prescribing (54%), and practical skills (58%) |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Harrington 200193 Country: USA Study design: Case series Data collection method: Review of patient records, survey of patient views Aim: To evaluate the impact of guidelines on referral and a referral management programme Detail of participants (number, any reported demographics): Patients with low-back pain, records of 581 patients reviewed over 1 year |
Intervention: Referral management programme – guidelines for referral including a flow chart (algorithm) for care, plus system for separating urgent cases from others – physician contacts surgeon or managers for advice on patients with red flag symptoms as per guidelines, receptionist takes information, information verified by nurse co-ordinator, physician manager reviews information obtained to determine care plan instigated by nurse co-ordinator Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 3 years’ data reviewed, 1 year pre, transition year, year after implementation Response and/or attrition rate: NA Context (from what/who to what/who): GP to spine orthopaedists |
Outcome measures: Patient visits for low-back pain to either a primary care or a specialist care provider |
Main results: Following introduction of the guidelines little change was documented from traditional referral patterns (no other information). Three years later in response to long waiting lists the referral management programme was put in place Shift of care from spine orthopaedists to primary physicians. Before, 28% of patient visits for low-back pain were to specialist care and 72% were to primary care. During transition year 13% of patient visits were to specialist care and 87% were to primary care Year after implementation 17% of visits were to specialist care and 83% of visits were to primary care Total patient visits for low-back pain increased 16% over the time period from 7988 to 9297. Estimated cost saving of $4000,000 per year in manpower cost. 90% of patients were satisfied with referral management process |
Reported associations between elements for logic model: Pre-appointment management can reduce specialty appointments (although shifts appointments to primary care) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Heaney 2001159 Country: UK Study design: RCT Data collection method: Use of health services audited from patients’ general practice notes in 12 months after receipt of booklet Aim: To investigate the effect of patient information booklets on overall use of health services Detail of participants (number, any reported demographics): 20 general practices in Lothian, Scotland Random sample of patients from the community health index (n = 4878) and of those contacting out-of-hours services (n = 4530) in the previous 12 months in each of the study general practices |
Intervention: Booklets were posted to participants in intervention groups (3288 were sent ‘What Should I Do?’ and 3127 were sent ‘Health Care Manual’). Patients randomised to control group (2993) did not receive a booklet ‘What Should I Do?’ was part of a patient education programme implemented in the Netherlands in 1993. The booklet outlines 40 common health problems and provides information on when to consult a doctor and on self-care, when appropriate ‘Health Care Manual’ was developed by a GP and a practice nurse in Dunkeld, Scotland. It outlines about 50 common health problems and also provides information about keeping healthy Control: No booklet Length of follow-up: 12 months Response and/or attrition rate: The final response rate from general practices was 20/30 (67%) Context (from what/who to what/who): Patient use of GP services |
Outcome measures: Types of service use, interactions between use, deprivation category of the area in which respondents live, and age |
Main results: Receipt of either booklet had no significant effect on health service use compared with a control group Total contacts: Book – before 4.19, after 4.20 Control – before 3.95, after 3.91 Difference (95% CI) 0.14 (–0.18 to 0.45) However, 9 out of 10 matched practices allocated to receive Health Care Manual had reduced consultation rates compared with matched practices allocated to ‘What Should I Do?’ |
Reported associations between elements for logic model: Widespread distribution of information booklets about the management of minor illness is unlikely to reduce demand for health services |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hemingway 200673 Country: UK Study design: Before-and-after Data collection method: NR Aim: To evaluate a protocol-driven referral system for colorectal cancer tests Detail of participants (number, any reported demographics): eight colorectal surgeons and 10 GI physicians |
Intervention: Leicester Colorectal Test Protocol – included list of presenting symptoms, age criteria for test and appropriate diagnostic test for each symptom. Patients had investigation before seeing outpatient clinician or on the day of the clinic. Referrals processed by ‘2-week wait’ administration staff using the protocol and assessments booked by these administration staff Protection of slots within the testing suites Referrals not complying with protocol were redirected to appropriate test without referral back to GP Number of hours: NA Delivered by who? Predominantly administrators in department Control: None Length of follow-up: Up to 2-year period Response and/or attrition rate: NA Context (from what/who to what/who): GP to colorectal outpatient clinic |
Outcome measures: Time referral to diagnosis Percentage of patients referred as urgent who were seen within 31-day target timescale |
Main results: Data for intervention period were not clear; reported by year rather than before and after Baseline before protocol: Year 1 median time to diagnosis non-emergencies 35 days (interquartile range 13–80), fast-track (categorised as 2-week wait or ‘soon’) 21 days (10–48) 62% of cancers referred as either 2-week wait or ‘soon’ were diagnosed within 31 days Year 2 non-emergencies 22 (9–59) emergencies 15 (7–37) After introduction of protocol (pilot and full implementation): Year 3 non-emergencies 20 (10–59) emergencies 13 (8–29) Year 4 non-emergencies 20 (10–51) emergencies 13 (9–23) During the 2-month full implementation period during year 3 service received 256 referrals, 64% came through 2-week wait protocol office and 36% referred directly to consultants. In these referrals 70% were diagnosed with a pathology and 19 patients were diagnosed with cancer, all within 31 days Overall, during year 3 79% of patients with colorectal cancer diagnosed who were referred as 2-week wait or ‘soon’ were diagnosed within 31 days; in year 4 the figure was 82% |
Reported associations between elements for logic model: Protocol-driven intervention had a positive impact on speed of diagnosis |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hermush 2009137 Country: Israel Study design: Before-and-after Data collection method: Retrospective examination of patient data Aim: To describe and evaluate a new model used in caring for the elderly in the community Detail of participants (number, any reported demographics): n = 512 elderly patients; mean age 79 years; 66% female |
Intervention: GP refers difficult or complex cases to a geriatrician who carries out a clinic in the same primary care location Number of hours: NA Delivered by who? Geriatrician Control: None Length of follow-up: Data collected over 3 years Response and/or attrition rate: 5086 patients over 65 years treated in the time frame Context (from what/who to what/who): Large primary care clinics in a city to geriatrician |
Outcome measures: Number of referrals Type of clinical problem |
Main results: Referrals to geriatrician increased significantly from 133 at baseline to 207 2 years later (p = 0.01) Number of visits to GP decreased in the 6 months following the consultation with the geriatrician (p < 0.01) |
Reported associations between elements for logic model: Relocation of specialist service to primary care can increase referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hill 200049 Country: UK Study design: Before-and-after (audit) Data collection method: GP audit Aim: To assess how appropriate referrals were just before and after distribution of the guidelines Detail of participants (number, any reported demographics): 33 GP practices. Data on 155 patients pre distribution of guidelines and 153 patients post distribution. In the 2-year follow-up audit, a sample of 114 new patients, seen consecutively over a period of 3 weeks, was taken |
Intervention: Referral guidelines for dermatology were compiled by the dermatologist at the Royal Surrey County Hospital in consultation with local GPs. An audit was undertaken to assess how appropriate referrals were just before and after distribution of the guidelines and was repeated 2 years later to determine whether or not they had made any significant impact Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 2 years Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to dermatology |
Outcome measures: Appropriate referrals |
Main results: In the original audit a 40% increase in the numbers of referrals which were recorded by the dermatologist as appropriate immediately after the guidelines were sent (from 57% to 80%) was seen. The 2-year follow-up audit, however, demonstrated that the improvement had not been sustained, with a decline to 48% appropriate referrals Five common conditions accounted for two-thirds of inappropriate referrals before and after the guidelines were sent |
Reported associations between elements for logic model: In response to referral guidelines, appropriate referrals increased in the short term but did not persist The need for continued GP education in dermatology to reinforce referral guidelines is demonstrated |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hilty 200624 Country: USA Study design: Before-and-after Data collection method: Videoconferencing consultations Aim: To assess changes over time in the utilisation of telepsychiatric services by individual PCPs and clinics in rural areas Detail of participants (number, any reported demographics): First 200 and the subsequent 200 telepsychiatric initial consultations |
Intervention: 400 consecutive patients received an initial telepsychiatric consultation delivered from an academic medical centre to rural or suburban primary care sites from July 1996 to December 2002 The following educational strategies were implemented:
Control: NA Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP to psychiatry |
Outcome measures: Patient demographics, diagnoses, reason for consultation, medication dosing and satisfaction |
Main results: Adult patients were primarily referred for mood and anxiety disorders, particularly for diagnosis and medication treatment planning. Over time, PCPs significantly improved medication dosing and asked for more treatment planning help. PCPs’ satisfaction also improved over time Among the first 200 consultations, only 47.4% of the medication doses for depressive and anxiety disorders were adequate, according to national guidelines. Among the second 200 consultations, dosing adequacy improved to 63.6% (p < 0.001) PCPs rated the quality of consultation as significantly higher over time (95% Cl 4.45 to 4.83; p < 0.001), and likewise with overall satisfaction (95% Cl 4.49 to 4.73; p < 0025) |
Reported associations between elements for logic model: Telepsychiatric consultation, in combination with specific educational interventions, appears to facilitate the enhancement of skills and knowledge of PCPs |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hockey 200491 Country: Australia Study design: Longitudinal (no control) evaluation Data collection method Aim: To examine the feasibility of a low-cost store and forward teledermatology service for GPs in regional Queensland Detail of participants (number, any reported demographics): 63 referrals |
Intervention: Digital pictures and a brief case history were transmitted by e-mail. A service co-ordinator carried out quality-control checks and then forwarded the messages to a consultant dermatologist. The co-ordinator returned the message to the GP. The aim was to provide advice to rural GPs within 1 working day Control: None Length of follow-up: None (6-month study) Response and/or attrition rate: NA Context (from what/who to what/who): GP to dermatology |
Outcome measures: Referral |
Main results: Over 6 months, 63 referrals were processed by the teledermatology service. In the majority of cases, the referring doctors were able to treat the condition after receipt of e-mail advice from the dermatologist. In 10 cases (16%) additional images or biopsy results were requested because image quality was inadequate The average time between a referral being received and clinical advice being provided was 46 hours |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Hughes-Anderson 2002136 Country: Australia Study design: Before-and-after Data collection method: Prospective data collection from all patients undergoing upper and lower endoscopy procedures between January 1996 and June 2000 Aim: To assess whether or not an outreach surgical service offering open-access endoscopy to rural areas was being overutilised Detail of participants (number, any reported demographics): A total of 4400 patients were seen by the outreach programme in the 5 years 1996–2000 The mean age of patients was 50.8 years (range, 15–94 years); 45% were women |
Intervention: Indications for referral between the GPs and the visiting surgeons were reviewed in patient records and assessed for compliance with the American Society for Gastrointestinal Endoscopy (ASGE) guidelines Two groups of patients were defined: those referred directly for open-access endoscopy and those selected by the surgeons. The open-access endoscopy patients were assessed on the day, prior to the procedure by the visiting surgeon Records for all patients undergoing colonoscopy were reviewed to determine the reason and number of cancelled procedures Control: None Length of follow-up: January 1996 to June 2000 Response and/or attrition rate: NR Context (from what/who to what/who): GP referral for endoscopy |
Outcome measures: The groups were analysed for appropriateness of referrals and frequency of positive pathology investigations |
Main results: A total of 772 endoscopies were performed and 75% were booked as open-access services. The referral rate for procedures was greater for GPs (583: 75%) than for the visiting surgeons (189: 25%) The overall compliance rate for approved indications using the ASGE guidelines for both groups was 92%. There was no significant difference in pathology found between groups The appropriateness of referrals for colonoscopy indicated that 28 of the colonoscopies were outside the ASGE indications. There was no significant difference between the two groups on the basis of the guidelines Difference between GP and visiting surgeon (appropriate indications) for endoscopy is 3.2%, 95% CI –1.8% to 8.2%; p = 0.34827, not significant Difference between GP and visiting surgeon (appropriate indications) for colonoscopy is 6.8%, 95% CI –1.8% to 15.4%; p = 0.14782, not significant |
Reported associations between elements for logic model: Outreach surgical service did not induce unnecessary procedures |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Idiculla 200044 Country: UK Study design: retrospective survey Data collection method: Referral letter contents Aim: To ascertain whether or not local guidelines for diabetes management influence the content of GP referral letters to a diabetes specialist clinic Detail of participants (number, any reported demographics): 400 GP referral letters |
Intervention: Analysis of 200 GP referral letters submitted before (set 1) and 200 submitted after (set 2) local guidelines on the management of adult diabetes had been issued to local GPs The frequency with which micro- and macro-vascular complications of diabetes were documented in the GP letters was compared with frequency ascertained at the first attendance at the specialist clinic Control: None Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for diabetes complications |
Outcome measures: Content of referral letters |
Main results: Following the distribution of the guidelines there was no significant change in the frequency with which specific conditions were documented in referral letter (set 1 vs. set 2): hypertension 72 vs. 79%, cerebrovascular disease 89 vs. 80%, etc. Many unreported complications were found in painters who had been referred after various periods of treatment in primary care However, the guidelines did appear to have encouraged the active treatment of hyperglycaemia by GPs before referral |
Reported associations between elements for logic model: Diabetes guidelines had very little effect on increasing the information provided in GP referral letters |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Imkampe 200647 Country: UK Study design: Before-and-after Data collection method: Aim: To determine whether or not GP grading of referrals into urgent and non-urgent had improved after the introduction of the 2-week rule was introduced Detail of participants (number, any reported demographics): All new GP referrals |
Intervention: A retrospective review of GP referrals over 8 months, between September 2003 and April 2004, with regard to their urgency, subsequent diagnosis and the use of pro formas (standardised referral formats) was carried out. The results were compared with the 1999 audit Control: None Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for breast cancer |
Outcome measures: Appropriate referral |
Main results: 82 of 1178 patients referred by GP had breast cancer vs. 115 of 1176 patients referred in 1999. Sixty-eight per cent (56/82) of breast cancer patients were referred as urgent, compared with 47% (54/115) in 1999 (p = 0.005) A pro forma was used in 47% (548/1178) of GP referrals, while no pro forma was used in 1999 Sixty-five of the 82 cancer patients were referred with a pro forma and 85% (55/65) were referred as urgent |
Reported associations between elements for logic model: GP prioritisation of referrals has improved since 1999. With the use of pro formas a significant number of patients with cancer were referred urgently |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Iversen and Luras 2000151 Country: Norway Study design: Economic analysis and modelling Data collection method: NR Aim: To explore whether or not the payment system for GPs has an impact on referral Detail of participants (number, any reported demographics): 150 GPs across four municipalities |
Intervention: Change from contract system (whereby GP receives a fixed practice allowance plus charges fee per item to each patient) to a capitation system where each person registers with a particular GP and GP income based on the number of patients on their list Number of hours: NA Delivered by who? NA Control: None Length of follow-up: Study over 3 years Response and/or attrition rate: 37% of GPs who took part in the intervention provided data Context (from what/who to what/who): GP to specialist |
Outcome measures: Number of referrals to specialists |
Main results: In the capitation system where GP income is determined by number of patients on list the GP referral rates to specialists increased by 42%. It was hypothesised that it is less profitable for the GP to provide services themselves and more profitable for them to let the specialists provide the services Number of years GP practised in the area did not have a significant impact on referral rate |
Reported associations between elements for logic model: Model of GP payment and referral rate |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Jaatinen 200295 Country: Finland Study design: RCT Data collection method: Questionnaires Aim: To consider teleconsultation as a replacement for referral to an outpatient clinic Detail of participants (number, any reported demographics): 93 patients. After non-attendance, n = 54 intervention and n = 24 control |
Intervention: GPs had to decide whether to refer for electronic consultation with the hospital, or whether to refer to outpatients as usual. Electronic communication with the hospital was through a secure web-based system Control: Conventional referral letter sent to hospital outpatient clinic Length of follow-up: None – 5-month study Response and/or attrition rate: 15 non attendees Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral satisfaction |
Main results: All patients treated by teleconsultation said they wanted the same procedure in the future and 63% of the control group said they would prefer a teleconsultation next time (p = 0.02), although they were nearly as satisfied as those who received a teleconference (p = 0.37) The doctors quickly learned to exploit the telecommunication model. The responsibility for treatment was maintained with the primary-care centre in 52% of cases using teleconsultation without any hospital visit required. The GPs and doctors agreed on follow-up treatment |
Reported associations between elements for logic model: Teleconsultation increased the probability of GPs maintaining responsibility for treatment |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Jiwa 200423 Country: UK Study design: nRCT Data collection method Aim: To determine if written feedback about the contents of GP referral letters mediated by local peers was acceptable to GPs and how this feedback influenced the content and variety of their referrals Detail of participants (number, any reported demographics): 26 GPs |
Intervention: In a controlled trial, 26 GPs were offered written feedback about the documented contents of their colorectal referral letters over 1 year. The feedback was designed and mediated by two nominated local GPs. The contents of referral letters were measured in the year before and 6 months after feedback. GPs were asked about the style of the feedback. The contents of referral letters and the proportion of patients with organic pathology were compared for the feedback GPs and other local GPs who could be identified as having used the same hospital for their referrals in the period before and after feedback Control: Control subjects were up to 50 practitioners who referred to the same local district general hospital Length of follow-up: 1 year before and 6 months after feedback Response and/or attrition rate: None withdrew from the project Context (from what/who to what/who): GP referral to specialist |
Outcome measures: Referral letter quality |
Main results: All GPs declared the method of feedback to be acceptable but raised concerns about their own performance, and some were upset by the experience There was a difference of 7.1 points (95% CI 1.9 to 12.2 points) in the content scores between the feedback group and the controls after adjusting for baseline differences between the groups There was a considerable improvement in the content of the referral letters from the feedback group from before to after feedback as illustrated below. There was no improvement in the scores for the control group in the same period Feedback group/control group: Mean scores before feedback 34.1/28.2 Mean scores after feedback 39.5/28.7 Mean difference and CIs 5.3 (1.5 to 9.2)/0.55 (–1.4 to 2.5); t-test df 20/36; p = 0.008/0.6 Of the GPs who referred to the same hospital before and after feedback, the feedback GPs referred more patients with organic pathology than other local colleagues |
Reported associations between elements for logic model: In some cases feedback improves the content of GP referral letters and may also impact on the type of patients referred for investigation by specialists |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Jiwa 200668 Country: UK Study design: cRCT (clustered by practice) Data collection method: Semistructured interviews were conducted to identify key themes relating to the use of the software. Questionnaire to practitioners and interview Aim: To evaluate a referral guideline intervention for lower bowel symptoms Detail of participants (number, any reported demographics): From 150 invitations, 44 practices were recruited with a total list size of 265,707. 44 practices with 180 GPs and 504 patients over 6-month period. GPs aged 30–60 years |
Intervention: Practices were offered an electronic interactive referral pro forma, an educational outreach visit by a local colorectal surgeon, both or neither 1. Pro forma: They developed and piloted an interactive electronic pro forma for processing referrals to colorectal surgeons (General Practice Referral Assessment Facilitator or G-RAF). The interactive pro forma requested information on drop-down menus for 15 clinical signs and symptoms previously identified by GPs and colorectal surgeons as those of significant colorectal disease. The interactive software offered the practitioner guidance on which cases needed urgent referral with reference to current UK Department of Health guidelines. A referral letter was automatically produced seeking an appropriate appointment at a hospital clinic 2. Education: A colorectal surgeon delivered short educational sessions. During the 45-minute educational outreach meeting, the presenter summarised the features of significant organic colorectal disease and encouraged questions. The published guidelines and the potential for the improvement to the management of patients were emphasised 3. Both interventions Number of hours: Education = 45 minutes Delivered by who? Local colorectal surgeon Control: No intervention Length of follow-up: NR Response and/or attrition rate: From 150 invitations, 44 practices were recruited Context (from what/who to what/who): GP referral to a colorectal surgeon |
Outcome measures: The main outcome measure was the proportion of cases with severe diverticular disease, cancer or precancerous lesions and inflammatory bowel disease in those referred by each group A secondary outcome was a referral letter quality score |
Main results: There were 716 consecutive referrals recorded over a 6-month period, of whom a diagnosis was available for 514 In the combined software arms 14% (37/261) had significant pathology, compared with 19% (49/253) in the non-software arms, RR 0.73 (95% CI 0.46 to 1.15) In the combined educational outreach arms 15% (38/258) had significant pathology compared with 19% (48/256) in the non-educational arms, RR 0.79 (95% CI 0.50 to 1.24) Pro forma practices documented better assessment of patients at referral Pilot work suggested proportion of patients referred with significant pathology is approximately 0.14. Only 18% of referrals in intervention one arm used the software. No significant difference in proportion of cases with significant pathology for either intervention or compared with no intervention Point estimates suggest that the interventions performed worse than no intervention. About a 4% absolute improvement; intervention arms could give as much as a 7% lower absolute percentage in referrals with significant pathology than control Themes in interviews: Concerns regarding the pro forma creating an additional task in the process – quicker to dictate a letter, and therefore poor adoption of the software The ‘don’t know if it is cancer’ option led to processing as an urgent referral, whereas most referrals were because the GP did not know for sure – potential overuse of urgent referral slots |
Reported associations between elements for logic model: There was a lack of evidence that either intervention increased the proportion of patients with organic pathology among those referred. The interactive software did improve the amount of information relayed in referral letters although we were unable to confirm if this made a significant difference to patients or their health-care providers The potential value of either intervention may have been diminished by their limited uptake Computer pro forma systems unpopular as administrative burden shifted to clinicians from administrative staff Study did not take account of how innovation was to be used in practice and impact on professional identify and established practices |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Jiwa 2012105 Country: Australia Study design: Before-and-after study Data collection method: Analysis of referral letters Aim: To explore if increasing the amount of relevant information in referral letters between GPs and hospital specialists helps in the scheduling of appointments for patients Detail of participants (number, any reported demographics): NR |
Intervention: Referral Writer Software – a software system to assist referral writing, consisting of a pro forma that selects relevant information from the electronic patient record and requests the doctor to choose one of six specialties for referral: urology, breast, gynaecology, upper GI, colorectal and respiratory. The doctors were finally prompted to enter details about the patient’s condition The amount of relevant information in the referral letters were assessed with reference to a published schedule 3 months before and 4 months after the intervention start date The letters were scored by a researcher for the amount of relevant information and independently checked by two specialists to determine if the urgency of the referral could be established, and what the most likely outcome was. This was later compared with the actual diagnosis Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 4 months after intervention start Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Relevant information in referral |
Main results: Each GP referred 5.6 patients on average (range 1–14) before the RW and 4.8 patients (range 0–14) after the RW. The amount of relevant information in the referrals improved substantially after the RW, mean difference 37%, 95% CI 43% to 30%; p < 0.001 For 91% of referrals after the RW, both specialists in each specialty were confident or very confident that they had enough information to decide when the patient should come to their clinic; this increased from 50% before RW, p = 0.001 There was no association between the amount of relevant information and the final diagnosis |
Reported associations between elements for logic model: Standardising and using electronic communications to refer appears to facilitate referral scheduling of specialist appointments |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Johnson 2008139 Country: UK Study design: Cross-sectional analysis of services and referral patterns Data collection method: Survey Aim: Does the provision of acupuncture in primary care reduce need for referral? Detail of participants (number, any reported demographics): Three PCTs; rural and urban mix. 109 practices; 13% offered acupuncture service |
Intervention: Acupuncture service in primary care Number of hours: NA Delivered by who? Acupuncture clinic Control: None Length of follow-up: NA Response and/or attrition rate: 57% response to first e-mail, 73% to second e-mail, remaining practices contacted by telephone Context (from what/who to what/who): Specialist clinic provided in primary care |
Outcome measures: Referral rate to orthopaedic, pain, physiotherapy, rheumatology Cost of painkillers |
Main results: ‘No evidence from the data that provision of acupuncture is associated with lower referral rates’ Note: Data presented outline mean referral rates for practices providing acupuncture clinics and ‘some’ versus ‘higher’ number of acupuncture appointments but NOT practices with no acupuncture, so this conclusion needs modification. Wide variation between different PCTs. Variation between PCTs possibly associated with local differences in referral patterns and sociodemographic characteristics |
Reported associations between elements for logic model: Variation between referral rates between providers |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Joyce 2000147 Country: USA Study design: Retrospective study Data collection method: Retrospective analysis of routine patient data Aim: To assess utilisation of ambulatory visits to primary care physicians and to specialists in two different managed care models – a closed-panel gatekeeper health maintenance organisation (HMO) and an open-panel point-of-service HMO Detail of participants (number, any reported demographics): 16,192 working-age members of the gatekeeper HMO and 36,819 working-age members of the point-of-service HMO |
Intervention: Retrospective study of patients enrolled in a single managed care organisation with two distinct product lines – a gatekeeper HMO and a point-of-service HMO. Both plans shared the same physician network Estimated the number of primary care physician and specialist visits using negative binomial regression models and predicted the number of visits per year for each person under each HMO type and copayment option Number of hours: NA Delivered by who? NA Control: Two different managed care models Length of follow-up: 2 years Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to specialists |
Outcome measures: Referral, number of visits |
Main results: There were more annual visits to primary care physicians and a greater number of total physician visits in the gatekeeper HMO than in the point-of-service plan. However, they did not observe higher rates of specialist visits in the point-of-service HMO |
Reported associations between elements for logic model: No evidence that direct patient access to specialists leads to higher rates of specialty visits in plans with modest cost-sharing arrangements |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Julian 200762 Country: UK Study design: nRCT Data collection method: Referral data and patient diaries Aim: To examine the outcomes of an integrated model that lends weight to GP-led care Detail of participants (number, any reported demographics): Large teaching hospital and GP practice; 99 Bridges, 94 one-stop menstrual clinic |
Intervention: Women attending the new Bridges pathway were compared with those attending a consultant-led one-stop menstrual clinic The Bridges pathway involved the use of shared care evidence-based guidelines for the management of patients in primary and secondary care, which determined the timings for investigations and surgical treatment. Management decisions were made by GPs in all but atypical/complex cases Control: Consultant-led one-stop clinic Length of follow-up: 8 months Response and/or attrition rate: 8/89 GPs declined Context (from what/who to what/who): GP to gynaecology |
Outcome measures: Outpatient appointments Clinical outcomes Patient views |
Main results: At 8 months there were no significant differences between the groups in terms of surgical and medical treatments of in the use of GP clinic appointments. Significantly fewer hospital outpatient appointments were made in the Bridges group than in the one-stop menstrual clinic (p < 0.001) The patient diaries demonstrated a significant improvement in the Bridges group for patient information, ease of access (p < 0.001), choice of doctor (p < 0.002), waiting time (p < 0.001) and less ‘limbo’ between primary and secondary care (p < 0.001) |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Junghans 2007109 Country: UK Study design: RCT Data collection method Aim: The effect of patient-specific ratings vs. conventional guidelines on appropriate investigation of angina Detail of participants (number, any reported demographics): n = 145 physicians |
Method: RCT of 145 physicians receiving patient-specific ratings (online prompt stating whether the specific vignette was considered appropriate or inappropriate for investigation, with access to detailed information on how the ratings were derived) and 147 physicians receiving conventional guidelines from the American Heart Association and the European Society of Cardiology. Physicians made recommendations on 12 web-based patient vignettes before and on 12 vignettes after these interventions Control: Conventional guidelines Length of follow-up: NR Response and/or attrition rate: NR Context (from what/who to what/who): GP referral for angina |
Outcome measures: Proportion of appropriate investigative decisions as defined by two independent expert panels |
Main results: Decisions for exercise electrocardiography were more appropriate with patient-specific ratings [819/1491 (55%)], compared with conventional guidelines [648/1488 (44%)] (OR 1.57; 95% CI 1.36 to 1.82). The effect was stronger for angiography [1274/1595 (80%) with patient-specific ratings, compared with 1009/1576 (64%) with conventional guidelines (OR 2.24, 95% CI 1.90 to 2.62)] Within-arm comparisons confirmed that conventional guidelines had no effect but that patient-specific ratings significantly changed physicians’ decisions towards appropriate recommendations for exercise electrocardiography (55% vs. 42%; OR 2.62, 95% CI 2.14 to 3.22) and for angiography (80% vs. 65%; OR 2.10, 95% CI 1.79 to 2.47) These effects were robust to physician specialty (cardiologists and GPs) and to vignette characteristics, including older age, female sex and non-white race/ethnicity |
Reported associations between elements for logic model: Patient-specific ratings result in more appropriate investigations of angina than conventional guidelines |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kennedy et al. 2012106 Country: UK (Scotland) Study design: Retrospective audit Data collection method: Analysis of electronic referral system data over 1 year Aim: To evaluate an electronic referral system Detail of participants (number, any reported demographics): n = 190 patients referred with suspected squamous cell carcinoma of the head and neck; 55% female; aged 19 to 92 years; mean age 58 years |
Intervention: A fast-track electronic referral system including referral guidelines. Suspected Cancer Urgent Referral Electronically System containing specific alarm symptoms Number of hours: NA Delivered by who? Electronic system Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): From GP to head and neck cancer clinic |
Outcome measures: Appropriateness of referral: number of patients referred who were subsequently diagnosed with cancer |
Main results: 52% of the urgent referrals required no further investigation following assessment and were discharged Head and neck cancer detection rate (% of patients with confirmed diagnosis from total number of referrals) was 8%. Overall cancer detection rate 15% During the time period of system operation only 14% of the total number of head and neck cancers diagnosed were referred via the electronic system. All others had been referred by non-urgent referral channels (by the same group of practitioners) 27 different GP practices used the system to refer; however, one city-centre practice accounted for 17% of referrals Author conclusion: GP referral guidelines and fast-track clinic did not work, with 86% of patients diagnosed with cancer bypassing the system |
Reported associations between elements for logic model: Referral rate disproportionately high for one city-centre practice suggesting a lower threshold to refer Analysis of referral letters revealed disappointing level of compliance with referral guidelines with 12% not mentioning any of the alarm symptoms and many not detailing risk factors Speeding of referral via the system for some patients may have resulted in longer waiting times for other patients |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kerry 200059 Country: UK Study design: RCT Data collection method: All doctors were sent a questionnaire about the guidelines Aim: To see if the introduction of radiological guidelines into general practices together with feedback on referral rates reduces the number of GP radiological requests over 1 year; and to explore GP attitudes to the guidelines Detail of participants (number, any reported demographics): 69 practices |
Intervention: In February 1995 a GP version of the RCR guidelines was sent to each GP in the 33 practices in the intervention group. After 9 months’ intervention, practices were sent revised guidelines with individual feedback on the number of examinations requested in the past 6 months. The total number of requests per practice was compared for the year before and the year after the introduction of the guidelines Guidelines for examination of chest, hips, knees, spine, skull and sinuses were printed verbatim on two sides of a sheet of A4 paper, which was then laminated Number of hours: NA Delivered by who? NA Control: Control practices were sent the guidelines at the end of the study Length of follow-up: 9 months Response and/or attrition rate: GP questionnaire 60% response rate Context (from what/who to what/who): GP referral to radiology for spinal examination |
Outcome measures: Referral rates Attitudes to guidelines |
Main results: A total of 43,778 radiological requests were made during the 2 years 1994–6 The number of referrals for all spinal examinations fell by 18% in the intervention group, compared with a 2% rise in the control group (p = 0.05) Taking requests for the lumbar spine alone, there was a reduction of 15% in the intervention group compared with a rise of 5% in the control group, giving a difference of 20% between the groups (95% CI 3% to 37%) Overall, an 8% reduction in total numbers of radiological requests was observed in the intervention group, compared with a 2% increase in the control group, giving a difference of 10% between the two groups, but this did not achieve statistical significance |
Reported associations between elements for logic model: Introduction of radiological guidelines together with feedback on referral rates was effective in reducing the number of requests for spinal examinations over 1 year |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Khan 200871 Country: UK Study design: Cohort Data collection method: NR Aim: Efficacy of direct GP referral to the hospital respiratory specialist team in the Hot Clinic in avoiding hospital admissions Detail of participants (number, any reported demographics): Data from 173 patients enrolled between 1 January 2007 and 30 June 2007 were studied. Ninety-seven (57%) were men and 75% were either current or ex-smokers |
Intervention: Hospital at-home schemes are popular for the management of acute exacerbations of COPD aimed at reducing demand for hospital inpatient beds and promoting a patient-centred approach through admission avoidance GPs and community nurses directly referred patients threatening an acute hospital admission, by fax, for a rapid assessment. The Hot Clinic service operates Monday to Friday, 09:00–16:00 hours. Patients are seen within 24 hours of the receipt of the referral letter. The consultation includes clinical assessment, chest radiograph, laboratory data and a decision whether to treat the patient in the community or to admit to the hospital. The GP would be informed by a returned typed faxed letter the same day Number of hours: NA Delivered by who? The Hot Clinic team is led by the respiratory consultant and assisted by the specialist registrar and respiratory nurse Control: None Length of follow-up: None Response and/or attrition rate: NA Context (from what/who to what/who): Referral from primary care to COPD clinic (respiratory) |
Outcome measures: The efficacy of this service was assessed in terms of admission avoidance and the rate of readmission within 1 week and 1 month of the consultation |
Main results: 27 patients (16%) were admitted directly from the Hot Clinic and 146 (84%) were treated in the community. Of those 146 patients, nine (5%) were later admitted within 1 week and 12 (7%) were admitted over 1 week to 1 month after the Hot Clinic appointment. Overall, 125 (72%) were thus treated successfully in the community without the need for hospitalisation It is unclear if all would have been hospitalised without the clinic |
Reported associations between elements for logic model: Suggests potential effectiveness of a direct GP referral system to the hospital respiratory team in avoiding hospital admissions |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kim 2004155 Country: USA Study design: Evaluation Data collection method: Telephone/postal interview Aim: To examine the effect of referral management on diabetes care Detail of participants (number, any reported demographics): n = 6941; mean age 61 (SD 13) years; 54% female |
Intervention: Translating research into Action for Diabetes (TRIAD) was a multicentre study of managed care enrolees with diabetes Prospective referral management consisted of gatekeeping and mandatory authorisation from the management office. Retrospective referral management consisted of referral profiling and appropriateness reviews Control: No referral management strategy Length of follow-up: 1 year Response and/or attrition rate: NR Context (from what/who to what/who): GP to specialist |
Outcome measures: Self reported visit to specialist Difficulty in getting referrals (perceived) |
Main results: Referral management was commonly used by health plans (55%) and provider groups (52%). In adjusted analysis, there were no associations between any of the referral management strategies and any of the outcome measures |
Reported associations between elements for logic model: Referral management strategies did not affect referrals or perceptions of referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kim 200998 Country: USA Study design: Cross-sectional Data collection method: Survey web-based Aim: To survey PCPs to assess the impact of electronic referrals on workflow and clinical care Detail of participants (number, any reported demographics): n = 298 |
Method: 18-item, web-based questionnaire to all 368 PCPs who had the option of referring to San Francisco General Hospital Asked participants to rate time spent submitting a referral, guidance of workup, wait times and change in overall clinical care compared with prior referral methods using five-point Likert scales Length of follow-up: None Response and/or attrition rate: Two hundred ninety-eight PCPs (81.0%) from 24 clinics participated Context (from what/who to what/who): Primary care to clinical care |
Outcome measures: Practitioner views Referral |
Main results: Over half (55.4%) worked at hospital-based clinics, 27.9% at county-funded community clinics and 17.1% at non-county-funded community clinics. Most (71.9%) reported that electronic referrals had improved overall clinical care. Providers from non-county-funded clinics (AOR 0.40, 95% CI 0.14 to 0.79) and those who spent ≥ 6 minutes submitting an electronic referral (AOR 0.33, 95% CI 0.18 to 0.61) were significantly less likely than other participants to report that electronic referrals had improved clinical care |
Reported associations between elements for logic model: PCPs felt that electronic referrals improved health-care access and quality; those who reported a negative impact on workflow were less likely to agree |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kim-Hwang 2010102 Country: USA Study design: Before-and-after Data collection method: Questionnaire Aim: To determine the impact of the e-Referral, compared with paper-based referral, on specialty referral rates Detail of participants (number, any reported demographics): Specialist clinicians, n = 505 |
Intervention: The study was based on a visit-based questionnaire appended to new patient charts at randomly selected specialist clinic sessions before and after the implementation of e-Referrals (using web-based pro forma). A specialist reviewer (physician or nurse) reviews the referrals and determines whether or not it is appropriate to schedule an appointment Control: Paper-based referral Length of follow-up: 2-year study Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Self-reported difficulty in identifying the referral question Referral appropriateness, need for, and avoidability of follow-up visits |
Main results: It was difficult to identify the reason for referral in 19.8% of medical and 38.0% of surgical visits using paper based methods vs. 11.0% and 9.5% of those using e-Referral (p = 0.03 and p < 0.001) Of those using e-Referral, 6.4% and 9.8% of medical/surgical referrals using paper methods vs. 2.6% and 2.1% were deemed not completely appropriate (p = 0.21 and p = 0.03) Follow-up was requested for 82.4% and 76.2% of medical and surgical patients with paper referrals vs. 90.1% and 58.1% of e-Referrals (p = 0.06 and p = 0.01) Follow-up was considered avoidable for 32.4% and 44.7% of medical/surgical follow-ups with paper-based methods vs. 27.5% and 13.5% with e-Referral (p = 0.41 and p < 0.001) |
Reported associations between elements for logic model: e-Referral can improve communication and increase the appropriateness of referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| King 2001119 Country: UK Study design: Before-and-after Data collection method: Questionnaires and interviews Aim: Whether or not in practices with high referral rate, an invitation to review referrals could identify patients on the waiting list who considered their referral unnecessary, leading to a negotiated cancelling of their appointment Detail of participants (number, any reported demographics): 435 put patient referrals made in 4 months by one general practice with 6600 patients. n = 109 eligible for study |
Intervention: 4–7 weeks after referral, selected patients were sent a questionnaire and an invitation to a review appointment Exclusion criteria were symptoms which raise the possibility of significant disease; patient’s mental state precludes consent or co-operation; the referring doctors prefers the patient not to participate; such urgency that an outpatient appointment could be expected within 3 weeks Subsequently, a series of 22 semistructured interviews were undertaken to seek the review of patients on their willingness to review the referral decision Number of hours: NA Delivered by who? GP Control: NA Length of follow-up: NA Response and/or attrition rate: 109 of 435 referrals Context (from what/who to what/who): GP referral to any specialty |
Outcome measures: Outpatient appointment cancellation |
Main results: Of 435 referrals, 109 (25%) were eligible for this study. 77 (72%) responded to the questionnaire and of those, 10 (13% of responders) indicated uncertainty that referral was still needed Eight of these attended for review, but in none of these cases was the appointment subsequently cancelled Therefore, taking cancellation of hospital appointment as an end point, the effect shown is 0 out of 435 referrals and 0 out of 109 in the intervention group (95% CI for 0 out of 109 = 0% to 3%) |
Reported associations between elements for logic model: Referral review is not an effective way to detect avoidable referrals or enable negotiated cancelling of outpatient referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Knab 2001112 Country: USA Study design: Before-and-after Data collection method Aim: To determine whether or not CBDS could enhance the ability of primary care physicians to manage chronic pain Detail of participants (number, any reported demographics): 100 chronic pain patients |
Intervention: Structured summaries were generated for 50 chronic pain patients referred by primary care physicians to a pain clinic. A pain specialist used a decision support system to determine appropriate pain therapy and sent letters to the referring physicians outlining these recommendations. Separately, five board-certified primary care physicians used a CBDS system to ‘treat’ the 50 cases. A successful outcome was defined as one in which new or adjusted therapies recommended by the software were acceptable to the primary care physicians (i.e. they would have prescribed it to the patient in actual practice). Two pain specialists reviewed the primary care physicians’ outcomes and assigned medical appropriateness scores (0 totally inappropriate to 10 totally appropriate). One year later, the hospital database provided information on how the actual patients’ pain was managed and the number of patients rereferred by their primary care physician to the pain clinic Control: None Length of follow-up: 1 year Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for chronic pain |
Outcome measures: Appropriateness Rereferral |
Main results: On the basis of CBDS recommendations, the primary care physician subjects ‘prescribed’ additional pain therapy in 213 of 250 evaluations (85%), with a medical appropriateness score of 5.5 ± 0.1. Only 25% of these chronic pain patients were subsequently rereferred to the pain clinic within 1 year |
Reported associations between elements for logic model: The use of a CBDS system may improve the ability of PCPs to manage chronic pain and may also facilitate screening of consults to optimise specialist utilisation |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Knol 200690 Country: the Netherlands Study design: Before-and-after Data collection method: Referral data/interviews Aim: To reduce dermatology referrals Detail of participants (number, any reported demographics): n = 505 consultations by 29 GPs |
Intervention: One overview and two detailed digital photographs of the skin problems were taken on a digital camera and attached to an e-mail message containing standard clinical information. The e-mail was sent to a dermatologist who replied after evaluation. After a median follow-up of 548 days, GPs were interviewed about dermatology referrals Control: NA Length of follow-up: 2 years Response and/or attrition rate: follow-up data not available for 32 (6%) patients Context (from what/who to what/who): GP to dermatology |
Outcome measures: Referral rate |
Main results: Patients were split into those who GPs would have referred without the intervention (n = 306) and those who they would not have referred Using teledermatology, 163 patients were not referred, a reduction of 163/306 or 53% There was no significant difference between dermatologist for secondary referral (χ2 = 1.6, p = 0.45). Patient gender did not affect secondary referral (χ2 = 0.8, p = 0.36) When GPs had no prior intention to refer, there turned out to be a secondary consultation in 17% of cases (24/136) Older patients were more likely to be referred (χ2 = 10.6, p < 0.01) |
Reported associations between elements for logic model: The 51% referral reduction was similar to other studies of videoconferencing |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Kousgaard 200329 Country: Denmark Study design: RCT (unblinded) Data collection method: Questionnaires Aim: To investigate GP assessment of a structured oncology information pack sent to GPs when newly referred patients had visited a department of oncology for the first time, and to compare their assessment of this material with their assessment of traditional information provided by the department Detail of participants (number, any reported demographics): 248 cancer patients and their 199 GPs |
Intervention: Intervention group practitioners received a structured information pack when their patients attended the department of oncology for the first time. The patients were informed that their GP would receive this information and thus the study was unblinded. The pack included (1) a discharge letter written in accordance with specially developed guidelines and bearing the direct telephone number of a departmental contact person, (2) information about the cancer, its treatment and prognosis, (3) general information about radiotherapy and chemotherapy and treatment of nausea and sickness and (4) information that the patient had been advised to see his/her own practitioner about problems and question Control: Participating practitioners in the control group received the traditional information from the department (i.e. the discharge letter or an extract from the hospital record) Length of follow-up: NR Response and/or attrition rate: 88.3% of the 248 questionnaires were returned Context (from what/who to what/who): GP referral to oncology |
Outcome measures: Practitioner views GP assessment of the quality of the information material received for each patient |
Main results: The structured information pack improved GP knowledge of oncology; GPs found themselves better equipped to support and counsel patients during the course of their illness, and practitioner satisfaction with the department rose GP evaluations of the first discharge letter received from the department. The two groups were significantly different (p = 0.039): Intervention group practitioners gave a significantly higher score to the information value of the discharge letter than did control group practitioners. The most pronounced difference was seen for psychosocial conditions (p = 0.001) and information about what the patient had been told at the department (p = 0.001). Stratification according to sex, years as a GP and practice location revealed no differences between the groups (data not shown) |
Reported associations between elements for logic model: Intervention, though reasonably simple, inexpensive and not particularly time-consuming, improved co-operation between the specialist department and the GP |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Lam 201125 Country: UK/China Study design: Cross-sectional Data collection method: Questionnaire (postal) Aim: To examine the outcomes of a postgraduate training course in geriatrics for primary care doctors Detail of participants (number, any reported demographics): n = 98 |
Method: An evaluative study was conducted to examine the impact of the Postgraduate Diploma in Community Geriatrics, which is a 1-year part-time program for primary care doctors developed by the Family Medicine Unit of The University of Hong Kong The diploma includes the components of clinical attachment (20 sessions of clinical geriatric teaching and five sessions of rehabilitation and community health services), interactive workshops, locally developed distance-learning manual, written assignments and examination as well as a clinical examination Control: NA Length of follow-up: NA Response and/or attrition rate: Ninety-eight replies were received with a response rate of 52.4% (98/187) Context (from what/who to what/who): GP referral to geriatrics |
Outcome measures: Referral Views on training |
Main results: Most respondents felt it was more rewarding and had participated more in geriatric care, and the majority had improvement in their communication skills with elderly patients after taking the course. Moreover, the graduates are more confident in diagnosing and managing common geriatric problems, and deciding to which specialty to refer the elderly patients Of the referrals, there was a significant increase to private geriatricians and a significant reduction to other specialists. The average number of elderly patients seen per day had also increased However, little change was observed about making nursing home visits, the frequency of which remained low. Many graduates expressed difficulties in conducting nursing home visits |
Reported associations between elements for logic model: Education can affect referral and confidence |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Leggett 200485 Country: UK Study design: RCT Data collection method: NR Aim: To compare outcomes of referral for dermatology appointments between patients whose referral letters do or do not include instant photograph(s) Detail of participants (number, any reported demographics): n = 136 (20 GPs: 10 intervention and 10 control) |
Intervention: Instant photographs, taken by the GP, were included in the referral letters. The GP took photograph(s) of the skin condition and sent them with a referral letter to the dermatologist in a numbered, sealed envelope. If a diagnosis was not possible, patients were given an appointment. If diagnosis was possible, a letter was sent to the GP with advice on management: some patients were also given an appointment for further management Number of hours: GPs were trained for 15 minutes to use a camera to produce digital photos of the presenting condition Delivered by who? NA Control: Control group patients were given outpatient appointments in the usual way Length of follow-up: NA Response and/or attrition rate: NR Context (from what/who to what/who): GP to dermatology |
Outcome measures: The numbers of study group patients needing an appointment for diagnosis or management and with a changed diagnosis after face-to-face consultations were recorded Waiting time from referral to appointment or management plan was recorded for both groups |
Main results: For 63% of the study group (45/71), a diagnosis and a management plan were made without the patient requiring an appointment. This included 38% (27/71) who, after diagnosis and initial management, needed an appointment and 25% (18/71) who did not The remainder of the study group (37%; 26/71) required a face-to-face consultation The mean time for formulation of a management plan for patients without an appointment was 17 days (SD = 11); waiting times for appointments in study and control groups were similar (mean 55 days; SD = 40) |
Reported associations between elements for logic model: Instant photography is helpful in managing dermatology referrals and offers the potential to reduce numbers requiring an outpatient appointment by 25% |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Leiba 2002130 Country: Israel Study design: nRCT Data collection method Aim: Evaluation of easy-to-access to specialists on health service use, cost (time and money) and GP attitude Detail of participants (number, any reported demographics): None |
Intervention: A specialist outreach clinic was established in a home-front military primary care clinic. Patients were referred to nearly by specialists but no further referral was required for continuity of specialist care The same analysis was applied to a similar clinic employing only GPs, which refers to military specialist centres or hospital outpatient clinics Number of hours: NA Delivered by who? NA Control: No outreach clinic Length of follow-up: 6 months Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to specialist outreach clinic or usual hospital care |
Outcome measures: Health service use, cost (time and money) and GP attitude |
Main results: The incorporation of specialists did not result in a significant increase in the overall consumption of medical services (p < 0.05). It reduced the number of referrals out of the clinic to specialist centres from 1449 to 421 per month (p < 0.05). In the control clinic, referrals to distant specialist centres and outpatient clinics showed a slight and non-significant increase Loss of work days was reduced from 2891 days per month to 1938 days per month (p < 0.001) The total cost of all medical interactions and referrals did not significantly increase after the introduction of the outreach specialist clinic (p < 0.05). Primary physicians graded their satisfaction with the new clinic as 4.5 (out of 5) |
Reported associations between elements for logic model: Improving access to specialists geographically and removing the need for a referral for each specialist visit did not increase total health-care use and costs |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Lester 200939 Country: UK Study design: cRCT Data collection method: NR Aim: To assess the effect of an educational intervention for GPs on referral rates to early-intervention services and the duration of untreated psychosis for young people with first-episode psychosis Detail of participants (number, any reported demographics): A total of 110 of 135 eligible practices (81%) were recruited. 179 young people were referred: 97 from intervention and 82 from control practices |
Intervention: REDIRECT trial (BiRmingham Early Detection In untREated psyChosis Trial) Practices with access to the three early-intervention services in three inner-city PCTs in Birmingham were eligible for inclusion. Intervention practices received an educational intervention addressing GP knowledge, skills and attitudes about first-episode psychosis. The outcome of the theoretical and modelling work suggested that the educational intervention needed to impart knowledge about important symptoms and signs evident in first-episode psychosis, teach core questioning skills, and encourage more positive attitudes towards young people with the condition A 17-minute video made specifically for the study, depicting role-played primary care consultations with young people with first-episode psychosis, was shown to GPs in intervention practices. The study team then led a 15-minute question-and-answer session including referral guidelines to early-intervention services. Two refresher educational sessions were conducted Number of hours: NR Delivered by who? NR Control: no intervention Length of follow-up: Follow-up at 4 months Response and/or attrition rate: NR Context (from what/who to what/who): Referral from general practice to psychiatry |
Outcome measures: Difference in the number of referrals to early intervention services between practices Duration of untreated psychosis Time to recovery Use of the Mental Health Act, and GP consultation rate during the developing illness |
Main results: Ninety-seven people with a first episode of psychosis were referred by intervention practices, and 82 people from control practices during the study: RR of referral 1.20 (95% CI 0.74 to 1.95, p = 0.48) No effect was observed on secondary outcomes except for ‘delay in reaching early-intervention services’, which was statistically significantly shorter in patients registered in intervention practices (95% CI 83.5 to 360.5, p = 0.002) |
Reported associations between elements for logic model: GP training on first-episode psychosis is insufficient to alter referral rates to early-intervention services or reduce the duration of untreated psychosis |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Levell 2012129 Country: UK Study design: Before-and-after study Data collection method: Standard hospital systems Aim: To assess the effect of introducing dermatology integrated intermediate-care services on the numbers of dermatology referrals to secondary care Detail of participants (number, any reported demographics): None |
Method: The dermatology intermediate care service was set up in 2005, providing services in two locations by two GPwSIs in dermatology. The GPwSIs were supported by experienced dermatology nurses and in total six clinics weekly were held, seeing approximately 30 new patients weekly Control: None (before-and-after) Length of follow-up: 2004–10 Response and/or attrition rate: NA Context (from what/who to what/who): GP referral to dermatology |
Outcome measures: Dermatology new patients |
Main results: The numbers of dermatology new patients seen, which had been stable for 5 years, showed an increase in 2007 followed by a substantial increase in 2008 and then 2009 The mean number of new patients seen in dermatology in 2004–6 was 6927 patients per year; in 2007, 7844 patients; and the mean number of new patients seen between 2008 and 2010 was 11 535 patients per year. This was an increase of 67% in the number of new patients seen. Overall, over this period, there was a 23% increase in dermatology new patients seen in secondary care dermatology in England |
Reported associations between elements for logic model: The introduction of dermatology intermediate care services was followed by a 67% increase in secondary care new patients |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Lucassen 200145 Country: UK Study design: Before-and-after Data collection method: NR Aim: To see whether or not guidelines on whom to refer to a regional genetics service could improve appropriateness of referrals Detail of participants (number, any reported demographics): NR |
Intervention: Referral guidelines for ORGS family cancer clinic were drawn up in discussion with local GPs, surgeons, radiologists, gynaecologists, public health physicians and geneticists. Evidence from national consensus was incorporated where available. The guidelines were sent to all Oxfordshire GPs and subsequent content of referral letters was analysed. A retrospective analysis of referral letters sent during the previous 6 months was also made Number of hours: NA Delivered by who? NA Control: NA Length of follow-up: 8 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to regional genetics service |
Outcome measures: Appropriate referral |
Main results: Post guidelines, more referrals met the criteria than before (χ2 = 15.79, p < 0.001) Fewer lower-risk referrals were made: 34% of letters (36/103) were high risk pre guidelines, whereas 47% (46/110) were high risk post guidance (not significant: χ2 for change in proportion of low risk pre and post = 1.34, p = 0.24, and for high risk = 3.33, p = 0.07), and that the description of the risk in the GP letter improved so that a greater proportion of generic clinic risks agreed with those described in the GP letter |
Reported associations between elements for logic model: The use of referral guidelines can improve appropriateness of referrals to secondary care (regional genetic screening service) |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Lyon 2009160 Country: UK Study design: Before-and-after Data collection method: GP data Aim: To promote the early presentation and diagnosis of breast, bowl and lung cancer Detail of participants (number, any reported demographics): NR |
Intervention: Involving local people working in partnership in their communities to raise awareness of cancer symptoms and promote early presentation. The teams work with primary care, other statutory organisations and with the voluntary sector. The specific contribution of the local people was in the identification of hard-to-reach groups and the tailoring of effective health messages Number of hours: NR Delivered by who? Local people and primary care Control: NA Length of follow-up: 1 year Response and/or attrition rate: NA Context (from what/who to what/who): GP to cancer screening |
Outcome measures: Referrals |
Main results: Interim results show an increase in the number of urgent 2-week referrals and the proportion of new cancer cases diagnosed through the urgent 2-week referral route (from 43% to 51%) for all three cancers. These results were statistically significant for the bowel cancer (χ2 = 22.193, df = 1; p < 0.001) and lung cancer pathways (χ2 = 8.886, df = 1; p = 0.003). There was also an increase in the proportion with no spread at the time of diagnosis for bowel cancer (38–43%) and breast cancer (41–44.5%), but these results did not reach statistical significance |
Reported associations between elements for logic model: Community awareness raising led to an increase in 2-week cancer referrals |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Maddison 2004154 Country: UK Study design: Before-and-after Data collection method Aim: Impact of the targeted early access to musculoskeletal services (TEAMS) programme on accessibility to musculoskeletal services Detail of participants (number, any reported demographics): No information |
Intervention: Establishing with central clinical triage a common pathway for all musculoskeletal referrals so that patients attend the appropriate department A back pain pathway led by extended scope physiotherapists was developed, and GPwSIs and extended scope physiotherapists were trained to provide services for patients with uncomplicated musculoskeletal problems in the community Control: NA Length of follow-up: 18 months Response and/or attrition rate: NA Context (from what/who to what/who): GP to musculoskeletal |
Outcome measures: Number of patients referred and seen with musculoskeletal problems, waiting times, number of duplicate referrals, and surgery conversion rates in orthopaedic clinics |
Main results: After the introduction of the targeted early access to musculoskeletal services in April 2002, there was a major increase (116%) in the total number of referrals for musculoskeletal problems. In contrast, the number of orthopaedic referrals was slightly reduced Over 18 months the total number of referrals more than doubled. Despite this, waiting times for musculoskeletal services fell; this was noticeable for rheumatology and pain management (primary data not given) Duplicate referrals were abolished. Surgery conversion rates did not, however, change The community musculoskeletal clinics were well received by GPs, and the short waiting time of 4–6 weeks put them in demand. Patients were generally seen on a one-off basis; < 10% were referred on or followed up. Patient satisfaction questionnaires showed that 88% of patients rated the service as excellent or good, and 75% were completely satisfied with the service provided |
Reported associations between elements for logic model: Community-based multidisciplinary clinics run by specially trained GPwSIs and extended scope physiotherapists are an effective way of managing patients with uncomplicated musculoskeletal problems and have been well received by patients and GPs |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Magill 2009115 Country: USA Study design: Before-and-after Data collection method: Analysis of referral rate data Aim: To evaluate a computer-based system to enhance referral for colonoscopy Detail of participants (number, any reported demographics): Patients aged 50 years or older with no record of having a colonoscopy in last 10 years. No detail of staff characteristics beyond description of outpatient practices varying in size, with smallest having two physicians and largest having 25 |
Intervention: 1. Pop-up prompt for screening colonoscopy on EMR modified 2. Also education sessions for primary care providers comprising epidemiology of colon cancer, strategies for early detection, how to use EMR and optimal clinic workflow to facilitate screening 3. Medical assistants asked to discuss screening with eligible patients before seen by physician and initiate preliminary order for test + best practice alerts, computerised documentation of referral status, individual physician feedback implemented later Number of hours: No detail of how long/many education sessions Delivered by who? NR Control: None Length of follow-up: Baseline January 2003, intervention through to July 2007 (4 years) Response and/or attrition rate: NA Context (from what/who to what/who): Primary care to colonoscopy service |
Outcome measures: Colonoscopy referral rate |
Main results: Individual site providers experienced very different local conditions and changes during the course of the project (e.g. relocation, new services, personnel change, introduction of revenue for screening site and physician from referrals) At baseline monthly referral rates 5–7% Pop-up prompt and provider education introduced over 2-month period showed little or no immediate correlation Initiation of MA workflow change 2 months later was associated with 11% increase in referral rate. Following 29 months all had referral rates above the baseline point (p < 0.001) Small increases observed after best practice alerts and computerised documentation of referral status implemented 2.5 years after initial intervention (no details of these intervention methods). Also small increases after unblinded individual physician feedback implemented 3 years later At 4-year point referral rates remained above baseline Wide variation in performance between providers, even those practising in the same clinic. Improved performance data mostly due to performance at the two largest clinics |
Reported associations between elements for logic model: Physicians responded differently to the interventions. Of those who did respond initially many did not sustain improvement and reverted to baseline Only a few demonstrated improvement trend over longer than 9 months Different local conditions and aggregated data masking individual differences between clinicians |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Malik 200741 Country: UK Study design: Audit Data collection method: Patient records/referral letters Aim: To determine if the 2-week wait referral guidelines had been followed, and what proportion of patients referred under the guideline had malignant tumours Detail of participants (number, any reported demographics): 40 patients |
Intervention: Referral letters were evaluated to see if they met Department of Health guidelines for referral of a suspected bone or soft tissue tumour Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): GP referral for cancer |
Outcome measures: Referral meets guidelines |
Main results: 40 patients were referred under the guideline between January 2004 and December 2005. Ten of these patients (25%) had malignant tumours, compared with 243 of 507 (48%) of those referred from other sources Most (31 of 40, 78%) ‘2-week’ referrals met the published referral guidelines. In 9 of the 40 cases, the patient did not meet the criteria for urgent referral. None of the nine patients had malignant tumours |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Mariotti 2008113 Country: Italy Study design: Audit Data collection method: Clinical data Aim: To evaluate a new method of prioritisation of patients suffering from significant GI disorders needing rapid access to diagnostic procedures Detail of participants (number, any reported demographics): n = 438 outpatients |
Intervention: GPs used a ranking of waiting times for different levels of clinical priority called homogenous waiting groups. Specialists also assigned a priority level for each patient as well as evaluating the appropriateness of the referral and the presence of significant endoscopic disorders. Agreement between GP and specialist was evaluated Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 7 months of intervention data Response and/or attrition rate: NA Context (from what/who to what/who): GP to specialist |
Outcome measures: Referral priority GP/specialist agreement |
Main results: Most referrals (74.4%) were deemed low priority by GPs, with no maximum waiting time assigned. The level of agreement between GPs and specialists as regards patients’ priorities was poor to moderate; for gastroscopy the kappa was 0.31, and for colonoscopy 0.44 There was an association between the proportion of significant disorders identified with endoscopy and the priority assigned to the referral (χ2 = 18.9, 1 df; p < 0.001). The overall proportion of referrals deemed inappropriate by specialists was 22.1% |
Reported associations between elements for logic model: There is value in liaison between GPs and specialists for achieving timely referrals and avoiding delayed diagnosis. High levels of agreement need to be achieved |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Matowe et al. 200250 Country: UK Study design: Before-and-after Data collection method: Admin system data Aim: To evaluate the effect of disseminating guidelines Detail of participants (number, any reported demographics): 376 GPs in 87 practices in Grampian. 117,747 referrals, mostly chest X-rays followed by limb and joint and then spine |
Intervention: Copies of Royal College of Radiology guidelines were mailed to GPs Number of hours: NA Delivered by who? NA Control: None Length of follow-up: Data for 3 years Response and/or attrition rate: NA Context (from what/who to what/who): GP to radiography |
Outcome measures: Data from two radiology departments – effect of intervention on total number of referrals – absolute change in referral, underlying trend, and change in referral trend Effect on investigations requested average more than 20 times per month |
Main results: Month of May had highest number of referrals; December had the lowest No significant effects of intervention on total number of general practice imaging requests. Total referrals decreased by 32 (95% CI –226.7 to + 291.4) in month following guideline dissemination while trend decreased by –1.82 requests per month (95% CI –11.8 to + 8.2). Referral decreased by average 1.2 per month for the entire 35-month period None of 18 examinations evaluated changed significantly after introduction of guidelines on time series analysis. Eleven of the 18 did show significant difference before and after introduction of the guidelines, however, with 10 having significant underlying trends |
Reported associations between elements for logic model: No effect of passive distribution of guidelines. Before-and-after studies may erroneously find an effect |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| McGarry 2009148 Country: Australia Study design: Evaluation survey Data collection method: NR Aim: To examine changes in patient management and referral for care following the BOiMHC initiative Detail of participants (number, any reported demographics): One hundred and thirty-three (33%) GPs responded |
Intervention: Significant government spending has resulted in substantial changes to the Australian primary mental health-care system. Initially producing the BOiMHC initiative, this has been replaced by the Better Access to Mental Health Care programme, which allows all GPs to refer patients for allied psychological health care under BOiMHC. Incentives commenced August 2002. GPs working in accredited practices who had completed accredited mental health training were able to receive service incentive payments (SIP) for providing care to patients with ICD-10-diagnosed mental illness. Trained GPs able to refer patients for psychological therapies to the Access to Allied Psychological Service (ATAPS) via divisions of general practice. GP Psych Support provides GPs with access to advice from psychiatrists via telephone, e-mail or fax BOiMHC: Commenced November 2006. All GPs regardless of training or practice location receive higher Medicare rebates to complete GP mental health plans for patients with ICD-10-diagnosed mental illness, as well as higher rebates for mental health consultations. Patients with an ICD-10 diagnosis and a GP mental health plan are eligible for Medicare rebates for psychological care, for up to 12 sessions per year (individual) and 12 sessions (group therapy) This study was a comparison of results of a 2006 postal survey of Australian GPs examining self-reported management of patients with depression with a similar survey conducted in 2001–2, prior to the BOiMHC initiative Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 4 years Response and/or attrition rate: 133 of 410 responded Context (from what/who to what/who): GP referral to mental health care |
Outcome measures: Referral to mental health care |
Main results: The main self-reported strategies for managing patients with depression were similar to the previous study: supportive counselling and medication Compared with the original study, significant differences between rates of formal training (short course, diploma, certificate, degree or work at the level of psychiatry registrar or above) were only found in Institute for Psychological Therapies (p = 0.03) and relaxation therapy (p = 0.03), with fewer responders reporting formal IPT training (current: 8.6%, 11/128; original: 17.5%, 24/137) and more reporting formal training in relaxation strategies (current: 18.6%, 24/129; original 9.3%, 13/140) in the current study. Otherwise there were no significant differences in rates of formal training Significantly higher rates of referral for psychological treatments were reported in 2006 than in 2002. Significantly higher proportions of responders in the current study reported referring half or more of their patients with mild to moderate depression for PST (p < 0.001) or cognitive–behavioural therapy (p < 0.001). In fact, significantly more responders reported higher rates of referral for most modalities than in the original study |
Reported associations between elements for logic model: While GPs’ main reported strategies for managing patients with depression were unchanged, reported referral for psychological therapies was significantly higher in 2006, possibly reflecting the impact of changes to the primary mental health-care system |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| McGowan 2008107 Country: Canada Study design: RCT Data collection method: Survey Aim: Evaluated whether or not information provided by librarians to answer clinical questions positively impacted time, decision-making, cost savings and satisfaction Detail of participants (number, any reported demographics): Physicians (93.2%; n = 82), with a small number of nurse practitioners (4.5%; n = 4), residents (1.1%; n = 1) and nurses (1.1%; n = 1) |
Intervention: The ‘just-in-time information’ librarian consultation service was designed to provide a rapid response to clinical questions during patient visit hours. The questions were submitted by the participants and each question was randomly assigned to the intervention (librarian information) or control (no librarian information) group. If the question was randomised to the control group, participants received a message within 1 minute that their question would not be answered. The librarian still answered the question, but the software blocked the response from being sent to the participant. Thus, they would need to try to answer the question themselves. The object of the randomisation was a clinical question Each participant had clinical questions randomly allocated to both intervention (librarian information) and control (no librarian information) groups. Participants were trained to send clinical questions via a hand-held device Control: No library information Length of follow up: Survey sent 24 hours after a question was submitted Response and/or attrition rate: A total of 110 individuals signed consent forms; 21 of these individuals withdrew from participation before randomisation, leaving a final group of 88 individuals who participated in the RCT Context (from what/who to what/who): GP to specialist |
Outcome measures: Impact of the information provided by the service (or not provided by the service), additional resources and time required for both groups |
Main results: The average time for ‘just-in-time information’ librarians to respond to all questions was 13.68 minutes/question (95% CI 13.38 to 13.98 minutes). The average time for participants to respond their control questions was 20.29 minutes/question (95% CI 18.72 to 21.86 minutes). Using an impact assessment scale rating cognitive impact, participants rated 62.9% of information provided to intervention group questions as having a highly positive cognitive impact. They rated 14.8% of their own answers to control question as having a highly positive cognitive impact, 44.9% as having a negative cognitive impact, and 24.8% with no cognitive impact at all In an exit survey measuring satisfaction, 86% (62/72 responses) of participants scored the service as having a positive impact on care and 72% (52/72) indicated that they would use the service frequently if it were continued |
Reported associations between elements for logic model: Providing timely information to clinical questions had a highly positive impact on decision-making and a high approval rating from participants |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| McKoy 200489 Country: USA Study design: Before-and-after Data collection method: Clinical data Aim: To evaluate the accuracy, access time, cost and acceptance by patients and physicians of an asynchronous teledermatology referral intervention in primary care Detail of participants (number, any reported demographics): n = 52 patients aged 25–89 years. 46% female |
Intervention: Primary care physicians in a multispecialty group referred patients for teledermatology consultation. Same-day history and digital images taken by a nurse were electronically sent to a dermatologist who returned a diagnosis to the referring physician Control: NA Length of follow-up: NA Response and/or attrition rate: 52 of 54 enrolled patients completed the study Context (from what/who to what/who): GP to dermatology |
Outcome measures: Diagnosis Referral |
Main results: History was adequate for diagnosis in 81% of cases; images were adequate in 75% of cases. Accuracy of the teledermatology diagnosis in cases with adequate images was 97%; accuracy for all cases was 92% A dermatology visit was recommended in 26% of cases with adequate images and in 42% of all cases Access time for a teledermatology opinion was 1.9 days, compared with 52 days for a regular dermatology appointment |
Reported associations between elements for logic model: Unclear |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| McNally 200374 Country: UK Study design: Retrospective data analysis before and after introduction Data collection method: Examination of case notes Aim: To assess the impact of a fast-track clinic Detail of participants (number, any reported demographics): 295 cases over a 6-year period; patients with primary ovarian cancer. 109 cases prior to intervention and 133 cases after |
Intervention: Clinic appointment within 2 weeks to fast-track clinic. Clinical referral criteria. GPs informed of the clinic and referral criteria by individual letter, GP newsletter, and meetings Number of hours: NA Delivered by who? NA Control: None Length of follow-up: NA Response and/or attrition rate: NA Context (from what/who to what/who): Referral for ovarian cancer |
Outcome measures: Time to diagnosis |
Main results: Median waiting time for referral to specialist was 3 days (range 0–188 days). This did not change significantly after clinic introduction (p = 0.05). The impact of fast-track clinic on referral and diagnosis time variables was not significant The fast-track clinic saw 10%, 20.1% and 10.3% of ovarian cancers diagnosed by the service during the first 3 years of operation. 13.5% of patients were referred to the fast-track clinic |
Reported associations between elements for logic model: Rapid access clinic may have some limited impact but may be underused |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Melia et al. 200851 Country: UK Study design: Before-and-after Data collection method: Request for GP data, data from pathology labs Aim: To evaluate whether or not guidelines for GPs impacted on GP referral for potential prostate cancer screening Detail of participants (number, any reported demographics): GPs referring to pathology lab in four study areas (Chichester, Sutton, Truro and York). 200 GP partners in 48 practices. Male patients aged 45–84 years, n = 1520 |
Intervention: Prostate Cancer Risk Management Programme (guidelines for GPs on age-specific prostate-specific antigen cut-off levels in asymptomatic men) Number of hours: NA Delivered by who? NA Control: None Length of follow-up: 1–2 years pre intervention to post Response and/or attrition rate: 48 of 69 practices invited took part (70%), 79% of patients’ baseline data, 90% at intervention Context (from what/who to what/who): GP to urologists |
Outcome measures: GP demographics Awareness of receiving guidelines pack Proportion of asymptomatic men with raised antigens referred to urologists |
Main results: Awareness of pack acknowledged by 112 (56%) of GPs, 24 unaware, 64 did not know. Awareness not significantly different by area, age, gender, MRCGP registration, number of years working or number of sessions per week of GP Proportion of asymptomatic men referred who had raised antigen levels did not increase significantly from baseline to intervention (24% pre intervention, 29% post p = 0.42). No significant difference in referral rate by area (p = 0.33) |