Notes
Article history
The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 15/77/10. The contractual start date was in August 2016. The final report began editorial review in August 2017 and was accepted for publication in November 2017. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.
Declared competing interests of authors
Andrew Booth is a member of the National Institute for Health Research Complex Review Research Support Unit funding board.
Permissions
Copyright statement
© Queen’s Printer and Controller of HMSO 2018. This work was produced by Baxter et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.
2018 Queen’s Printer and Controller of HMSO
Chapter 1 Introduction and background
Rationale
It has been argued that the growing financial and service pressures in the NHS cannot be tackled without transforming how health and social care are delivered. The NHS Five Year Forward View1 sets out a view on how services need to change and what models of care will be required in the future, with a common thread being the need to break down barriers between services through greater integration of care. 2 It is proposed that there should be new networks of provider organisations (organised horizontally as multispecialty community providers or vertically as primary and acute care systems), which form the bedrock of provision. 3 Thirty-seven ‘vanguard sites’ have been identified to develop and test new models of care, including enhanced health in care homes and new approaches to urgent and emergency care as well as multispecialty community providers and primary and acute care systems. The rationale underpinning the development of the vanguard sites programme is to evaluate a small number of different models while allowing flexibility in the way that models are implemented to meet local needs.
A study by The King’s Fund4 highlighted that change was needed at a whole-system and governance level, with the development and rapid implementation of integrated models of care requiring resources and expertise from across the local health system. This primarily qualitative work analysed five case study sites that have developed more integrated models of care. The significant barriers to implementing changed systems included a lack of clarification of roles and responsibilities, competition for funding and challenges in engaging primary care providers. The authors of this work also highlighted that ‘one size does not fit all’, with different models developed at each of the case study sites. Common themes regarding the requisite types of changes included delivering more care beyond the hospital walls, changing the future size and shape of acute hospitals, an increased role in prevention and population health and new organisational models with local partners.
Other studies5 have similarly reported that cultural and structural barriers have proved difficult to overcome in the drive to provide integrated care. A key challenge is to create an environment of collaboration between all providers, including primary and secondary care providers. 3 Authors have highlighted the need for greater understanding regarding how integrated care can be best delivered, and a requirement for further clarity regarding how new models of integration may have an impact on patient outcomes. 6
It has been suggested that models from the USA, such as Accountable Care Organizations, may be useful frameworks from which the NHS can learn. However, it is important to consider the difference in contexts before implementing the same models in different areas. 5 The NHS vanguard sites have been encouraged to develop new models of care by adapting systems to local needs and configurations. 7 However, commentors note that although individual models should be rooted in local communities and have a ‘local resonance’, it is important to identify simple standard approaches and products that can be replicated across the country. 7
The proposed study intended to add to the existing body of knowledge by providing a critical summary of the international literature on new models of care. The review of the literature aimed to provide knowledge regarding key ingredients of care models and identify best practice and areas of learning that may be important in contributing to the success of a programme. It also intended to detail potential outcomes and impacts reported in the literature, which would inform the planned multifaceted approach to the measurement and evaluation of new care models in the NHS.
The proposed methods to synthesise the review findings and add to the body of knowledge were selected to illuminate complex pathways between models of care and long-term health impacts. The work aimed to provide an accessible summary of the literature and inform the logic models being developed within vanguard sites. The proposed study particularly sought to analyse factors related to generalisability and applicability in order to provide key information to commissioners, service managers and practitioners regarding the implementation of care models in their local context. The focus of this element of the work was to support understanding of how care models might be replicated in other local care systems.
Objectives
The proposed study aimed to carry out a rigorous and inclusive systematic review of evidence underpinning new models of health care. It had the following specific objectives:
-
To carry out a systematic review of the international literature on new models of health care.
-
To use a logic model method to outline mechanisms of change underpinning the introduction and outcomes from new models of health-care delivery, including potential barriers and facilitators.
-
To explore how the developed model resonates with the views and experiences of key stakeholders.
-
To develop a framework that details factors that may have an impact on the generalisability or applicability of the research literature, and to use this framework to evaluate the models of care reported.
Research questions
The research questions for the study were:
-
What can be learned from the international literature regarding the key elements of implementation and the potential impacts of new models of care?
-
What are the reported mechanisms of change and outcomes and impacts associated with new models of care?
-
How applicable are the findings from the international literature to different local populations and contexts, and how might mechanisms of change operate differently in differing local services?
Changes from the protocol
The study was carried out in line with the protocol.
Chapter 2 Methods
The proposed study combined established systematic reviewing methods for the identification of literature, with innovative methods of analysis and synthesis to examine mechanisms of change, generalisability and the applicability of international evidence to local contexts.
Identification of literature
Search strategy
A systematic search of key health, medical and social care databases was undertaken to identify relevant studies published from 2006 onwards. We searched from this year as a previous review8 is available that included relevant studies published up to 2006. We were aware (and our scoping review had confirmed) that there was potentially an extensive amount of literature available in the area; we intended that, by including previous systematic reviews, we would capture the main findings from primary studies pre 2006.
Search terms included a combination of medical subject heading (MeSH) (health-care reform, organizational innovation, quality improvement, health priorities, Accountable Care Organizations, delivery of health care and integrated) and free-text [care model(s) and ‘new’ service delivery model(s), health-care model(s), transformation of service or care, integrated care, integrated health system(s), vanguard(s), accountable care, future-proofing, service redesign and ‘five year forward plan’] terms, with the search strategy led by the information specialist on the team. An outline of the search strategy is provided in Appendix 1. The search was limited to studies of humans and those that were published in the English language.
Sources searched
Searches were conducted in the following sources between September and October 2016:
-
MEDLINE via OvidSP (1946 to September 2016)
-
MEDLINE In-Process & Other Non-Indexed Citations, MEDLINE Epub Ahead of Print and MEDLINE without Revisions via OvidSP (2013 to September 2016)
-
EMBASE via OvidSP (1974 to September 2016)
-
Cumulative Index to Nursing and Allied Health Literature via EBSCOhost (1981 to September 2016)
-
PsycINFO via OvidSP (1967 to September 2016)
-
Cochrane Database of Systematic Reviews via The Cochrane Library (2005 to September 2016)
-
Database of Abstracts of Reviews of Effects via The Cochrane Library (1994 to April 2015; no longer updated, archive only)
-
Cochrane Central Register of Controlled Trials via The Cochrane Library (1898 to September 2016)
-
Health Technology Assessment database via The Cochrane Library (1989 to September 2016)
-
Science Citation Index via Web of Science (1900 to September 2016)
-
Social Sciences Citation Index via Web of Science (1956 to September 2016).
The search process was recorded in detail, with lists of databases searched, date of search, limits applied, number of hits and duplication as per Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.
In addition to standard electronic database searching, other iterative searching techniques were employed, including citation searches of a key review8 and hand-searching of the reference lists of primary studies and other reviews. We searched for grey literature using reference lists and UK websites including those of The King’s Fund [www.kingsfund.org.uk (accessed 3 October 2017)] and NHS England [www.england.nhs.uk (accessed 3 October 2017)]. In May 2017, we conducted a citation search to identify any further literature that might have appeared subsequent to the formal bibliographic searches.
Study selection
Inclusion and exclusion criteria
The following criteria were used to set the parameters of the review:
-
Target population – patients receiving a health-care service and staff delivering services.
-
Target interventions – numerous definitions of integrated care are in common use. In 2011, Shaw et al. 9 reported that around 175 definitions and concepts exist. For the purposes of this study, we drew on the work by Shaw et al. ,9 which highlighted the importance of delineating integrated care from integration processes, and we defined new models of care as ‘changes to service delivery which aim to increase integration and co-ordination’. Examples of potential interventions identified in our scoping search that we aimed to include were:
-
interventions with a focus on service redesign/reconfiguration, such as those with a single point of access, joint clinics or sessions, integrated care pathways (ICPs) or relocation of services
-
interventions with a focus on workforce changes, including changed roles, and provision of further education/training to assist workforce change
-
interventions with a focus on the integration of different services or working across service boundaries
-
initiatives to encourage greater co-operation between services (e.g. using common assessments)
-
interventions with a focus on integration of financial strategies
-
information systems or other technology to promote new ways of integrated working.
-
-
Controls/comparators – the review examined interventions with comparator groups (such as care networks) and those with no comparator.
-
Outcome measures – as one of the objectives of the work was to identify and report measures of outcome and impact that have been used in the literature, we made the decision to not specify a priori the outcome measures that would be included. We therefore included studies with any outcome related to the delivery of services (effectiveness or efficiency measures) or that reported an impact on the delivery of patient care or on staff delivering services. These outcomes included quantitative measures, together with views and perceptions of patients/service users, staff and other key stakeholders.
-
Study design – we included systematic reviews, randomised and non-randomised controlled trials, prospective and retrospective cohort studies with and without comparators, and other before-and-after/longitudinal studies. As a result of the increasing recognition that a broad range of evidence is needed to inform the depth and applicability of review findings, the review encompassed both experimental and observational studies, together with qualitative work reporting views of service users or staff delivering services. Descriptive or discursive papers were excluded.
-
Other criteria – we included studies from any of the 35 countries that are members of the Organisation for Economic Co-operation and Development, as studies in these developed countries are of most relevance to UK health systems. We examined studies published in English; however, we considered translation if any key international papers that had abstracts in English were identified.
The screening process
Retrieved citations were uploaded to EndNote [Clarivate Analytics (formerly Thomson Reuters), Philadelphia, PA, USA], and titles and abstracts (when available) of papers were screened by three reviewers against the inclusion and exclusion criteria, with any queries regarding inclusion being discussed by the full team at regular (fortnightly) team meetings. We initially independently blind-screened citations that had been identified in the scoping search (n = 7256) to establish the degree of consensus between reviewers. The level of agreement between the reviewers regarding citations that should be included or excluded following independent sifting was > 95%, with differing decisions related to overinclusion (when one reviewer had inappropriately suggested inclusion) rather than excluding potentially relevant studies. Further screening was therefore carried out by a lead reviewer, and other team members checked a sample of the citations.
During the screening process, we developed a coding list for marking and provisionally categorising citations of relevance, distinguishing studies that appeared to report initiatives or interventions from the UK, reported UK-based qualitative data, appeared to be systematic reviews, and intervention and qualitative studies from other countries.
Following screening of the database, full-paper copies of citations that were identified as potentially relevant were retrieved for systematic screening. Papers that were excluded at this full-paper screening stage, and details regarding the reason for exclusion were recorded (see Report Supplementary Material 1).
The data collection process
Studies that met the inclusion criteria were read in full and a data extraction was completed for each. A data extraction form was developed using the previous expertise of the review team and trialled on a sample of papers of different study designs by each reviewer. The form was designed to be suitable for all types of primary study design, with a slightly modified version being developed for the extraction of systematic reviews. In privileging higher-quality international studies alongside the UK evidence, we carried out a ‘light touch’ extraction for non-UK studies that did not have comparator group designs or were qualitative studies. We identified and noted where these studies provided data that supported our findings, or where additional or contradictory data were present.
We particularly aimed to extract data relating to factors that might influence the applicability of study findings and, therefore, included an item specifically on the context of the study, and also another category for data in which an author had identified factors that may influence applicability or generalisability. We also included an element highlighting reported associations, which was intended to inform the development of the logic model. The extraction tables were completed by one reviewer and checked by a second. The completed extraction forms for each study are available in Report Supplementary Material 2.
Data items
The extraction form collected data on first author/year, study design, sample size, population characteristics (type of group, condition/department, sex, age and other details reported), context, data collection method, outcome measures, type and details of the intervention, summary of results, main author conclusions, reported associations and potential factors related to applicability. The slightly modified systematic review form included the number of studies included in the review and details of countries and other inclusion criteria. We used a ‘light touch’ extraction form for the international non-comparator and qualitative studies (examined later in the process than the UK, international comparator and systematic reviews), which detailed author, study design and participants, with a column for recording data supporting the results from the other included studies and a column for data that were additional or contrasted with the other findings.
Risk of bias in individual studies
The critical appraisal of included evidence is considered a key part of the review process, although it remains the subject of debate in the field, with no single recognised tool. There is also variation in views regarding the use of scoring systems and methods for appraising strength of evidence across studies (particularly in qualitative work). The approach to appraisal of risk of bias and quality of evidence in the proposed study needed to take account of the wide range of study types that we were likely to find. This was particularly important given that the most rigorous evidence base [in the form of randomised controlled trials (RCTs)] was likely to be dominated by studies from outside the UK, whereas the most relevant studies to answer our research question would be those derived from the UK.
Our approach to consideration of individual study quality and risk of bias in individual studies was therefore based on the established hierarchy of study design, together with use of a variety of checklists designed for each study type. For studies reporting evaluations of integrated care initiatives using controlled designs, we considered sources of potential bias as recommended by Higgins and Green10 (selection bias, performance bias, attrition bias, detection bias and reporting bias). For studies using before-and-after (pre–post) designs with no comparator group, and for systematic reviews, we used the two US National Institutes of Health checklists11 (although we concurred with the Cochrane view10 that scoring/totalling of individual elements is not recommended as individual items may have differing relative importance).
For qualitative studies, we used the Critical Appraisal Skills Programme checklist. 12 Any concerns regarding study quality were identified and recorded using these checklists (the National Institute of Health checklists and the Critical Appraisal Skills Programme checklist) and fully considered during the synthesis of findings. Appendix 2 provides the completed study appraisal checklists. International studies of non-comparator group designs were not quality assessed in line with the ‘light touch’ approach to their inclusion.
Risk of bias and strength of evidence across studies
The summarising of quality and strength of evidence within and across studies is a source of debate in the field of systematic reviews, with the calculation of overall scores discouraged. 10 However, indicating where there is greater strength or confidence in the evidence provides an important indicator for the interpretation of the results of a review. 13 Following assessment of the individual quality and risk of bias of each study, we therefore considered the literature in terms of risk of bias and whether there was a greater or lesser strength of evidence underpinning the outcomes reported.
A key limitation when evaluating the strength of evidence is that a higher number of papers in an area does not necessarily indicate greater strength of evidence; it only indicates that there is a greater amount of research activity. Therefore, it is important that any overall assessment considers not only quality and number of studies, but also considers consistency of the evidence. 14 For this study, we drew on work by Hoogendoorn et al. ,14 together with principles from the GRADE (Grading of Recommendations Assessment, Development and Evaluation)15 and GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative research)13 rating schemes, and our work from a previous systematic review with diverse evidence16 to consider risk of bias and overall strength across studies.
It is important to note that we deliberately used comparator labels for the rating (stronger vs. weaker), therefore strength for each element is relative. Owing to the nature of the interventions, we knew that there were few examples of studies that were able to achieve the ‘gold standards’ of blinding and randomisation and, therefore, provide strong evidence. However, by adopting a relative evaluation, we were able to provide information regarding where there were stronger versus weaker signals in the data ‘noise’.
It is also important to note that we recorded study outcomes in terms of either ‘increase’, ‘reduction’ or ‘no significant difference’, as for many outcomes the perception of whether it is positive or negative depends on the point of view. For example, an increase in service usage may be positive for patients or the service, but may also be negative in terms of costs or detrimental effect on other services. An appraisal of strength of evidence was undertaken by the research team at a series of meetings to establish consensus.
We tabulated the data in terms of outcomes reported and source of the data (comparator study, non-comparator study or systematic review), and applied a rating scale to give an indication of where the evidence was stronger or weaker for each outcome. We separately rated evidence from the UK studies, systematic reviews, international comparator studies and international non-comparator studies and then provided an overall rating across the study types. The rating scale was as follows:
-
stronger evidence = generally consistent findings in multiple studies with a comparator group design or three or more systematic reviews
-
weaker evidence = generally consistent findings in one study with a comparator group design and several non-comparator studies, two systematic reviews or multiple non-comparator studies
-
very limited evidence = a single study is available or perceptions of change only
-
inconsistent evidence = inconsistent findings in multiple studies (< 75% of studies report the same outcome).
Summary measures
It had been proposed that a meta-analysis calculating summary statistics would be used if heterogeneity permitted. However, the wide variety of interventions/initiatives included, with the majority of these containing multiple and complex elements, together with the vast range of outcomes measured, counterindicated the calculation of statistical summary measures to compare effectiveness.
Synthesis of results
We used several methods to synthesise the findings from the included studies. These methods included narrative synthesis, tabulation, use of a logic model framework and consideration of the findings within an applicability framework.
The literature on new models of care presents two key challenges for systematic review synthesis methods. The first challenge arises from the increasing recognition that any intervention in health care can be considered to be complex, with individual and organisational factors affecting if and how interventions lead to improved outcomes. 17 Interventions such as new care models, which act at a system or organisational level, or even across organisational boundaries, provide considerable additional complexity because of their multifactorial processes. The new care models programme has been described as being ‘complex in its breadth and depth’. 7 This complexity presents difficulties for systematic review methodologies that seek to quantify or report clear intervention outcome effects.
A second challenge to the review and synthesis of this literature comes from the variety of models that have been introduced. This diversity in the type of models may be the result of the development of the models in response to differing local needs and configurations. New models of care may adopt different approaches and organising principles that respond to the local context. This diversity presents challenges for considering how applicable the evidence from varied national and international research may be to a particular local context. Our methods of synthesis were therefore required to overcome these two key challenges, and were also needed to enable the integration of a wide variety of different forms of evidence.
Narrative synthesis
Quantitative studies
Data were synthesised with use of tabulation and graphs, in addition to narrative summary of interventions, outcomes and influencing factors. We initially scrutinised the included effectiveness studies that originated in the UK, before an examination of systematic reviews from any country, followed by international studies with comparator group designs. We anticipated that UK studies would provide evidence of most relevance to the NHS, so these were examined first. We included UK effectiveness studies of any design in the synthesis as data from these were considered to be of most relevance, whereas we focused on international systematic reviews and international primary studies with a comparator design, as these were considered to have more rigour. We completed a ‘light touch’ analysis of the international non-comparator studies and qualitative studies, whereby we focused on where data echoed the findings from the other studies and aimed to highlight new or dissonant data. In the synthesis, we explored similarities and differences between findings from the UK and those from other countries. Subgrouping and examining characteristics including the age of participants, intervention content and delivery agent were examined where the number of studies permitted. Relationships between studies and outcomes within these typologies were scrutinised. The heterogeneity of the interventions and outcomes precluded summarising the studies via meta-analysis.
Qualitative studies
Qualitative data were synthesised, drawing on thematic synthesis methods18 to identify recurring perceptions within the data. This method comprised extracting themes reported by authors within the finding sections of studies (which constitute the ‘data’ for the synthesis) and then comparing the themes across studies. The qualitative papers were read in detail and themes noted by the authors were extracted. When data from different studies described the same view or perception, they were combined into a single theme (e.g. data from different studies describing the influence of staff having different status were categorised within an overarching ‘power and status’ theme). When data could not be combined into existing categories, additional themes were added. The ‘views and perceptions’ data form an integral part of the overall logic model data synthesis. In particular, the qualitative data make a specific contribution to the elements of the model that document influencing factors, but are also reported alongside quantitative data outlining measured outcomes, supplementing this evidence with data regarding perceived outcomes.
The use of both qualitative and quantitative data in a single review has potential to shed light on negative trial results, identify social factors, examine issues of implementation and assist in the interpretation of significance and applicability for practitioners and service planners. We intended that our mixed-methods approach to synthesising the data would enable the strengths of each form of evidence to be utilised, in order to understand highly complex interventions and outcomes within health-care systems.
Logic model framework
We employed a logic modelling method to provide a systems perspective to data synthesis. This method, developed by the team,19 has been used successfully in previous systematic review studies20,21 and is ideally suited to the analysis of complex, system-based interventions. Logic model methods provide a graphical description of a system and are designed to identify important elements and relationships within that system. 22 They constitute one form of theory-based evaluation as a means of relating hypothesised links between an intervention and its constituent parts to its outcomes and long-term impacts. Logic models examine the processes of implementation, mechanisms of change and participant responses in order to develop hypothesised links or a ‘theory of change’. 23 They serve as a tool to represent the causal system of interest, set out proposed causal pathways in the relationship between the intervention and its health-related and other outcomes, and identify potential moderators of that relationship. 22 Outcomes are conceptualised as the end of a chain of intermediate changes that the evaluation process seeks to track, with each intermediate point predicting potential future outcomes. 24 Theory-based approaches focus on assessing the validity of the theory on which an intervention is built and are concerned with opening up the ‘black box’ of interventions and outcomes to uncover underlying mechanisms. 25 It is argued that without a clear understanding of the assumptions underlying an intervention and how it is supposed to work, evaluators cannot ascertain whether or not it did work and why it did or did not achieve the intended benefits. 26 The key strength of logic models lies in linking complex system processes to system outcomes, and thus guiding the development of strategies and research tools for making system improvements. 27
We anticipated that a logic model approach to synthesis was well suited to analysing and reporting this literature, as well as examining the mechanisms and impacts underpinning new models of service delivery. This approach to synthesis also resonates with the logic models that the NHS vanguard sites have been developing.
In the logic model method, data from included studies are extracted and analysed to produce elements of a framework that typically comprises an intervention typology, detail regarding the range of outcomes reported, factors that may be influential in the pathway from an intervention to health impacts, and reported associations between elements of the model. The extraction table includes the collection of data regarding associations and outcomes, which facilitates the construction of the framework. Both quantitative and qualitative data underpinned the construction of the elements of the model. The completed model aimed to outline evidence in the literature regarding different models of care, relationships between contextual factors, inputs, processes and outcomes. 22,28 Each element in the model demonstrates the logic or theory of the interventions and portrays ‘if–then’ relationships in the chain from new models of care to system-wide impacts.
Initially, the logic model synthesis used UK-published literature to develop an initial or ‘start’ logic model. Following the development of the UK-based logic model, the second phase of the work entailed examination and synthesis of the international (non-UK) literature to add to a further draft of the model and to enable comparisons and contrasts between the identified evidence. Initially, systematic review-level international studies were added to the model alongside the UK evidence, followed by international primary evidence from the more rigorous comparator design studies and then non-comparator studies to explore gaps in the evidence base. The logic model framework was therefore intended to act as a translation tool between findings from the review of the UK evidence and the international literature, and to enable contextually sensitive synthesis of multiple and diverse evidence types.
Applicability framework
In a parallel phase of work, we developed a framework of factors related to applicability and generalisability from examination and synthesis of the included studies. As outlined earlier in The data collection process, we had included specific sections on context and applicability in our data extraction form to facilitate identification and synthesis of these factors. The development of the applicability framework was intended to inform our analysis of the data and reporting of the systematic review. We planned to use the framework to evaluate and present the evidence with regard to factors that may influence generalisability and applicability of the international literature to varied local contexts. A recent review29 of the assessment of applicability during systematic reviews recommended that authors of reviews should consider applicability for at least one target population, setting and context. We aimed to go beyond this recommendation and develop a framework to act as a reporting tool that would be suitable for the complex organisational interventions that we were evaluating.
The terms ‘generalisability’ and ‘applicability’ are often considered to be synonymous; however, ‘generalisability’ (synonymous with external validity) is usually used to describe how relevant the results of a study might be to general sites and populations, whereas ‘applicability’ typically refers to information regarding the study processes and insights into if and how an intervention may be implemented elsewhere. 30,31 Numerous checklists that include items related to external validity are available, although there is little detail regarding how to report generalisability. 32 The understanding of how evidence (particularly international evidence) is, or is not, applicable to varying local situations and contexts is a key challenge for systematic review synthesis.
In order to develop our applicability framework, we carried out an initial literature review of existing studies using terms such as ‘applicability assessment’ in MEDLINE, and using Google/Google Scholar (Google Inc., Mountain View, CA, USA). The review was not intended to be a rigorous systematic review but to identify any existing examples of frameworks that we could learn from. We found little literature on applicability frameworks; however, a review in 201132 identified 11 assessments of applicability or transferability. None of the methods of assessment were found to be comprehensive, with little evidence regarding their value for systematic reviews. The most well-developed method formed part of a Canadian knowledge translation toolkit,33 which was a useful model but not sufficiently detailed for our purpose. In the absence of existing tools that would suit our needs, we developed a new framework by identifying elements related to applicability within the data extracted from the included studies. We tabulated all of the references to applicability that had been noted on the extraction for each study in a Microsoft Word 2013 (Microsoft Corporation, Redmond, WA, USA) document, and then, through a process of cutting and pasting, brought similar text together (e.g. text related to population factors and to the type of context). In a process akin to content analysis, we developed categories and subcategories within the framework. Following the development of a succession of drafts, we consulted with users of systematic reviews (see Stakeholder involvement) to gain views regarding the clarity of our items. In response to the feedback that was received, we further refined and developed the framework to produce the final version.
During the synthesis, the framework was used as an analysis and reporting tool, guiding examination of the evidence within each element. Studies were grouped and tabulated according to framework elements, with examination and reporting of evidence using these items.
Protocol and registration
A review protocol was developed prior to the commencement of the study. The protocol outlined the research questions and detailed the methods for carrying out the review. The protocol encompassed methods for identifying research evidence, the method for selecting studies, the method of data extraction, the process of assessing the methodological rigour of included studies, and synthesis methods. The protocol was registered with the PROSPERO database number 37725 and was made available on the PROSPERO website [www.crd.york.ac.uk/prospero/display_record.php?RecordID=37725 (accessed 19 February 2018)].
Patient and public involvement
We formed a project-specific public involvement advisory group for the duration of the work, with 10 members drawn from across England. Group members were recruited via our meetings with patient advisory groups during the development of the study proposal, from members of advisory groups linked to the local NHS trusts and from advertising on the People in Research website [www.peopleinresearch.org (accessed 25 September 2017)]. We intended for the advisory group to have varied participants in terms of characteristics such as age, sex, health conditions, disability, experience of health care and geographical region. We were successful in achieving this, and our group included both retired and employed people, those with health conditions/disabilities and those with no particular health concerns, and people from the south east, west, central and north-west regions of England, in addition to those who were local to Yorkshire.
The group met three times during the project: in the second month, the seventh month and the 11th month. The involvement of public advisors was facilitated by reimbursement for time and travelling expenses, use of teleconferencing for those unable to attend in person and ensuring that any documentation was sent several weeks in advance of meetings. At the first meeting, the group finalised the wording of the Plain English summary for the NIHR website; discussed experiences of the integration of care, terminology and understanding of the study; and considered areas or issues of most interest to the public. At the second meeting, the intermediate findings were discussed, including formats for making the findings understandable (such as types of visual representations) and potential areas to highlight for a public audience. At the third and final meeting, discussion included the content of the Plain English summary, maximising meaningful messages in the report for lay readers and suggestions for disseminating the findings of the study so as to reach the wider public.
Stakeholder involvement
We accessed knowledge and expertise from stakeholders during the study at key points in the process, including consultation regarding items for the applicability framework and regarding the logic model and findings of the review. We held individual and group meetings to discuss the applicability tool with managers, clinicians, researchers and commissioners during its development, which formed the basis of several iterations. In particular, this consultation informed the grouping of items and the terminology to be used.
In the later stages of the study, we convened a workshop that was attended by 15 delegates, including researchers, NHS staff, commissioners and managers, to explore how the findings of the study resonated with experiences; where the analysis should focus, including key questions to explore; and to discuss the draft logic model and applicability framework. Feedback regarding the draft model was positive, with agreement that it was coherent and provided a clear summary of a highly complex area. Suggested areas of focus had already been identified within our draft applicability framework, although delegates suggested that differentiating interventions with more than three elements and interventions with three elements or fewer may be helpful, as it had been reported that organisations could best cope with a maximum of three elements of change at one time. Workshop attendees also suggested distinguishing between studies in which patient satisfaction was patient-reported and those in which it was staff reported, and to explore whether or not complex conditions compared with non-complex conditions were possible to identify in the analysis.
In the last month of the study, we also provided a seminar for 18 primary care staff members, including researchers and general practitioners (GPs), in which we outlined the findings of the study and sought input regarding the clarity of our approaches to synthesis and key questions for the analysis to explore. Feedback regarding the clarity of the logic model was very positive, and attendees found the strength-of-evidence reporting to be helpful and clear. The group were particularly interested in the results for particular patient subgroups, and commented that outcomes for patients seemed to be under-reported in the literature. Other comments related to the data on organisational and financial change, which were perceived to be of interest by stakeholders.
In the last month of the study, we presented the work to 25 commissioners and senior practitioners; the model and findings were discussed, and feedback on the logic model and presentation style was sought. The group appreciated the clarity of the strength-of-evidence indicators and the wide extent of the review. Further clarity regarding the source of the finding in relation to perceived improved quality (staff or patients) was suggested, and feedback regarding the checklist of factors to be considered during the development of initiatives resulted in further refinement. The group identified particular challenges of the evidence for them, including potential conflicts between the vision of new models and required outcomes, which provided helpful input for the study reporting.
Chapter 3 Results
Study selection
From a database of 13,323 unique citations, we included 267 documents in the review. Figure 1 is a PRISMA flow diagram that illustrates the study selection process.
Type of studies excluded
The lists of studies that were excluded at the full-paper selection stage and reasons for their exclusion are provided (see Report Supplementary Material 1). During the study selection process, we encountered particular challenges regarding the definition and reporting of intervention type. The term ‘new models of care’ lacks a precise definition and was used variously by authors. In particular, many retrieved citations that were excluded during the initial sifting related to clinical care models rather than service delivery models, such as initiatives aiming to enhance patient self-management delivered by a single practitioner. The term ‘integration’ was often used to refer to different types of clinical treatments or approaches, rather than integration of service delivery. In addition, the term ‘models of care’ often described studies examining particular clinical approaches rather than relating to any aspect of integration. Studies describing ‘case management’ approaches also required careful scrutiny. Such studies often described interventions with a key worker; however, there was no integration between key workers and services, or apparent integration between different services. For other ‘case management’ studies, enhancing integration between services was the key element of the approach.
In addition to the need for close scrutiny of the ‘intervention’ parameters of our inclusion criteria, the ‘outcomes’ inclusion criteria also required careful study selection. The review was focusing on service delivery and care outcomes, rather than clinical outcomes; however, this distinction was not always clear cut and decisions on the inclusion of these documents often required discussion at a team level.
Study characteristics
Of the 267 documents included in the review,34–299 54 were UK studies reporting the effectiveness of interventions34–87 and 64 were qualitative studies from the UK. 88–151 The over-representation of UK literature reflects to some extent the greater inclusion of studies of lower-quality designs from the UK, together with grey literature. It may, however, indicate the attention that has been paid to integration in the NHS, with nationally funded programmes including evaluation components.
We identified and included 43 systematic reviews8,152–193 and 49 higher-quality studies from outside the UK using comparator group designs. 194–242 In addition, 20 lower-quality non-UK studies (no comparator group)243–262 and 37 qualitative studies from outside the UK263–299 were included within a ‘light touch’ analysis. Figure 2 provides a summary of the country of origin for the different types of study design.
The majority of the systematic reviews included only papers published in English and/or from developed countries. Four reviews153,155,157,180 stated that they placed no restriction on country of origin or language.
Within the group of UK studies, several papers and reports originated from the integrated care pilots. 45,101,126 In particular, nine papers73,74,80,100,115,122,123,134,151 were derived from the work in north-west London. There were four papers81,87,106,144 that reported on the Partnerships for Older People Project. There were two papers48,142 on the Evercare programme.
The international primary studies report an array of programmes and initiatives, although there are several studies that are reported in two or three papers. For example, there are two papers on the System of Integrated Care for Older Persons programme. 196,197 In total, 202 unique studies are represented in the set of 267 included papers.
We examined the studies included in the systematic reviews to ascertain the extent to which we may be ‘double counting’ papers. We observed little overlap between primary studies and reviews because of the publication dates of the reviews, meaning that the majority of their primary studies were published prior to our inclusion period of post 2006. One review,179 however, had four qualitative papers126,142,146,147 that we had included and three reviews182,183,185 each had three papers8,45,49,87,100,109,125,154,196 in common with ours. Two reviews177,189 each had two overlapping papers48,168,196 (the Gravelle et al. 48 paper was included in both). Three reviews181,184,300 each included a single overlapping paper. 108,168,171
The literature in the review comprised predominantly journal papers, although 12 grey literature reports34,45,49,80,81,87,134,139,147,151,178,294 were included.
Risk of bias within studies
We assessed the methodological quality and risk of bias across the UK studies, international reviews and the international comparator studies using assessment tools that were appropriate to each study design. Appendix 2 contains these assessments. For the UK literature, 16 studies39,42,44,45,48,52,54,58,63,74,77,78,81,82,85,86 used higher-quality comparator designs. Of these studies, only two58,63 had utilised some form of random allocation to condition, and the nature of the intervention precluded allocation concealment within studies. Blinding of participants and personnel was also limited or not possible, with only four studies44,45,63,86 achieving this. Blinding of outcome assessment had been achieved in five studies. 45,48,52,58,63 The studies fared better with regard to completion of outcome assessment, and reporting was assessed as being accurate for all but one study,58 which had insufficiently discussed the study limitations. Overall, the studies were all considered to be at risk of potential bias, with none meeting the six criteria for reducing potential sources of bias.
The international comparator studies were rated slightly better in terms of randomisation: 19 studies (reported in 26 papers194–197,201–206,209,210,214,215,217,220,222,225–227,230,233,234,239,241,242) had a random allocation, although only nine studies (reported in 14 papers194,196,197,201–203,205,206,209,210,217,239,241,242) achieved allocation concealment. As with the UK studies, blinding was problematic because patients were unable to be blinded to their study arm. The incomplete reporting of outcomes data meant that in many cases it was not possible to judge the extent of attrition; for three studies (reported in six papers201–203,214,223,224), a large loss to follow-up was reported. Reporting was poor in around one-third of the studies, making it difficult to judge the extent of possible selective reporting. Other limitations included small sample sizes leading to inadequate statistical power, with some concerns regarding the processes of allocation. As with the UK comparator studies, none met all of the criteria for the highest quality.
The quality assessment illustrates the considerable challenges that are inherent in rigorous evaluation of highly complex health-care initiatives such as these, in particular, the difficulties of meeting requirements for randomisation and blinding.
The UK non-comparator before-and-after/longitudinal studies demonstrated similar issues regarding blinding, with only one study80 clearly reporting that outcome assessors were blinded. The studies were generally considered to have recruited participants who were representative of the population of interest, although it was often difficult to ascertain the recruitment process. Just over half of the included studies clearly reported sample sizes that were sufficiently large enough to have confidence in the findings. Only one-third were judged to have clearly described the intervention and its delivery. None of the studies reported taking measurements at multiple time points prior to the intervention and only just over half used statistical measures (such as p-values) to evaluate change over time. The UK qualitative studies were generally of a satisfactory standard, with a minority providing only limited data or containing weak methods sections. Four studies92,93,110,135 met all of the quality criteria. Fifteen89,94,95,97–99,107,108,117,120,130,131,134,136,145 of the qualitative studies met all but one of the criteria.
The systematic reviews exhibited variable quality, with none meeting all 11 quality criteria and only three165,168,189 meeting all except two of the criteria. None of the systematic reviews assessed the likelihood of publication bias, fewer than half considered the quality of studies in their conclusions, around half searched at least two databases and one-third reported that they searched for grey literature.
Synthesis of results
Given the complexity of the interventions, processes and outcomes involved in new models of care, we have summarised the findings of the review using a logic model that sets out the integrated care systems, processes and outcomes identified (Figure 3). Through a mixed-methods approach, rather than separating the quantitative and qualitative data into separate sections, we have integrated and drawn on findings from both the studies that evaluated interventions using quantitative or mixed methods (‘intervention studies’) and studies that examined the views and perceptions of staff or patients (‘qualitative studies’), when they provide findings (‘evidence’) of relevance. Integrating the different types of data in this way may be unfamiliar to those who primarily encounter systematic review findings reported via meta-analyses (statistical summaries), more conventional narrative summaries or qualitative evidence syntheses. However, our focus in the review was the understanding of complex relationships and systems, and using a mixed-methods logic model approach enabled findings from each form of data (quantitative and qualitative) to contribute to an explanation of the pathway. Although we have not separated the data into sections for quantitative and qualitative data within the report, study design and robustness of evidence will be fully outlined as part of the analysis.
In Figure 3, progressing from left to right, the model sets out the elements of new models of integrated care as identified by the review, followed by core areas of change within these elements, followed in turn by changes in the process that may result from the intervention elements, and then by factors that may influence implementation and effects, the outcomes that may result from integrated models of care, and finally, data related to system-wide impacts. The model is intended to provide a summary overview of the elements identified in the pathway, from interventions to impacts. In the following sections, we will provide a detailed analysis of the findings in relation to each element of the model. Following a narrative synthesis describing the results of the review in relation to each section of the model, we will examine areas in which the included studies provide stronger or weaker overall evidence.
Elements of the interventions
The evidence regarding elements of integrated care initiatives is derived from both the UK and international intervention studies. The majority of the reported new models of integrated care comprised complex and multielement interventions. The elements can be grouped into four main areas: (1) those with a focus on improving patient care directly, (2) those that focused on making changes to organisations and systems, (3) those that focused on changing how staff were employed and worked (workforce) and (4) those that changed financial and governance aspects. Many studies reported new models that included several of these elements, and it was often challenging to elucidate exactly the form and components of new models because of limited reporting. Figure 4 provides a summary of the elements of new models of integrated care that are described in the literature.
We acknowledge that itemisation of each identified element is likely to under-represent elements that were contained within the reported interventions. In identifying these elements, we were reliant on authors’ descriptions, which could be brief overviews rather than detailed outlines. The interventions that were described as case co-ordination/case management in particular were often described as an overall approach, with little detail regarding precisely the processes or systems involved.
The introduction of ICPs was the most commonly occurring element, either as a ‘stand-alone’ intervention or as part of a more complex package. This was closely followed by multidisciplinary teams (MDTs), which was also a frequently described component of UK and international studies. The greatest number of elements in an intervention described by authors was nine, with the least being only one element. Interventions typically included between four and six elements. In general, the non-UK interventions contained fewer elements than those reported from the UK, although one study (two papers)227,301 reported seven components and another211 reported six. The international literature had a greater number of examples of interventions that were described as including a case manager/case co-ordinator, and the UK literature included more studies in which authors described integrated care pathways/plans as an element. There was less description of professional role change and colocation of staff as elements of interventions in the international intervention studies.
The most common identifiable interventions were ICPs, multidisciplinary teams/meetings and the provision of a nominated care co-ordinator or case manager. Many interventions combined two of these elements but only one intervention217 involved all three. Analysis revealed considerable heterogeneity in terms of details of the interventions, patient and staff groups involved, and outcomes. Studies involving the three principal interventions were categorised as simple if the initiative had only one component (e.g. introduction of an ICP) or complex if additional components (including other common interventions, as mentioned previously) were involved. Summary tables are presented for the UK (Table 1) and non-UK (Table 2) studies.
Study (first author and year) | Simple or complex initiative? | Principal intervention | ||
---|---|---|---|---|
ICP | MDT | CM | ||
Ahmad 200735 | C | ✓ | ✓ | |
Bakerly 200936 | S | ✓ | ||
Beacon 201537 | S | ✓ | ||
Clarkson 201142 | S | ✓ | ||
Cunningham 200844 | S | ✓ | ||
Department of Health and Social Care 201245 | C | ✓ | ✓ | |
Dodd 201146 | C | ✓ | ✓ | |
Graffy 200847 | C | ✓ | ✓ | |
Gravelle 200748 | S | ✓ | ||
Ham 201049 | C | ✓ | ✓ | |
Harris 201350 | S | ✓ | ||
Hockley 201053 | S | ✓ | ||
Huws 200854 | S | ✓ | ||
Jha 200755 | S | ✓ | ||
Johnstone 201156 | S | ✓ | ||
Johnstone 201257 | S | ✓ | ||
Julian 200758 | S | ✓ | ||
Kent 200659 | S | ✓ | ||
Lamb 201460 | S | ✓ | ||
Letton 201361 | S | ✓ | ||
Levelt 200862 | S | ✓ | ||
MacLean 200864 | S | ✓ | ||
Mertes 201365 | S | ✓ | ||
Ng 201466 | C | ✓ | ✓ | |
Paize 200768 | S | ✓ | ||
Pettie 201270 | S | ✓ | ||
Roberts 201273 | C | ✓ | ✓ | |
Roland 201274 | C | ✓ | ✓ | |
Rowlandson 200975 | S | ✓ | ||
Ryan 200776 | S | ✓ | ||
Smith 201279 | S | ✓ | ||
Soljak 201380 | S | ✓ | ||
Steventon 201181 | C | ✓ | ||
Stokes 201682 | C | ✓ | ✓ | |
Tucker 200984 | C | ✓ | ✓ | |
Waller 200785 | S | ✓ | ||
Wilberforce 201686 | C | ✓ | ✓ | |
Windle 200987 | C | ✓ |
Study (first author and year) | Simple or complex initiative? | Principal intervention | ||
---|---|---|---|---|
ICP | MDT | CM | ||
Aiken 2006194 | S | ✓ | ||
Battersby 2007195 | C | ✓ | ||
Béland 2006;196 Béland 2006197 | C | ✓ | ✓ | |
Bird 2007;199 Bird 2010198 | C | ✓ | ✓ | |
Boult 2008;202 Boult 2011;203 Boult 2013201 | C | ✓ | ||
Brännström 2014204 | C | ✓ | ||
Brown 2012205 | C | ✓ | ||
Callahan 2006206 | C | ✓ | ||
Counsell 2007;209 Counsell 2009210 | C | ✓ | ✓ | |
Dorr 2008211 | C | ✓ | ✓ | |
Ettner 2006212 | C | ✓ | ||
Fagan 2010213 | C | ✓ | ||
Farmer 2011214 | C | ✓ | ✓ | |
Gray 2010;215 Hogg 2009220 | C | ✓ | ||
Hajewski 2014216 | C | ✓ | ✓ | |
Hammar 2009217 | C | ✓ | ✓ | ✓ |
Hébert 2010218 | C | ✓ | ||
Hullick 2016221 | C | ✓ | ✓ | |
Janse 2014;223 Janse 2014224 | C | ✓ | ✓ | |
Martinussen 2012226 | C | ✓ | ✓ | |
McGregor 2011;227 Katon 2012225 | C | ✓ | ✓ | |
Morales-Asencio 2008228 | S | ✓ | ||
Olsson 2009229 | C | ✓ | ✓ | |
Parsons 2012230 | C | ✓ | ||
Paulus 2008;231 Paulus 2008232 | S | ✓ | ||
Rosenheck 2016233 | C | ✓ | ||
Sahlen 2016234 | C | ✓ | ||
Salmon 2012235 | C | ✓ | ||
de Stampa 2014236 | C | ✓ | ✓ | |
Stewart 2010237 | C | ✓ | ✓ | |
Taylor 2013238 | S | ✓ | ||
Theodoridou 2015239 | C | ✓ | ✓ | |
van der Marck 2013240 | C | ✓ | ||
van Gils 2012;302 van Gils 2013241 | C | ✓ | ||
Wennberg 2010242 | C | ✓ | ✓ |
Much of the UK literature evaluated simple interventions, typically ICPs. Many of this group of studies evaluated process outcomes rather than service-level outcomes, finding improvements in areas such as staff contact, documentation and decision-making. 56,61,62,85,303,304 Evidence of other effects was limited, with two studies65,68 indicating a reduction in length of stay, but others62,80 suggesting little change in outcomes, with one of these80 suggesting a higher number of admissions in areas with integrated pathways. The findings were also mixed for studies of complex initiatives involving ICPs, although a greater proportion of these studies indicated positive effects than the those concerned with simple interventions. 36,46,47,49,66,74
Multidisciplinary teams were commonly implemented as part of a larger initiative (often in combination with case management and/or patient education). Only three UK studies37,50,60 reported simple MDT initiatives and two of these50,60 looked at process outcomes, reporting mixed results. However, Beacon et al. 37 reported that a multidisciplinary primary care team reduced secondary care activity and the number of emergency admissions. The only non-UK study231 to report on MDT care in isolation found higher total mean costs relative to traditional care in a nursing home setting. In contrast, a number of UK studies reported positive outcomes from more-complex initiatives involving MDTs, including reductions in length of stay and hospital costs;36 reduced length of stay;46 reduced numbers of admissions and readmissions;66 reduced waiting times;75 reductions in the numbers of inappropriate referrals;76 and perceived improvements in care and access to services. 84,86 Some studies81,189 of complex MDT interventions failed to show benefits. Overall, however, there was a tendency for complex MDT interventions to be relatively successful, and this was also observed in the non-UK studies. In two non-UK studies,217,229 initiatives involving both an ICP and a MDT were successful in reducing costs.
Only one UK study48 reported on a ‘simple’ case management initiative, finding no benefit of the intervention using a controlled before-and-after design. One non-UK study238 reported on ‘simple’ case management interventions and found process change improvements in care co-ordination. The sample of studies reporting ‘complex’ case management interventions was considerably larger and the results were correspondingly more challenging to unravel. Most studies reported improvements in one or more outcomes associated with the intervention, but there were also examples in which no effect was detected. 82,195,213,218 In some cases, study authors attributed this to specific factors such as insufficient length of follow-up195 or possible inappropriate use of the emergency department (ED). 218
In summary, ICPs were frequently implemented as ‘stand-alone’ interventions, with some evidence of improved process outcomes but little evidence of change in service delivery outcomes, and only a slightly more positive indicator regarding effectiveness when included within complex interventions. By contrast, MDTs and possibly case management interventions appeared to be generally more effective when combined as part of a number of different elements (although not as stand-alone elements). However, this analysis should be treated with caution as it does not take account of other possible influences on intervention effectiveness such as the setting and the characteristics of the patient group receiving the intervention.
Target areas for change
We scrutinised the included literature to identify the stated aims for the different intervention elements, with the objective of exploring commonalities of purpose across these varying elements. These areas can be considered the espoused mechanisms of change within the model (active ingredients in the interventions), which were presumed to underpin improvements in health service delivery and patient care. The evidence regarding these target areas for change draws heavily from the UK qualitative studies, although the findings were echoed in international systematic reviews and a small number of UK intervention and international primary studies. The intervention studies (both UK and non-UK) tended to describe these areas as targets for change, but instead reported outcomes at a service and system level (e.g. hospital admissions and quality of care).
Seven target areas for change were identified across the different types of initiative, predominantly related to interpersonal aspects of health-care delivery. The areas were (1) the holistic and patient-centred models of care; (2) common values, beliefs and priorities between staff and/or organisations; (3) relationships between staff; (4) relationships between organisations; (5) levels of trust and support among staff; (6) mutual understanding among staff; and (7) communication between staff. Although these target areas referred to individual elements in the studies, it is important to note that there is considerable interconnectedness between them. For example, achieving a patient-centred model is influenced by values, beliefs and priorities; trust and support will be influential in staff and organisational relationships; mutual understanding will influence relationships; and communication may underpin all of these. The evidence we identified in relation to each of these target areas for change is described in more detail in the following sections.
Holistic and patient-centred models of care
The aim of achieving more holistic or patient-centred care in new models was described most frequently in the UK qualitative studies,103,111,125,130,134,135,143,144 but was also reported in the included systematic reviews. 160,183,185,190 The aim of patient-centred care was described particularly in studies that were carried out in populations of older adults and in those focusing on end-of-life care, but was also described in interventions for general patient populations. One UK qualitative study of stroke, kidney and sexual health services111 reported that a patient-centred culture had spread following a modernisation initiative that included integrating services, role changes and new service provision. Other UK qualitative studies129,135 described achieving holistic care as being an important aim in the introduction of MDTs. A more pragmatic reason for including patient-centred care as a key aim of new models was reported in two systematic reviews. 185,190 These reviews described the importance of a patient-centred focus as a way of involving stakeholders involved in new integrated care initiatives.
Common values, beliefs and priorities
The UK qualitative studies with different settings, patient groups and different intervention types described the importance of developing shared beliefs between staff and between organisations, such as views and beliefs regarding the rationale for, and benefits of, change. 126,134 They highlighted that integrated care initiatives need to achieve similar priorities among organisations127 and between different individuals. 99 The development of shared views and goals was described as important in the development of a shared identity. 89
Systematic reviews echoed the importance of goals being aligned within large-scale transformation157,185 and smaller-scale transformation,181 with shared understanding of the purpose and vision underpinning reconfiguration. Reviews also described the need to overcome differing levels of commitment, values, beliefs and ways of thinking in multidisciplinary teamworking171 and when implementing new integrated pathways. 183 A non-comparator international paper255 evaluating the Program of Research to Integrate the Services for the Maintenance of Autonomy concluded that sharing a common philosophy was the key to implementing the initiative. A qualitative study from the USA,286 exploring perceptions of an Accountable Care Organization, described how change was presented as a cultural rather than a structural change, with co-ordination and equal partnership rather than integration. The presentation of this as being the priority was perceived to avoid a perception of reduced autonomy and to overcome mistrust among physicians. Another qualitative study from the USA298 identified alignment of the initiative with organisational priorities as a factor in the success of health-care transformation.
Relationships between staff
The UK studies with various settings and patient groups60,73,108,111,120,141,145 emphasised the importance of developing enhanced working relationships between staff as part of integrated care. Two UK (cross-sectional) intervention studies60,73 reported that relationships improved following integrated care initiatives. One UK qualitative study,145 in contrast, described that on the surface MDT members might appear to have good relationships, but that this could hide an undercurrent of mistrust and criticism. Another UK qualitative study141 found that, 1 year after the introduction of integrated teams, there was a mixed picture regarding relationships. A further UK qualitative study111 found that improved interpersonal relationships had not been sustained in the longer term.
Two systematic reviews echoed the need for initiatives to aim to improve staff relationships, if high-quality team performance was to be achieved. 171,193 One additional review8 of primary care interventions for patients with chronic disease, mental health or aged/palliative care, which focused on relationships between staff or between staff and patients, linked these to a positive effect on health and patient satisfaction outcomes. A non-comparator study from the Netherlands254 found that relational co-ordination (the interaction between communication and relationships) was positively related to integrated care delivery (p > 0.05).
Relationships between organisations
A non-comparator study from Germany219 reported that, although integration had enhanced staff relationships within the same service, relationships with organisations that were external to the initiative were less satisfactory. This was echoed by a non-comparator study from Canada,246 which described reform as having a ‘territorialising effect’. Collaboration with other organisations in the areas that were external to the network areas decreased (from 15% to 4% for collaboration with other primary care organisations and from 21% to 15% for collaboration with hospitals outside the network). A further qualitative study from Germany293 also highlights that, although co-operation between staff based in the same organisational grouping was reported as excellent, relationships with other external organisations were less satisfactory. A qualitative study from the USA287 that explored perceptions of Accountable Care Organizations described them as typically having formed a new relationship or formal partnership between themselves and other health-care providers.
Trust and support among staff
The need for trust and support was described in UK qualitative studies related to MDT interventions, ICP interventions and large-scale integration initiatives in both primary care and hospital settings, and with a range of patient groups including children and older adults. In these studies,90,91,110,113,127 the need to overcome misunderstanding, tension, and organisational and professional barriers in order to achieve trust was described. Systematic reviews92,93 described trust as being vital if co-operation between staff was to be achieved. Other systematic reviews also highlighted the importance of achieving trust and support among staff,162 and the value of processes being put in place to support staff to achieve new ways of working. 184 An Australian qualitative study90 linked staff having trust in other stakeholders to embracing an integrated health service identity.
Mutual understanding among staff
The UK qualitative studies described increasing staff understanding of each other’s roles and skills as a key aim for integrated care interventions. Much of the evidence derives from studies of older adults; however, we also identified studies based in EDs and in primary care mental health and learning disability services. Most of the studies relate to MDT interventions. Papers described limited understanding within a MDT92 and how collaborative approaches could change a ‘blame culture’ to one of mutual understanding. 101 One study90 in particular highlighted a lack of understanding between doctors and other staff, and another147 described a lack of understanding between staff working in primary care and staff working in hospitals. A further UK qualitative study121 suggested a link between improved understanding and more-rapid service delivery.
A systematic review162 similarly reported that there could be a lack of understanding between, and negative perceptions among, different staff groups, including health and social care staff, and another179 reported that a common understanding was important.
Communication between staff
The improvement of communication between staff was frequently described as a target area for change. Staff communication was outlined as a key part of interventions for a wide range of patient populations [including children, older adults, those receiving end-of-life care, those requiring mental health care and those with chronic obstructive pulmonary disease (COPD)] in both acute and primary care. Regarding whether or not change had been achieved, seven UK intervention studies35,47,60,61,64,74,83 reported that communication had improved, and one50 reported that it was unchanged. Two studies (eight papers)35,47,60,61,64,74,83 that reported an improvement were of a higher-quality comparator design, although they used self-reported data. All of the other intervention studies35,47,60,61,64,74,83 also used self-reported or subjective measures of communication.
Eight systematic reviews,161,171,179,181,183,184,191,193 three of which included only UK studies,161,179,181 echoed the UK effectiveness studies in that communication was a key area for change in integrated care initiatives. Five of these systematic reviews181,183,184,191,193 concluded that integration (general collaboration, team education, MDTs, multidisciplinary case conferences or multidisciplinary meetings) led to improved communication. Three161,171,179 linked improved communication to improved team performance, and, as a result of this, enhanced care. None of the reviews were of the highest quality (meeting nine or more appraisal criteria).
One non-UK comparator intervention study from Canada (of reasonable quality)197 refers to ‘communication challenges’ that were attributable to case managers with poor communication skills. Another non-comparator intervention study (of reasonable quality) from the USA205 describes communication between a case co-ordinator and physicians as being a feature of success. A qualitative study from Canada279 reported that there were perceived views that ICPs enhanced staff communication. A qualitative study from the USA285 supports the importance of in-person contact between staff, and communication in integrated care.
The UK qualitative studies provide mixed evidence regarding whether or not the target of improved communication was met. Five94,98,108,125,147 reported perceptions of improved communication and seven92,113,115,118,141,142,145 reported an ‘uneven’, ‘patchy’ or ‘limited’ effect, communication breakdowns, substantial barriers or medical dominance impeding communication initiatives. A qualitative study from the USA272 concluded that communication and co-ordination between agencies are important influences that moderate the effectiveness of interventions.
Process changes
Within a systems perspective, the intervention elements and target areas for change have the potential to effect changes in the process of care delivery, which may then lead to outcomes and impacts at a service and whole-system level. We analysed the included literature for evidence regarding any effect of new models of integrated care on the process of care delivery. We grouped the evidence that we identified into the four areas of (1) service user change, (2) change in care provision, (3) change in information and (4) organisational change.
Patient role
Relationship with health-care provider
A qualitative study from the UK,100 of the North West London Integrated Care Pilot, indicated mixed results regarding whether or not the intervention changed relationships between patients and providers. Although 54% of participants responded positively to a question regarding an improved relationship in the survey evaluation, the study reports that 54% of participants also responded that they had not experienced any changes. Two international qualitative studies273,279 report more-positive outcomes: that integration of a community worker into a team had improved patient knowledge and engagement,273 and that an ICP had improved patient involvement. 279
Patient understanding of treatment
One UK intervention study59 reported that ICPs were associated with improved patient understanding of treatment and reduced anxiety among patients.
Care provision
Discharge planning
Two UK intervention studies59,63 reported that discharge planning had improved. The first of these59 evaluated the introduction of ICPs in hospitals and concluded that there was strong evidence that they improved discharge planning. Nursing discharge planning rose by 20% (p < 0.01) and there was a significant increase in general discharge planning, from 33% to 93% of patient records being examined (p < 0.01). The second study63 (which had a comparator group) found that discharge planning for older patients at high risk began within 2 working days of admission in most cases, compared with at the end of a stay or not at all prior to the addition of a social worker to a general practice. One systematic review191 also concluded that case conferences assisted discharge planning (although it reported mixed evidence regarding care planning for stroke patients).
Timeliness of care and/or patient flow
One UK intervention study49 reported that timeliness in terms of patient assessment, care packages being put in place and equipment being provided improved (across all of these elements) at three sites adopting a Kaiser Permanente® (Oakland, CA, USA) approach. UK qualitative studies report positive perceptions that integrated care could co-ordinate services more effectively,98 improve the flow of patients out of hospitals,101 streamline services,109 lead to efficient and timely discharge,116 lead to more-rapid service delivery,121 increase co-ordination,124 provide continuous and pro-active care125 and improve efficiency. 146
Two systematic reviews suggested improved efficiency and reduced overlap following the introduction of ICPs184 or collaborative working. 153 Two international comparator studies214,238 supported this. The first study214 reported parent perceptions of greater satisfaction with care co-ordination (p = 0.058) compared with a control group, with parents reporting that services were more co-ordinated following the intervention (50% prior to the intervention and 80% post intervention; p = 0.003). A second study238 found that, compared with a control group, patients who received a care-co-ordinator intervention were more positive about the co-ordination of care (p = 0.001). However, the difference between groups in reporting improved care co-ordination was not significant. An international qualitative study282 similarly reported perceptions of a positive effect on care timeliness and efficiency. A non-comparative evaluation257 of a patient-centred medical home found that participants reported improved care co-ordination (p > 0.001).
Continuity of care
The international literature highlighted the increased continuity of care that integrated care may provide for patients. Bird et al. 198,199 described how a care co-ordination intervention provided continuity of engagement with patients, which allowed care facilitators to better understand the needs of patients and provide a point of contact. The authors of another study204 attributed frequent visits, continuity of care and ease of access to staff to the ability to deliver structured care at home, thereby reducing hospital usage. A MDT intervention in a psychiatric hospital was similarly described as providing continuity of care. 239 Continuity of care was highlighted as a key change by other international intervention studies. 225,227,238 One UK intervention study64 evaluating an ICP referred to increased continuity of care following the intervention.
Systematic reviews provide a more varied picture regarding whether or not continuity may improve as a result of integrated care. Myors et al. 184 described that continuity of care was achieved via a link worker or identified contact. However, another review160 reported patient perceptions of continuing issues in communication between agencies, resulting in continuity of care being broken. Another review175 concluded that there were few significant findings in favour of short-term programmes related to continuity of care, although the short-term nature of the programmes may have influenced this finding. A review of nurse-delivered care co-ordination interventions186 that reported staff perceptions found that they did not think that the programmes fostered increased communication or continuity of care.
A qualitative study from Australia267 provides a different perspective on continuity by distinguishing three different elements: information continuity, management continuity and relational continuity. The authors reported that continuity of information was important in effective care as it was perceived to reduce repetition and provide shared access to records.
Variance in practice and adherence to guidelines
Reduced variance following the introduction of ICPs was reported by two UK evaluation studies. 56,59 Kent et al. 59 found an absolute improvement of 35–40% in the recording of indicators from clinical guidelines and best practice. Another evaluation of an ICP70 described implementation as standardising decision-making. A general service redesign including the introduction of standardised documentation reportedly ensured consistency between staff. 139
One systematic review180 reported that adherence to guidelines improved in 14 out of 19 studies that were examined. Another184 concluded that pathways and guidelines helped to ensure consistency.
A UK qualitative study of a pathway in palliative care described greater consistency in care. 109 Another study,97 however, warned that pathways could constrain judgement. Greenhalgh et al. 110 found that common guidelines and protocols were difficult and time-consuming to achieve in practice.
Duplication
One UK qualitative study119 described a perception of reduced duplication following an integrated care initiative. Another129 reported the perception that working in a fragmented way causes duplication and waste. A systematic review184 reported that benefits of collaborative working were perceived to include reducing wastage and overlap in staff roles and duties.
Practitioner time and workload
One UK intervention study60 found evidence of a potentially positive reduction in staff time. This cross-sectional evaluation of MDT working reported that 68% of staff perceived that the average of 2 hours that was spent in meetings each week saved time later. The UK qualitative literature, however, provides few examples of perceptions of integrated care as being time-saving, although one study109 reported views regarding the use of the Liverpool Care Pathway in hospices. Staff described the pathway as time-saving as it streamlined documentation.
In one study,105 staff reported concerns regarding potential increases in workload that may result from a Pioneer programme. GPs also expressed concerns that an integrated telephone service may increase their workload. 113 Another study122 described staff becoming preoccupied with additional administrative burdens, time spent in meetings and completing care plans in an integrated care pilot. Team meetings were described as being long and time-consuming, with another paper,123 originating from the same integrated care pilot, suggesting that their frequency should be reduced. In a further paper134 from the same study, views were described as variable, with some staff perceiving meetings as valuable, whereas others expressed concern regarding time and cost. An evaluation of a new integrated GP role in cardiology137 found that this could distract from other GP work and that the associated administrative tasks had been underestimated. Workload had increased to the point that GPs used their own time for the new role. In another study,147 staff also reported increasing their working hours to meet patient needs and expectations from secondary care following the introduction of a new MDT model. Additional time required to enter data in unfamiliar systems was also described by staff. 149,150
One systematic review153 concurred with these reports of increased time being required, with improved documentation (ICPs) reportedly leading to the ‘burden of increased workload’.
In contrast, a non-comparator evaluation of a team-based approach in the USA243 reported that practitioners, when surveyed, reported that care co-ordination was time-saving (saving at least 30 minutes per month per patient). A qualitative study in the Netherlands,268 however, echoed the UK literature by reporting that staff perceived that collaborative initiatives were overly time-consuming.
Information change
Shared information
A key purpose of improving communication between staff is to enhance the exchange of patient information. One of the UK intervention studies64 included outcomes related to the transfer or sharing of information. This survey of staff found that 77% of staff perceived that accessibility of information had improved. The UK qualitative studies provide variance in reported perceptions of the effect of integrated care on the process of information sharing. One study109 found that staff described the value of knowing what other members of the team were doing following the introduction of ICPs. They also valued having clearly documented decisions that had been made by the family available for all staff to access. Another study125 of palliative care also described the sharing of written reports to aid collaboration. A third study143 evaluating pathways in palliative care described benefits from having a structured record, which was especially useful out of hours. It was mentioned, however, that pathways could be used too rigidly by some staff. 143
A UK qualitative study139 exploring a variety of initiatives reported that there had been limited progress in information sharing across different sites. Similarly, another study149 exploring the implementation of a pathway for frail elderly patients and rapid response team described information as being difficult to share because of there being different systems for health and social care. An evaluation of the North West London Integrated Care Programme151 reported difficulty in putting data sharing agreements in place and achieving operability between different health and social care systems. Staff working in integrated initiatives within community disability services reported concerns regarding the reliability and accessibility of care records, with some groups being unable to access NHS records. 107 Team meetings were perceived to be an important means of sharing information. 107
A systematic review of models of joint working160 (which included only UK studies) concluded that there could be difficulty in sharing information and getting access to information (because of incompatible systems). An international qualitative study267 reported issues regarding confidentiality, multiple and/or incompatible data systems and data ownership (governance). A study in Germany292 evaluating team meetings found that key information was not available in 4–5% of the cases discussed. A study by Berendsen et al. ,268 from the Netherlands, echoed the UK study discussed previously143 by describing staff views of integrated pathways as being too restrictive.
Accuracy/completeness of recording information
Three UK intervention studies59,64,70 (none with a comparator design) reported improvements in the accuracy and/or completeness of the recording of information such as initial assessments by staff following the introduction of ICPs in hospital or community settings, although it was noted that some elements of care remained poorly reported (such as disinfection). One UK qualitative study107 described concerns regarding shared care records being incomplete or out of date.
Systematic reviews of ICPs for stroke and nursing home patients at the end of life154,173 report an increase in the documentation of rehabilitation aims, communication with patients and carers, discharge notification plans and plans for ‘do not attempt resuscitation’. In contrast, a systematic review of teamworking171 highlighted gaps in communication and documentation. A non-comparator study from the Netherlands260 of the Liverpool Care Pathway reported that the documentation was more comprehensive than it was before the introduction of the pathway.
Shared knowledge
A perceived increase in knowledge sharing between staff was described in four UK qualitative studies. 106,107,123,132 Three of these papers107,123,132 explored multidisciplinary teamworking. A systematic review171 (also of multidisciplinary teamworking) supported this, concluding that the sharing of knowledge may improve team performance. A qualitative study from Canada279 concurred that staff perceived that there was greater exchange of knowledge (attributable to the use of ICPs).
Organisational structure and processes
Reconfigurations
The UK intervention studies describe several models of organisational reconfiguration. Examples of organisational change include the merging of services delivered in two hospitals;41 a merger between several partner organisations in primary care;45 a merger between an acute trust, a primary care trust (PCT) and social care services;43 integration between hospitals and the community;49 the formation of partnerships between general practices, which may also include nursing/care homes;46 new units or reconfigured wards;71 and services rearranged around health networks. 72 A non-comparator study that explored health-care reforms in Canada259 reported that there were few changes at a system level but that there were new forms of primary care organisations. This is echoed by a qualitative study from the USA281 that reported only small process changes following a reorganisation.
A qualitative case study from Sweden,263 which compared areas of successful integration with areas that were less successful, concluded that none of the successful cases had initiatives focused on management change; all had a focus on improving the quality of services.
Governance and accountability
There are few data directly related to the process of organisational change, beyond one UK intervention study,87 which described 66% of staff agreeing that two or more statutory organisations could share risks in an effective way, and that strong governance was important. The authors reported that the partnerships that appeared to be the most strengthened were those between health-care services, local authorities and voluntary organisations. The UK qualitative literature provides some insights into the organisational change process. One qualitative study103 described differences in the way that organisations described and perceived the process of joint commissioning. The authors reported that many sites that were investigated rejected the term ‘joint commissioning’, preferring terms such as ‘integrated commissioning’, and that staff struggled to describe how joint commissioning differed from joint working generally, and what it should achieve or what the outcomes should be. The process of putting formal structures in place could be seen as the outcome rather than service improvement.
Challenges regarding changing commissioning were reported in another study. 105 This exploration of Pioneer sites describes many as having an official or nominal lead organisation, with a balance that needed to be achieved between driving change and shared ownership. This study105 also reported that varied governance arrangements were operating, with many sites choosing to maintain existing arrangements, and a perception that new contracting models were needed. Another study141 described how new management structures were at an early stage and budgets were yet to be resolved in a reconfiguration of community services. Effective governance was described as a key aspect of integration, with a health and well-being board overseeing the process and a named person having responsibility for the allocation of a pooled budget. 139 In this study of an integrated care community programme,139 progress was limited in the area of financial and contracting mechanisms.
Further evidence regarding the challenges in changing organisational structures and processes is described by Holder et al. 151 This evaluation of the North West London Whole Systems Integrated Care programme describes initial difficulties in relation to data sharing and governance arrangements, payment systems and accountability arrangements, which had led to delayed implementation. It was highlighted that the programme of change was led by NHS commissioners, and the study authors151 reported that there could be personal agendas operating and confusion regarding alignment with wider plans.
Financial arrangements
One UK qualitative paper91 provides a detailed description of the management and financial arrangements in multiple Children’s Trust Pathfinders, including the formation of joint boards that are responsible for the co-ordination of services (29/31 trusts), a commissioning unit (one Pathfinder) or NHS and local authority managers occupying similar positions in a single organisation. Mechanisms for integrating accounting and budgeting varied between the different sites: via pooling budgets through legal contracts, making formal or informal agreements, or sharing information but keeping budgets separate. Another UK qualitative study139 described the pooling of budgets as a frequently highlighted area of tension. In contrast, another UK qualitative study116 reporting views of leaders of integrated care programmes described a perception of there being no debates over budgets. A further UK intervention study87 reported that there were challenges in moving any cost savings to and from budgets, with an inability to move money from health care to local authority budgets.
A systematic review165 of review-level evidence in relation to general health system financing, funding allocations, direct purchasing arrangements, organisation of service provision and health service integration concluded that most financial and organisational system-level reforms have inconclusive or negative effects on funding processes. A second systematic review182 of financial integration across health and social care highlighted the difficulties of achieving financial integration, with a failure to break down service boundaries or take overall control of service use from individual providers. The authors concluded that even when budgets are pooled widely, total resources remain limited. A study by Battersby et al. 195 evaluated a change to an outcomes-based funding system, and reported that 2 years was an insufficient amount of time in which to assess its effects. A further systematic review161 concluded that having a unified budget to cover joint working was important.
Influencing factors
We examined the included studies to identify factors that were reported to act as barriers to, or enablers of, implementation/outcomes of new models of integrated care. We grouped the factors into a four-category typology: (1) those related to patients, (2) those related to the workforce, (3) those related to the organisation and system and (4) those related to finance and governance. The majority of the evidence is drawn from the UK qualitative studies (supplemented by a small number of systematic reviews).
Patient-related factors
Focus of the initiative
Seven UK qualitative studies99,105,114,122,134,136,139 across a range of conditions and service contexts highlighted the influence of what the focus of the new model of care was (or was perceived to be), and in particular whether the driver was improved patient care, service reconfiguration or cost saving. Individual practitioner and organisation interests in partnership working may be perceived as being prioritised over service user needs, which reduces motivation for staff. 99 A focus of integration on delivering care more efficiently and effectively provides motivation and hope for the initiatives,105 and alignment of perspectives regarding the purpose of change is important in enabling change or the development of opposition from staff. 114 Initial enthusiasm for the vision of improving care and a willingness to be involved could change to antipathy and withdrawal of involvement if a focus on service efficiency and reorganisation or potential financial gains was perceived. 122 Tension could be created when managerial focus on financial savings contrasted with clinician focus on improved services. 136 One study139 reported the use of patient case studies to focus change on outcomes for patients, and to break down staff barriers.
Patient engagement in changes
The importance of patients being engaged in the development and implementation of new models of care was emphasised in seven UK qualitative studies,95,100,105,112,127,134,139 two systematic reviews185,199 and one international comparator study. 199 The need for engagement to be sustained over time from the development stage through to implementation has been highlighted,105,199 in addition to the potential for different community groups to have different preferences. 112 A systematic review157 concluded that the involvement of patients and families raises awareness of their priorities and including these priorities increases the perceived validity and sense of equity of changes. Another systematic review185 highlighted that patient/community engagement was a core element that is necessary for integrated governance across primary/secondary care. A qualitative study297 from the USA reported that patients were unsure about the meaning of the term ‘integration’.
Sharing of patient information
Three UK qualitative studies91,126,151 highlighted that concerns regarding the security and confidentiality of patient data could be a barrier to integrating services. This was a particular challenge in reconfigurations across multiple partners. 126 Ensuring that agreements on information sharing were put in place at an early stage was key to ensuring progress in integration. 151
Workforce-related factors
Emotional response to change
Two studies88,196,197 described the need to pay attention to the emotional responses of staff when introducing change. The first of these (a UK qualitative study)88 described ‘painful feelings’ among staff in health and social care regarding reconfiguration of teams and roles, and the need to support them through periods of uncertainty. The second study (reported in two papers),196,197 from Canada, reported a lack of adjustment time for personnel as a particular barrier to an integrated care programme for older people.
Professional identity
A key area of uncertainty during change may be adjustment in professional identity. Seven UK qualitative studies89,90,94,113,128,145,146 with diverse settings and patient groups described professional identity as a key potential barrier to change. New models of care required the development of a shared group identity,89,90 and there could be concerns regarding a perceived erosion of roles and identities,94 which required trust and a sense of ownership to overcome. 113 There could also be a privileging of one professional discourse over another. 145
Professional role boundaries
A large number of studies referred to the challenges of existing professional role boundaries for new models of integrated care. Sixteen UK qualitative studies88,91,94,96–100,105,108,120,124,126,128,131,144 examining a variety of interventions, different patient groups and locations described how issues regarding professional roles had been encountered. The realignment of roles could be stressful and generate resistance, and needed time to be effective. 88,91 Adequate preparation and supervision were described as important,94,131 and some role changes had the potential to raise issues of accountability. 96 One study120 highlighted that resistance to change because of professional boundary changes was not always a barrier, as it could lead to greater professional engagement in the process. Another146 described initial wariness and friction subsiding over time.
A systematic review exploring teamworking193 confirmed that blurring and misunderstanding of professional roles can be common in integrated working, and are key barriers that could lead to professional conflict and intractable differences.
Stability of the workforce
Three UK qualitative studies94,118,126 described a lack of stability of the workforce as a potential barrier to integrated working. Vacancies, staff turnover and use of temporary staff could act as a barrier to continuity of care. 94 Macfarlane et al. 128 described a general climate of unwanted externally opposed change with frozen posts and job uncertainty, which led to challenges related to people accepting new roles and retention of staff. One international comparator study197 reported that resignations of key staff as a result of funding uncertainty was a key barrier to the implementation of a programme.
Employer
One qualitative study88 highlighted that staff in the same team having different employers could lead to staff having different employment practices and working to differing agendas and priorities. There could be a perceived dominance of health over social care, and hospital over community. 116
Power and hierarchies
Established hierarchies of power and status constituted a frequently reported barrier to the successful introduction of initiatives in the UK qualitative literature. 91,97,100,103,105,110,115,116,120,132,145,147,305 In particular, MDT meetings were often described as being dominated by medical professionals. 100,115 Individuals with less power were restricted in the changes that they were able to make. 110,120,145 New models of integrated working could, however, facilitate the removal of this barrier. An advantage of an ICP reported in one study97 was reportedly enabling midwives to deflect medical dominance and pressure. Integrated management was also reported as being helpful to ensure that all professions had equal respect and influence. 116 One systematic review193 supported the UK evidence, showing that the status of team members was an influencing factor in multidisciplinary working. A qualitative study from Italy295 described how a more balanced role of medicine was perceived to exist since integration reforms. The traditional distinction between management and professionalism had been eroded, with a new concept of professionalism that incorporated management. 295
History of co-operation
Five UK qualitative studies93,105,110,126,133 described history and working relationships between staff and different services as influencing the introduction of new models of integrated working. Previous and ongoing changes could prepare the ground for further change,93,105 and a history of collaboration was an enabler of integrated working. 110
Provision of training
The importance of providing adequate training for staff during the introduction of new models of working across different patient groups, types of intervention and contexts was frequently emphasised. Ten UK qualitative studies91,94,96,116,118,125,128,129,146,148 described training as being essential for changes in role or working practice. A systematic review162 reported that studies describing higher levels of integration tended to provide staff with ongoing support or training, rather than at periodic intervals. A second systematic review183 reported that education regarding why and how to use an integrated pathway increased confidence and openness among staff, which then improved communication and collaboration. A third systematic review185 described training as important to support new ways of working and in aligning cultures, and also as being required to address the need for rigorous evaluation and leadership.
Staff involvement and engagement
A lack of engagement in new models of care by staff was frequently reported in the UK qualitative literature. 100,101,111,113,114,118,120,122,126,127,139,142,147,151 Engagement among clinicians could be variable,100 with those outside hospitals reported as being less engaged. 100 GPs were reported to be particularly difficult to engage. 113,120,126,142,147
One UK intervention study77 compared outcomes following a diabetes mellitus integrated care initiative between fully engaged and less engaged general practices. In the more engaged practices, mean hospital admission rates reduced, with comparable changes not seen in the less engaged practices. International comparator studies196,197,205 have also noted difficulties in engaging GPs in interventions. One comparator study from Germany219 described a plan to introduce incentive payments for clinicians. In a UK qualitative study,134 it was reported that staff were concerned about the incentives offered to GPs to be involved in an integrated care pilot. A qualitative study from Spain282 highlighted that previous experiences are influential, finding that individuals who lacked experience of collaboration could have negative views of its usefulness.
Management and leadership
Local leaders/champions
The identification of local leaders or champions to drive initiatives forward was described as an enabling factor in nine UK qualitative studies. 91,93,106,117,118,126,128,136,150 Leaders need to have enthusiasm;91 a strong commitment to implementing lasting change;126 an ability to take charge, and the time and skills to do so;106 oversee and motivate other staff;150 and both ‘hard’ and ‘soft’ skills in transformative change with ‘multiple’ and ‘flexible’ skills. The leader could be a senior member of staff with the support of a facilitator,118 or clinicians from primary or secondary care,136 and it was highlighted that additional training in change management for senior staff was beneficial. 128 The non-UK qualitative literature294 also echoed the importance of the management role in initiatives, with effective leadership described as ‘critical’.
One UK qualitative study117 described how champions appeared to be beneficial in the early stages of adoption of innovations. However, as projects were rolled out and new stakeholders became involved, there could be a perceived threat to the champions in terms of status and lack of exclusive rights to their work, which could create tensions between their supporters and other staff. The authors117 concluded that it should not be assumed that champions always have a positive effect; they may be useful at an early stage to lead the vision, but may then need to hand over the implementation to others. A second study128 also cautioned that although a clinical champion role could work well, it could also be problematic because of personal agendas and problems with back-filling time.
Effective leadership
Having strong and effective leadership was a commonly recurring theme in the UK qualitative literature across different settings, services and patient groups. Fourteen UK qualitative studies97,100,105,106,118,126–129,133,139,141,145,151 emphasised the importance of this factor in the successful implementation of new models of integrated care. Effective leadership included being an enthusiastic advocator but also challenging and debating with staff. 118 Leaders need to be clear about their own role, build informal networks and have buy-in from staff at a senior level and clear lines of accountability. 127,139 It has been reported that there should be an emphasis on leadership rather than management,129 which needs to be strong and fair. 133 The need for protected time for senior leadership to devote to specific projects was also emphasised. 139 A lack of leadership was reported to create tensions, questioning of legitimacy and other obstacles. 139 Changes in management staff adversely affected the management of change. 96
Five systematic reviews162,179,185,190,193 also emphasised the importance of leadership as an enabler of new models of care. One185 highlighted that managers need organisational support to develop the ability to make change happen. Another190 emphasised the need for leadership to have vision, and suggested that physicians should be integrated at all levels, including in leadership roles. A mixed-methods study from Australia269 reported that although survey data suggested a weak association between management and initiative implementation, qualitative interview data suggested that the commitment of proactive managers could influence the implementation of new initiatives. Non-UK qualitative studies288,289,294 also describe the critical importance of strong leadership.
Organisation- and system-related factors
Capacity and resources
The UK qualitative literature92,93,95,98,105,106,114,126–128,134,151 described how the level of funding available and the availability of resources/capacity within organisations could influence implementation and outcomes of varying types of new models of care. Limited funding was perceived to be a significant obstacle to new services98,114 and to achieving successful transition of care between services. 92 Severe financial restraints were reported by staff105 and successful outcomes were attributed to the availability of resources by staff participants in one study of an integrated care and support pioneer initiative. 106
Information technology and logistics
Frustration with information technology (IT) systems was frequently reported in the UK qualitative literature. 72,94,95,100,102,105,107,110,118,124,126,134,146,149 A lack of compatibility between systems used in different services, such as health and social care or general practices and hospitals, was a key barrier to integrated working. 94,95,102 In one study,100 over half of the clinicians (56%) reported that they were dissatisfied with IT systems.
Three systematic reviews179,185,190 echoed the importance of adequate IT systems. One185 described integrated IT as essential infrastructure that enables providers to focus and manage risk, provides accurate and detailed information to inform clinical decision-making, supports change management and allows tracking of high-risk patients and outcomes. A second190 concluded that other processes within integrated models of care are only possible with computerised information systems that are accessible in any location. International qualitative studies294,299 echoed the need for better IT to support co-operation.
Clarity of vision
In Focus of the initiative, we reported that the perceived target outcomes for new models of care (patient care vs. cost saving) could influence the views of staff regarding new initiatives. UK qualitative studies72,100,103 highlighted that the clear communication of aims and outcomes was required to ensure that staff fully understood the vision. Dickinson and Glasby103 described a lack of understanding regarding what joint working meant. Another study122 outlined a perception of ‘being dragged along’ by clinicians, and poor communication about the aims leading to a lack of involvement and ownership. This link between communication of the benefits of change and staff involvement was also made by another study. 126 Another study105 described the challenge of splitting the vision of integrated care into practical activities. The authors reported that staff working in some areas understood the vision less well than others, and that making changes that were meaningful to patients could be difficult. The authors also noted an absence of articulation of the implications of the vision for changing power relationships.
One systematic review185 supports the need for clarity of vision in service integration. Nicholson et al. 185 referred to a shared vision of optimal health care as getting stakeholders ‘to the table’. They suggested that having joint board members with directors on each other’s boards facilitates shared vision, trust and collaboration. A qualitative study from the USA288 similarly concluded that there is a need for a clear mission and vision, and a culture that reflects the values and norms of the organisation.
Organisational culture
Eleven UK qualitative papers91,93,99,110,111,126,127,133,134,139,151 referred to the influence of differing organisational cultures in acting as a potential barrier to implementation and outcomes of new models of integrated care in varying service settings. Barnett et al. 93 described organisational culture as ‘a critical factor in change’, in particular openness and having a fit between organisational values, beliefs and the innovation. Greenhalgh et al. 110 similarly described the compatibility of values across organisations as a key factor in enabling change. A 2-year follow-up of this study by the authors111 found evidence of stakeholders’ conflicting interests and values, which had led to lack of sustainability of the service transformation. Another study127 reported how a particular challenge to change was staff in different organisations working within different care models.
Another study133 exploring the merger of two NHS trusts described how, although there were differences (one was more entrepreneurial, team oriented and less hierarchical), there were key similarities in culture, such as the emphasis on staff development, working in partnership and overcoming adversity. The authors reported that there could be challenges in preserving desirable cultures in shifting landscapes, but that best practice and goodwill were important. 133 Other studies described the importance of establishing a culture of collaborative working, which required sharing of clear objectives and processes134 and the need for the alignment of perspectives to enable change,114 together with the need to lever existing partnerships and the requirement for perseverance. 139 Huby et al. 120 outline that good cross-boundary relationships and access to sources of power and political leverage are important for organisational change. In this study of reconfigurations, including new roles for nurses and GPs, a facilitator was the aligning of initiatives with existing organisational priorities. 120 In contrast, another study151 describes confusion regarding alignment with wider plans, with organisational fragmentation also slowing down implementation.
A qualitative study from Sweden263 described four organisational culture facilitators: (1) that development is perceived as desirable and appropriate, (2) that there is backing from management, (3) that there is trust between participating organisations and (4) that there is acceptance from the body of physicians in each organisation. A non-comparator intervention study from Australia248 described culture change as being key in any initiative. A non-comparator study from the USA253 found that motivational readiness and organisational climate for change were associated with higher odds of co-ordination. Kathol et al. 283 also described the need for a culture shift.
Systematic reviews echo the role of organisational culture in new models of care. Suter et al. 190 described the need for an organisational culture to match the vision of integrated care. This was echoed by Maslin-Prothero and Bennion,181 who reported the necessity for the development of a shared culture, including the purpose and vision of change. Boult et al. 158 highlight that ‘real world’ adoption of new models involves having to fit in with existing organisational cultures. A review of working practice between care home staff and health-care staff162 reported that different working cultures of staff acted as a barrier. A review of teamworking171 concluded that culture affected change. Organisational culture was also reported to influence the introduction of ICPs. 179
External threats and policy context
The UK qualitative literature has widely reported the influence of policies and decision-making outside the control of the organisations involved in new models of integrated care. For example, changes in the political landscape, such as devolution in Scotland, could change policies and priorities. 95 Commissioning mechanisms, severe financial restraint, national reorganisation (such as NHS foundation trusts and staff regrading) and conflicting government policies also had an impact on the delivery of initiatives. 105,114,122,128,133,151 One study114 described how the choice of model that was introduced was dictated by the funding stream. A systematic review of UK literature on integrated health and social care179 echoed the influence of national policy on the introduction of new models of care. A qualitative study from the USA265 similarly emphasised the influence of the external environment and need for policies to facilitate change.
Audit and evaluation
Three systematic reviews157,183,193 highlighted the importance of feedback loops in the introduction and sustainability of new models of integrated care. A review of ICPs183 concluded that positive feedback from a national audit on service improvements helped sustainability, and that audits that identified areas of need could facilitate change. A review of integrated governance systems185 emphasised the need for mechanisms of reporting and auditing to be in place. This was echoed in a review of the literature on health integration, which reported that performance monitoring systems should be in place to measure outcomes at different levels. A qualitative study from the UK101 described how having outcome targets in place for only one area of care (emergency services) meant that there was a lack of incentive for other services (such as community) to improve. Greenhalgh et al. 111 describe how the meaning and significance of measures of effectiveness were contested. Roberts et al. 139 highlight the importance of taking time out to monitor and reflect on progress. This was seen as important in maintaining focus and momentum. A qualitative study from Ireland280 highlighted that audit was an important tool in the implementation of new models of care.
Commissioning
The UK qualitative literature describes the challenges of commissioning integrated services. Studies105,111,114,128 highlight that funding mechanisms could be counterproductive, with conflicts regarding what each stakeholder stood to gain or lose. New models of care were reported to be not embedded in the commissioning process, with new commissioning systems needed, such as outcomes-based commissioning. 105,111 A study of Children’s Trust Pathfinder sites91 noted that NHS organisations and local authority children’s services were jointly commissioning a range of services, but GP and hospital services were rarely included. Another study103 noted that different services had varying ways of describing and perceiving joint commissioning. This study also noted that joining up particular services inevitably creates new boundaries by excluding others.
Greenhalgh et al. 111 reported considerable challenges in sustaining structures put in place following organisational change. This 2-year follow-up of a modernisation initiative found that the original programme board and management structures had been dismantled, with their activities either ceasing or being transferred to new structures. The authors highlighted that new models need to be closely linked to the commissioning and business-planning infrastructure if they are to be sustained.
Budgetary and financial factors
Roberts et al. 139 found slow progress in developing financial and contractual mechanisms at the different sites they studied. They reported that it was common for the sites to report significant challenges regarding this aspect of new models of care. Some cited a lack of capacity to lead the development of this area, and others found finance and contractual arrangements to be complex and challenging activities. Bespoke external expert help was helpful in enabling progress. Conversations regarding finance were reported to be extremely challenging, particularly when considering moving money from acute to community settings. Many left development of this aspect to later in the process.
Two systematic reviews185,190 provide further data regarding financial challenges. Suter et al. 190 described different service funding as a major barrier to integration, with financing mechanisms requiring pooling of resources and a population focus. Nicholson et al. 185 described the need for new financing and a change from an organisation focus to care for a whole population. A qualitative study from the Netherlands290 highlighted the need to develop incentives for financial collaboration. A US qualitative study294 reported the need for standard contracts and outcome measures to encourage integration and commissioning between providers, with a requirement to demonstrate improved patient experience and clinical outcomes in regulation of integrated care initiatives. The authors emphasised that the pricing strategy needs to incentivise integration, and bundled payments and local tariffs for particular conditions and pathways should be developed. 294
Non-UK intervention and qualitative studies283,284 discussed how payment systems could potentially act as barriers to, or enablers of, new models of integrated care. A comparator study from Germany219 described a plan to introduce incentive payments for clinicians. A non-comparator study from the USA252 recommended that GPs should be rewarded for interventions that avoid potentially preventable admissions/contacts. The authors highlighted that current payment schemes focus on physician effort, rather than patient characteristics or outcomes. A qualitative study from the USA265 described misalignment of reimbursement systems as a barrier to patient-centred medical homes. Another qualitative study from the USA291 reported that barriers to becoming part of an Accountable Care Organization were predominantly finance related, with legal and regulatory barriers also present.
A qualitative study examining various initiatives in Australia to improve co-ordination267 noted differences regarding the role of payment systems in different services. In one case study, financial incentives were not perceived as important for involving GPs, who were reported as being most interested in involvement for the benefit of patients. In the second case study, it was perceived that use of an electronic record had been hampered by a lack of incentive payment for doctors. In the third case study, it was reported that a lack of incentive payments made it difficult to attract GPs initially, but efficiency gains overcame this.
A qualitative study from the Netherlands296 reported that fee-for-service systems lead to inappropriate incentives for non-co-operation, an inability to influence the service, a service that is not patient-centred and inflexibility. The authors described the view among participants that integrated financing was incompatible with the existing financing system, was problematic for patients with comorbidity and lacked evidence. They also noted that integrated financing is challenging when there is a diverse patient population, varying care, differing requirements at points in the pathway of care and a lack of clarity regarding who has principal responsibility for care.
Governance
The need for strong accountability mechanisms and risk sharing across organisations was described by a UK qualitative study. 100 Governance was reported to be an issue in further UK qualitative studies,105,126,151 with effective governance being a key enabler of integration. 139 One study105 reported that governance arrangements varied in different Pioneer sites, with many choosing to maintain existing arrangements. A UK intervention study66 described having clear clinical governance as a key aspect of a new MDT initiative. Strong governance with clearly understood lines of responsibility and accountability was reported as important in another UK intervention study. 87
Three systematic reviews181,185,190 describe the importance of clear governance arrangements in new models of working. One185 recommended that goals and strategies are jointly agreed, with formal agreements put in place to manage deliverables, risk and process. Another190 concluded that a flatter and more responsive organisational structure facilitates integration, with a need for effect mechanisms of accountability and decision-making. A qualitative study from the USA274 described that having common objectives, contractual relationships, representation on the board of executives and having a large market share facilitated collaboration.
Outcomes from new models of integrated care
The included literature examined an extensive range of outcomes (effects) following the introduction of new models of integrated care. We grouped these outcomes into three main areas: (1) those related to usage of health-care resources, (2) those related to the quality of care received by patients and (3) outcomes for staff working experience. The group of outcomes to be described in this section relates to effects on individual parts of the health-care system. Outcomes in one part of the system will have a ‘knock on’ effect on other parts; for example, a reduction in length of hospital stay may increase the need for community care. Evidence related to wide impacts across the whole of a health-care system on community care activity, secondary care activity, health-care costs and health-care utilisation is outlined in System-wide impacts.
Resource usage
Clinician contacts
Two UK intervention studies,44,87 five non-UK comparator studies158,206,217,218,306 and one non-UK non-comparator study249 evaluated the number of clinician contacts as an outcome measure. A UK cluster RCT44 found that clinician contacts in the first 12 hours after admission increased following the introduction of an ICP for children with breathing difficulties in an ED (p = 0.0004). In contrast, a community intervention for older adults87 was reported to have reduced the number of physiotherapy or occupational therapy appointments and led to a 25% reduction in the number of visits to a practice nurse (p = 0.05).
Non-UK studies report a range of findings regarding the effect on clinician contacts, with three RCTs or cluster RCTs (two from the USA158,206 and one from Finland217) reporting a reduction in clinician visits (p = 0.03 to p < 0.001; 1% reduction) following the formation of new teams or standardisation of practice interventions for older adults. In contrast, one Canadian non-RCT study218 found an increase in health professional, voluntary and home help visits in frail older adults following a case management care plan and standardised assessment intervention. A Swedish RCT306 similarly reported higher resource use in terms of nurse visits among adults with heart failure following a team and key contact intervention, although the text and the data provided seem contradictory.
A non-comparator study from the USA249 evaluated an intervention comprising shared electronic records and a messaging system for patients and staff. The study highlights the need to consider system-wide effects resulting from new models of care. Total numbers of patient in-person contacts across all professions reduced by 26% over a 3-year period following introduction (p < 0.001) in both primary care (25% reduction) and specialist services (21% reduction). Numbers of ED attendances and urgent care visits increased, (p < 0.001) but not by as great an extent as the reduction in patient in-person contacts. However, the number of telephone contacts by patients also increased, resulting in an overall increase of 8% in patient contacts (p < 0.001).
General practitioner appointments
Three UK studies58,63,87 reported the effect of new models of integrated care on GP appointments (rather than all contacts or non-medical clinician contacts). Two studies63,87 reported a reduction, and one58 found no significant difference. Julian et al. 58 evaluated the effect of a GP-led ICP for women with menstrual problems via a small-scale non-RCT study, and found that there was no significant difference in the number of GP appointments (although it had a positive effect as there was a reduction in the number of hospital appointments).
The first58 of two studies reporting a reduction in the number of GP appointments had a comparator group design, and evaluated the placing of a social worker in a general practice, together with joint assessments and care packages for the care of older and at-risk patients. The number of GP consultations were reported to have reduced by 3% when comparing the project year to the previous year. A before-and-after evaluation of the Partnership for Older People Projects (POPPs)87 reported a 10% reduction in the number of GP appointments (p = 0.009), a 28% reduction in the number of telephone calls to GPs (p = 0.014) and a 25% reduction in the number of visits to a practice nurse (p = 0.05) for the case co-ordination type of interventions that had been put in place in some sites.
Length of stay
A large number of UK and international studies evaluated the impact of new models of care on length of stay in hospital. Of the UK studies, five34,44,61,66,85 reported that length of stay was not significantly different, one69 reported that it increased and 1434,36,40,46,49,54,59,63,65,68,71,72,81,87 reported that it had reduced. One additional study40 found contrasting results, with a reduced length of stay for elective surgery and emergency medical patients (p < 0.001), but longer stays for surgical, medical and paediatric emergency cases.
One study reporting no significant difference66 evaluated the introduction of a children’s community outreach team with an integrated pathway and extended hours. The other three studies evaluated ICPs for children with breathing difficulties,44 gynaecological patients61 and patients with diabetic ketoacidosis. 85 One study69 reported an increase in length of stay, but it has very limited reporting and is of very weak quality.
Of the 14 UK intervention studies reporting a reduction in length of stay, four36,54,63,81 are of a higher-quality controlled before-and-after design; three54,63,81 of these were carried out in populations of older adults in community settings. The first of these81 evaluated eight POPPs with older adults in the community (generic workers, integrated teams and out-of-hours services). It reported that the integrated health and social care team project reduced the number of bed-days following emergency admissions (no detailed data). This was in contrast to a signposting service that appeared to increase the number of bed-days. The second study54 evaluated a case manager role in general practice and found a shorter length of stay in intervention practices than in the control (adjusted relative risk 0.896, relative risk reduction 10.41%, 95% credible limit 0.820 to 0.979; p = 0.015). The third study81 evaluated having a social worker located in a general practice, with joint assessment and packages of care in conjunction with a district nurse. The study found that there was a significant difference in length of stay between patients from the project practice and patients from other practices (p = 0.02; mean 31% reduction). The fourth UK comparator study36 was a cost-effectiveness analysis of a hospital at home service, which found that the mean length of stay in the intervention group was 3.3 days [standard deviation (SD) 3.9 days], compared with 10.4 days (SD 7.7 days) prior to the introduction of the initiative.
The 10 other studies34,40,46,49,59,65,68,71,72,87 also reporting a reduced length of stay were uncontrolled before-and-after studies. They were carried out in a range of populations and settings encompassing general hospital patients,59 children in hospital,68 older adults in hospital and community settings,49,65,87 older adults and/or people with diabetes mellitus,72 hospital mental health services,71 urgent care,40 patients at the end of life,34 and adult patients with complex needs. 46
The intervention elements associated with a reduced length of stay also varied, including MDTs and care managers,46 whole service redesign with multiple elements,40,49,71,72 ICPs59,65,68 and multiple types of intervention, including case management and discharge planning. 87 Two studies59,65 highlighted a difference in the effectiveness of ICPs for different groups of patients. The first of these59 found a reduction in length of stay associated with only about half of the conditions in a hospital where ICPs had been introduced, and the second65 concluded that pathways in a hospital setting were more effective for the older and male patients among those undergoing hip or knee arthroplasty.
Ten systematic reviews158,168,169,171,172,175,177,180,182,191 included evaluation of length of stay outcomes. Six158,168,171,172,180,182 concluded that new models of integrated care led to reduced length of stay, and four169,175,177,191 concluded that findings were inconclusive or that there was no clear impact. The reviews reporting uncertain effects examined studies of older adults in services crossing both acute and community settings,177,191 hospital services for people with brain injury175 and hospital mental health services. 169 The interventions comprised case management and the formation of MDTs. All four studies169,175,177,191 are from Australia and meet just over half of the quality criteria for systematic reviews. The systematic reviews, which concluded that length of stay was reduced with new models of integrated care, included a range of settings and patient groups, including older adults,168,171,172 those with any condition182 and the chronically ill. 180 They included case management interventions in hospital or community settings,168 MDTs in hospitals and community settings,171 financial integration182 and any multidisciplinary intervention. 180 This group of systematic reviews includes one168 that met all but two quality criteria.
The international comparator studies echo the diversity with regard to length-of-stay outcomes, with two studies, the first in older adults in a community setting198 and the second with a range of patients in a hospital medicosurgery unit,216 suggesting a significant reduction of 19.2% in the number of bed-days (p = 0.001) and length of stay (p = 0.31), respectively. In contrast, two studies (the first in an emergency unit for older adults,221 the second in a psychiatric hospital239) reported no significant difference in length of stay or the number of bed-days in hospital. The first of these evaluations221 found that both the intervention arm and the control arm had a reduced hospital length of stay, and although the intervention group reduced to a greater extent, this difference was not significant. One of the four studies in this group239 used randomised allocation to a study arm. The findings are from studies carried out in Australia, the USA and Switzerland. The two studies finding a significant reduction in length of stay198,216 evaluated co-ordinated assessment and management and case management interventions. The two studies finding no significant difference221,239 evaluated a multielement co-ordinated case management intervention and a MDT intervention.
Three international non-comparator studies243,261,307 report divergent findings regarding effect, with two243,307 indicating reduced length of stay and one261 indicating increased length of stay in a nursing home. This study261 is the only one that used length of stay in a nursing home rather than hospital length of stay as a measure.
Unscheduled admissions
Most evidence regarding the effect of new models of integrated care on unscheduled and emergency admissions comes from the UK intervention studies. Four UK studies,46,48,78,80 including two with a comparator design,46,78 indicated no effect on the number of unscheduled admissions. 46,48,78,80 Four studies reported in five documents43,45,74,81,82 indicated a significantly increased number of admissions. Three of these74,81,82 were of a higher-quality comparator design. The Steventon et al. 81 study found that only the intermediate care intervention seemed to be associated with an increase; the health and social care team projects appeared to reduce the number of emergency admissions for a particular high-risk subgroup. Stokes et al. 82 reported that the increase was only 0.01 admissions per patient per month, an effect size of 0.03, which is of limited clinical significance.
Intervention studies from the UK have reported a reduction in the number of unscheduled admissions of 22%,37 a 16.3% decrease in the number of emergency medical admissions and a 3.9% decrease in the number of emergency surgical admissions,40 17 admissions for 13 patients,47 a 2% reduction for multiple admitters,54,66 a reduction from 935 to 840 admissions in a year for COPD patients,72 and a significant effect size following a service reconfiguration compared with other hospitals (p = 0.0002). 39 None of these studies was of comparator design.
The patient populations and locations for interventions in this group of studies were diverse, including children in primary care,66 older adults,47 people with COPD,72 urgent mental health care in a hospital,78 and mixed patients in acute and community settings. 74 There was no clear pattern to the effect of different intervention elements. Case manager, MDT and staff relocation interventions were in the no-effect group; MDT and large-scale service reorganisation interventions were present in the group indicating an increase; and MDT, ICPs and role change interventions were in the group of studies reporting reduced admissions.
Findings from UK qualitative studies were also equivocal, with one reporting no perceived difference100 and one reporting a perceived reduction in length of stay. 106
Three systematic reviews168,177,191 together provide unclear evidence, with one suggesting no difference,168 one suggesting that the effect was unclear177 and another suggesting a reduction. 191 All three reviews examined interventions for older adults and encompassed case management in hospital and the community,168 and community integrated care initiatives generally. 177,191 The single international comparator study236 reported that the risk of having at least one unplanned hospital admission was lower in the intervention group after adjustment (odds ratio 0.39). The single international non-comparative study243 outlined that unplanned charges (arising from unscheduled hospital admissions or ED interventions) decreased by 51% at 12 months and a cumulative 64% by 24 months.
Non-emergency and scheduled admissions
Eight UK studies reported in nine papers,39,41,45,46,63,71,74,77. 81 including two studies reported in three papers45,63,74 with comparator designs, indicated a reduction in admissions. Reductions were of 1% in the number of inpatient admissions in gynaecology;41 4% in the number of hospital admissions for a range of conditions following service reconfiguration (most of the reduction was reportedly in the case management interventions);45,74 15% in the number of admissions for older patients comparing general practices (no reduction pre–post intervention) following relocation of a social worker to a general practice;63 31% in the number of admissions for a subgroup of patients from early-engaging practices in a reconfiguration intervention;83 there was a decrease in standard admissions ratio;39 a decrease of the number of inpatients from 91% to 35%;71 a reduction from 19% more admissions than comparable areas to 8.7% more than comparable areas following a multielement initiative;77 and a reduction in the number of excess bed-days from 520 per 1000 patients in 2007/8 to 232 in 2008/9. 46
Two evaluations of ICPs44,80 reported no significant effect on admission rates (1) for children with asthma (comparator design)44 or (2) for admissions for fractures/falls in the elderly. 80 One of these studies80 also found indication of an increase in the number of admissions for patients with diabetes mellitus (p = 0.014) compared with before the intervention.
Another study43 reported a slight increase in the number of admissions to a community hospital following a hospital merger. Stokes et al. 82 similarly reported a very small increase (of 0.01 admissions per patient per month, effect size 0.03) following a MDT intervention. Another study of older adults81 also reported a slight increase of 0.64 additional admissions over a 12-month period compared with a control group for a MDT intervention with a support worker and an out-of-hours service. A study by Wilberforce et al. ,86 also in older adults, found that more-successful (higher-integration) teams had higher admission rates, and Clarkson et al. ,42 another study in older adults, found that integrating services could increase rates of care home admissions (hazard ratio 0.73; p = 0.12).
Systematic reviews have reported reductions in numbers of admissions of 24–31% among studies of ICPs,173 reductions in numbers of nursing home admissions in four of five studies and in hospital admissions in two of three studies of case management,177 reduced hospital admission in 10 of 17 studies of multidisciplinary interventions,180 and reduced admission in studies of case conferences. 191 In contrast, one review of patient-centred medical homes170 concluded that there was no significant effect on admissions.
The evidence from the international literature is mixed, with six comparator studies201,206,210,211,228,236 evaluating a range of intervention types indicating no significant effect on admissions (all but one201 of these studies was in older adults). Nine studies195,199,200,205,207,219,221,222,306 reported a reduction in the number of admissions for patients with a range of conditions; patients included older adults, general groups and those with chronic conditions. One of these studies195 found that, although admissions rates in one intervention area reduced, in another area they increased. Three international non-comparator studies echo the mixed picture, with two indicating reductions in numbers of admissions243,251 and one indicating no significant effect. 261
Readmission
Studies report conflicting findings regarding the effect on readmission rates, with three UK studies reported in four papers36,38,39,54 (including one study with a comparator group54) indicating no significant effect on (1) older at-risk patients,54 (2) patients originally admitted as emergencies38,39 or (3) patients with COPD following a hospital at home intervention, service reconfiguration or a case management intervention. 36 Two studies indicated an increase in readmissions,69,82 although one of these69 was of very poor quality. The other82 found only a slight increase in 30-day readmission (effect size 0.03). One UK study of a community outreach team for acutely ill children66 reported a reduction in the number of readmissions of 17.3%.
International studies carried out in populations of older patients, those with chronic conditions and those in acute care support the lack of evidence of a reduction in numbers of readmissions, with all five international comparator studies (reported in eight papers201–203,209,210,216,218,221) suggesting no statistically significant effect. In contrast, two international reviews171,180 suggested reduced numbers of readmissions, although a third182 concluded that there was conflicting evidence.
Accident and emergency department attendance
We identified three non-comparator design UK studies66,74,87 that reported accident and emergency (A&E) (ED) attendance outcomes. Two66,87 suggested a reduction of 5% and 69% for children and older adults, respectively, following varied interventions. The other74 reported no change (p = 0.04). One review170 concurred that interventions may lead to a reduction for patients with co-occurring conditions. A meta-analysis was conducted in this study, and found that patient-centred medical homes reduced the number of emergency visits [risk ratio 0.81, 95% confidence interval (CI) 0.67 to 0.98]. A second review197 found that although A&E attendance fell by 10%, this was not statistically significant.
Five of the international comparator studies also reported that A&E attendance reduced following interventions by 20.8%,199 by 57%,200 by 12.2 per 1000 patients annually,207 from 705 to 396 attendances (p = 0.03)210 and from 20.7% to 3% of patients visiting annually (p = 0.02). 237 The interventions included multiple elements of integration, an Accountable Care Organization model and a home-based team (they were delivered to older adult patients in all but two studies199,200). One non-comparator international study244 supported a reduction in A&E attendance in a general population.
Five studies reported in seven papers194,201–203,217,228,239 evaluated MDTs or integrated home care and found no significant effect on A&E attendance. Only one of the studies239 was not carried out in older adult patient groups, but in patients with a psychiatric diagnosis instead. Two studies reporting case management interventions in older adults found that the number of visits to an ED increased (odds ratio 1.28; p = 0.02;211 46% in the control group vs. 32% in the intervention group; p < 0.001218). The studies report opposing trends, with the first study211 finding an increase in the second year but not the first, and the second study218 finding a greater increase in the first year of the intervention.
Outpatient appointments
The UK evidence reports a reduction in outpatient appointments in all four studies outlined in five papers45,58,67,74,81 of 20–40% (one74 at the individual patient level only and not when analysed by general practice). The initiatives evaluated included a variety of interventions for a range of conditions. In contrast, one international comparator study of an ICP for older adults found no reduction in outpatient clinic use. This finding was supported by a non-comparator study of all-inclusive care for older adults,261 which similarly found no significant effect.
Prescribing rates
Few studies examined the effect of new models of integrated care on rates of prescribing. Among UK studies, no difference in prescribing rates is reported in one evaluation of an ICP,62 whereas a reduction is reported in another. 85 A systematic review158 found that pharmacy involvement in MDTs led to ‘improved prescribing’. ‘Improved prescribing’ was also reported in an international non-comparator study. 245
Access to resources
One systematic review examining UK-only evidence161 suggested a positive effect on access to resources, in that integrated working enabled organisations to access each other’s resources (e.g. training). An international non-comparator study244 described increased access to technical services.
Quality of care
Perceived quality
The outcome of ‘quality of care’ presented challenges in analysis, as a result of the varied ways in which the construct was defined and considered. Four UK intervention studies45,64,73,148 reported on staff perceptions of quality of care following service redesign, case management or integrated pathway interventions in hospital or primary care settings for older adults;45 general caseloads;73,148 or patients with clostridium difficile infection. 64 All four studies reported that staff-perceived quality of care had improved. One of these64 reported that 91% of staff perceived improved patient care and treatment following the introduction of a pathway. In other studies, it was found that 54% of staff perceived that patient care had improved,45 ‘more than half’ of staff reported improved services148 and 91% of staff reported that improved relationships would have or had improved patient care. 73
Four systematic reviews163,165,182,186 and four international comparator studies reported in seven papers201–203,213,215,220,235 reported the effect on quality of care in terms of staff or patient perceptions. A review of reviews of organisational and/or financial interventions165 concluded that there was some evidence of benefit to quality of patient care from interventions including MDTs, case management and provision of primary care in or alongside EDs. The authors highlighted the wide range of measures that were used as indicators of ‘quality of care’, and commented that some studies did not specify what aspect of quality was measured. 165 The outcomes considered included, most commonly, patient satisfaction, but also aspects that we defined as process improvements (such as continuity or compliance with guidelines), together with clinical outcomes. Another review163 concluded that there was moderate evidence that comprehensive care approaches could improve patient perceptions of improved care quality. Peikes et al. 186 reported that physicians perceived that quality of care had improved following a care co-ordination and lifestyle programme. Mason et al. 182 examined the effect of financial integration on quality of care. Studies in this review typically assessed perceived quality using surveys of staff, patients or carers (including aspects that we have identified separately, such as satisfaction and access to care), reporting mixed findings.
One international comparator study reported in three papers201–203 supported the finding that quality of care was perceived by patients (older with chronic conditions) to have improved. One study evaluating MDTs and shared training in Norway226 provided contrary findings, reporting no difference in staff-perceived service quality.
Quality standards
Three studies evaluated quality not in terms of perceived quality of care but in terms of whether or not recommended procedures (such as vaccination, screening, prescribing or testing) had been carried out. One215 found a significant improvement in procedures from before to after the collaborative team intervention (p = 0.0013), although the authors concluded that the intervention was not cost-effective. Another study235 evaluated three general practices that were Accountable Care Organizations. The authors reported percentage point differences between the three practices and comparison groups of 0.7, 2.6 and 4.7, respectively, and that ‘the three practices were superior to their comparison group peers on all care quality measures’ but they did not provide analysis of statistical significance. They noted that the before–after differences for the practices were not statistically significant, and that in fact the performance of one practice had declined over the time period. Fagan et al. 213 evaluated the effect of pay-for-performance incentives for care co-ordination. They concluded that there was either no significant difference between the intervention and comparison groups in rates of screening/testing for diabetes mellitus, or that the improvement was significantly lower for the intervention sites.
Patient satisfaction
A large body of literature reports a positive effect on patient satisfaction following the introduction of integrated care initiatives. Nine UK studies35,37,43,46,51,58,66,75,83 across a range of conditions and services (including diabetes mellitus, complex needs, children’s services, complex and vulnerable patients in primary care, and hospital gynaecology) and following a variety of interventions (including whole-service redesign, MDTs and ICPs) reported increased levels of patient satisfaction. Only two studies74,78 highlighted less positive outcomes. The first of these74 noted that following a case management intervention, older patients reported that they were less able to see the GP or nurse that they preferred. A second study evaluating the siting of a psychiatric nurse in an A&E department78 found no difference in patient satisfaction levels.
Four UK qualitative studies121,130,138,146 support the finding that interventions including MDTs and service redesign result in increases in patient satisfaction. One qualitative study,108 however, reports more mixed views of a mental health integrated partnership, and another119 describes how satisfaction may have been higher with health services but not with social care services.
The systematic reviews support an increase in patient satisfaction, with 118,88,160,163,164,170,177,180,188,189,192 concluding that the evidence suggests a positive effect. Four international comparator studies197,214,228,237 similarly report increased satisfaction among older, acute and paediatric patient populations following service integration, case management and patient-centred medical home interventions. One of these studies197 highlighted that although patients reported greater satisfaction, this was not shared by carers. This was echoed by another study of carers,223 which found no impact on satisfaction with care and support services. Another study209 reports a lack of significant difference following home-based integrated care (p = 0.31) and a final study of an integrated care protocol to prevent overuse of acute services212 reports that patients’ satisfaction was at the same level as prior to the intervention.
Meeting patient preferences
Three UK studies of MDTs or ICPs in end-of-life/frail older patients34,53,144 highlighted that the interventions were able to increase the fulfilment of patient wishes to die outside a hospital setting. The percentage of deaths in a hospital reduced from 15% to 8% in one study,34 the percentage of home deaths increased from 19% to 23% in another53 and the percentage of home deaths was 55% in the third study,144 compared with a national average of 33% (however, the requested hospice placement had only been achieved for 34% of patients in this study).
Time spent in accident and emergency departments
One UK study44 reported no difference in time spent in A&E for children with asthma following the introduction of an ICP compared with standard care. In contrast, an Australian comparator study221 found that older adults spent around 45 minutes less in the ED following a new co-ordinated service, although this is of borderline significance (p = 0.0575; a reduction from 496.3 minutes to 435.7 minutes).
Numbers of incidents and complaints
One UK non-comparator study71 reported that the frequency of aggressive incidents among inpatients with mental health and behaviour problems was reduced following the introduction of an integrated assessment and treatment service. Another UK before-and-after study39 reported that numbers of formal complaints, incident reports and patients leaving before treatment declined following service redesign and the establishment of an integrated emergency assessment unit.
Length of waiting time
One UK study55 found that following the introduction of an ICP in urology, the time between referral and contact and the time from referral to treatment, both reduced. ICPs were also found to reduce waiting times during the management of inpatients with diabetes mellitus, including reducing time before giving intravenous infusions. 85 The reconfiguration and integration of gynaecological services was found to reduce waiting times for scans (p = 0.0054) and getting blood test results, although not for blood testing. 41 Improvements in waiting times for assessment by a social worker were reported in a comparator study evaluating the relocation of a social worker to a general practice. 63 Waiting time for assessment/diagnosis of autism and related disorders in children was reduced from 2 years to 5 or 6 months following the introduction of a multiagency team with a filtering panel and joint assessment. 75
Five qualitative studies104,119,131,146,147 supported the finding of a reduction in waiting time. There were perceived reductions in waiting time for allocation to a social worker, admission to a ward, waiting for assessment and waiting for services generally.
In contrast to these positive reports, one systematic review172 found that the length of wait for surgery for older patients with hip fracture was reduced in only one of three studies analysed. A review of UK literature that was carried out in 2010181 evaluated speed of delivery following the introduction of integrated health and social care teams. The 18 included papers were reviews, qualitative studies, surveys and one non-RCT. The authors concluded that the speed of response may be quicker, and services may be more responsive, as a result of introducing MDTs. 181 However, the data to support this conclusion are limited, and the quality of the review was assessed as weak (meeting only one of the quality criteria).
One international comparator study196 reported a reduction in waiting times for nursing home placements among older adults (p < 0.05), and another203 reported no effect on waiting times for appointments among patients with chronic conditions. The two non-comparator studies258,308 are similarly divergent, with one indicating a reduction in waiting time for cancer testing258 and another suggesting an increase in referral delay. 308
Identification of unmet needs
There was no UK evidence from intervention studies in relation to the outcome of the identification of unmet needs. One UK qualitative study106 reported perceptions of increased unmet needs being identified and more referrals being made in care homes, and another142 reported the potential for increased referrals being caused by unmet needs being identified. One US systematic review of studies evaluating the Program of Research to Integrate Services for the Maintenance of Autonomy for older adults188 found that two of the 45 studies reported decreased prevalence of unmet needs in the community (p < 0.001). A comparator study,214 also from the USA, evaluating a medical home/case co-ordination intervention for children with special needs reported that GPs perceived that the programme had identified unmet child and family needs.
Access to services
Five included (non-comparator) UK intervention studies49,55,73,86,87 reported that access to services in the community and/or specialists/intermediate care had increased. These studies evaluated MDTs, general service redesign or integration of hospital and community services. One UK qualitative study135 reported perceptions among staff that colocation of health and social care services for older people had enabled access. Two systematic reviews154,182 also reported that access to services had improved. The first of these154 reported increased access to clinical assessments and interventions following the introduction of an ICP for stroke patients, and the second182 concluded that there is largely positive evidence that interventions that integrate financial or resource flows across both health and social care can improve access to care. A systematic review of UK health and social care integrated interventions161 described the key to successful integration as being the ability of patients to access care at home.
Two international comparator studies reported in four papers201–204 supported the finding that MDT and multicomponent home care unit interventions improve access to services (for patients with chronic conditions and severe heart failure). Two further comparator studies95,204 describe the use of domiciliary services increasing as a result of increased access,195 and the ease of access to staff as being an important enabler of care delivery at home. 204 A non-comparator study evaluating a wide-scale integration model245 similarly reports improved access to services for patients. Another comparator study evaluating a patient-centred medical home in the USA257 also reported that patient access to care had improved.
Staff working experience
None of our included UK studies reported staff working experience outcomes. Two systematic reviews (of patient-centred medical homes and MDT interventions)170,181 concluded that integrated models can improve staff working experience. The first170 reported that working experience improved to a small to moderate extent, and the second181 reported that initiatives increased staff job satisfaction. However, an international comparator study from Norway226 found no effect on staff exhaustion or work engagement. A positive effect on job satisfaction was indicated by a qualitative study from Spain. 282
System-wide impacts
In this section, we outline evidence related to system-wide impacts across health-care delivery. A systems perspective recognises that outcomes within one part of a system may have a ‘knock-on’ effect on other elements. In particular, in relation to health services, there are impacts on the elements of community services versus hospital services and health care versus social care. The review identified evidence related to the impact on community care activity, secondary care activity and health-care costs and utilisation.
Community care activity
Three UK studies (in four papers37,51,76,86) reported data regarding community care activity following interventions. Hawthorne et al. 51 found that the proportion of people with diabetes mellitus receiving their care in primary care settings increased from 48% to 67% over a 6-year period following an extensive service redesign. An evaluation of multidisciplinary community teams86 found that highly integrated teams provided a broader range of community services, but at a greater cost. The authors reported considerable variation between the teams with regard to the number of patients receiving care. A study of community mental health teams37,76 found that a gateway worker role to integrate primary care and mental health teams reduced inappropriate referrals. A UK qualitative study106 reported the perception of increased referrals to allied health professionals in the community following the introduction of a care home support team.
Two systematic reviews provide contrasting findings, with one189 indicating reduced primary care usage following a case management intervention [utilisation of primary and non-specialist care in the short term was –0.08 (95% CI –0.22 to 0.05; p < 0.001) and in the long term was –0.10 (95% CI –0.29 to 0.09; p < 0.001)]; the other182 concluded that there is some evidence for an increase in the usage of community services, but it is unclear regarding the impact on residential care, with some studies reporting a greater likelihood of nursing home admissions in integrated care initiatives. This lack of clarity is reflected in the included international comparator studies, with two indicating increased usage of community services (domiciliary, day care or respite),195,230 two indicating reduced use of home care or nursing home placements155,158 and one reporting no significant effect on nursing home placements at 12 months or 18 months following the introduction of a collaborative care model. 206 One non-comparator international study evaluating an Accountable Care Organization244 found that community care activity increased.
Hospital/secondary care activity
One UK intervention study37 reported a reduction in secondary care activity of 9% for vulnerable/high-risk patients with an integrated care plan in place. Two UK qualitative studies report perceptions that numbers of referrals to secondary care reduced following service redesign,91 and following a new GP role. 138
Two systematic reviews concluded that the evidence regarding the impact on secondary care activity is unclear, with Mason et al. 182 reporting no impact in 11 out of 34 studies of integrated financial initiatives and Trivedi et al. 192 describing the evidence as ‘mixed’. One review155 described only one integration programme of those evaluated in a single study as appearing to reduce hospital-based services. De Bruin et al. 163 reviewed comprehensive care programmes and concluded that there is moderate evidence that comprehensive care can improve (i.e. reduce) inpatient health-care utilisation. In support of this finding, an international comparator study199 reported a reduction in the use of hospital services by older adults and a non-comparator study245 reported a reduction in specialist referrals.
Health-care utilisation
Two systematic reviews164,177 endeavoured to assess the evidence regarding the overall impact of integrated care on health-care utilisation. Eklund and Wilhelmson164 reviewed co-ordinated and integrated interventions for older adults, and concluded that outcomes related to health-care utilisation significantly favoured the intervention in only five out of nine studies. A review of case managed or integrated programmes177 reported that good-quality RCT evidence indicates that case management interventions may reduce nursing home admissions and hospital use. However, evidence, mostly from non-randomised trials, indicates that integrated care may increase service use.
Health-care costs
Outcomes that are reported, such as reduced length of hospital stay and reduced admissions, may be expected to have a positive impact on the cost of providing health care. Data regarding the cost of providing health-care services were evaluated in 14 UK intervention studies reported in 15 papers. 34,36,37,42,43,45,46,52,63,67,72,74,77,86,87 Only one of these studies86 reported increased costs. This study with a comparator group86 evaluated a multicomponent community intervention (including MDTs, single point of access, and joint care plans for older adults with mental health needs) and found that more-highly integrated teams provided an improved service, but at a 44% greater cost than teams that were not well integrated (monthly cost of £762 per patient in high-integration teams vs. £508 per patient in low-integration teams).
Five studies34,42,43,45,87 reported no significant impact on costs. The first of these was a before-and-after evaluation of a rapid response team intervention in palliative care,34 which found that hospital costs (mean cost of inpatient stay for the last 8 weeks of life) did not significantly change (£3066 prior to and £3019 after the programme) and that the costs for patients enrolled in the programme were little different (£3067). There was also no significant difference between the mean cost of patients in the programme and the corresponding mean costs for all end-of-life patients over the previous 2 years. The second study reporting no significant impact was a RCT of a care manager with integrated reporting and assessment for frail elderly patients. 42 The authors reported that there was little difference in health or social care costs, although there was a suggestion of an increase for the most-frail patients and slightly higher informal care costs. In another RCT (in patients with breathlessness due to advanced disease),52 an integrated palliative care service was found to have no significant impact on formal care costs [6-week mean costs were £1422 in the breathlessness support service group (95% CI £897 to £2101) and £1408 in the control group (95% CI £899 to £2023)]. However, the authors noted that costs varied greatly between individuals.
A before-and-after evaluation of a multicomponent organisational merger43 compared the estimated impact of reconfiguration with the actual impact. It describes that savings for the NHS trust had been estimated at the outset to be in the region of £0.3M to £12.5M per year. However, the targeted savings were not achieved (the trust ended the year in deficit, although this was not attributed to the intervention). A study evaluating 63 integrated care pilots in England45 highlighted the importance of evaluating costs across the whole system. The authors reported that cost gains attributable to reductions in emergency admissions were balanced by increased elective and outpatient attendance, with an overall cost-neutral outcome for secondary care costs (p = 0.36). However, the authors highlighted a 9% reduction in secondary care costs (£223 per patient; p = 0.01) for those pilots that included case management interventions. This finding regarding cost reduction for case management interventions for patients aged > 65 years is reported in the second paper from the study. 74
An evaluation of the POPPs87 calculated that a £1.00 additional spend on the projects overall would lead to an approximate £1.20 reduction in emergency bed spend. Spending on hospital-facing projects had a greater saving on emergency bed-day cost than community-facing projects, and larger projects were potentially more cost saving. The overall estimated secondary care cost reduction was £2180.43 per day, with an increase in primary care costs of £14.08 per day and an overall estimated cost reduction of £2166.35 per day. The authors concluded that there was an 80% probability that the programme was cost-effective, comparing project areas to areas with no projects. However, another report evaluating eight of the POPPs81 seems to provide contrary findings to the main project report. This study concluded that, overall, the interventions did not appear to be associated with a reduction in use of acute hospitals (which would seem to lead to doubt regarding the reported cost savings).
A redesign of services for COPD patients was reported to have led to a reduction in annual costs of admission, from £1772.865 to £1528.080. 72 Another study of services for patients with COPD36 evaluated a hospital at home service and found that the cost for patients in the integrated care group was £600 less than for patients in the comparator group (£1653 vs. £2256; p < 0.001). The additional costs for community nurse specialist visits and emergency contacts were less than the reduction in costs of hospital length of stay.
Another UK intervention paper37 reported cost savings of 17% (it is unclear how this figure was calculated) following the introduction of integrated teams for vulnerable people, with this saving described as being largely attributable to the reduced numbers of emergency admissions. Another team intervention with case management was estimated to have led to an annual cost saving of £54,111 on hospital beds. Purchasing beds in care homes avoided hospital costs and was estimated to save £33,200 annually. 46 Simmons et al. 77 compared cost outcomes for more-engaged general practices with those for less-engaged practices during a multicomponent integrated care initiative for patients with diabetes mellitus. They reported that the monthly tariff paid for hospital inpatient care reduced in practices that were fully engaged with the initiative.
Perceptions of the impact of new models of care on health-care costs in the UK studies were similarly mixed. One study91 reported staff perceptions that the costs of hiring agency staff were reduced. Another105 reported scepticism among staff that initiatives were cost saving.
Evidence from systematic reviews also indicates a lack of clear effect regarding costs. Two155,158 concluded that, in older populations, costs may be reduced. Another,160 also considering older adult populations, reported some limited evidence of cost reduction. One systematic review163 reported that there was moderate evidence that comprehensive care can improve health-care costs, although there was insufficient evidence for outpatient costs.
Six reviews170,180,182,189,191,192 concluded that there was no significant difference or that the effect was uncertain in a range of populations, including older adults, patients with chronic conditions and general populations of patients, for interventions including case management elements, patient-centred medical homes, financial integration and general service redesign.
One international comparator study reported in two papers215,220 found that the annual cost of providing a family health network programme in Canada was more expensive than usual care (CA$12,923 vs. CA$9222). The authors calculated that, in order to achieve the desired improvement in quality of care, a CA$407 additional spend is required for each 1% increase in quality. The costs of another intervention (team meetings and co-ordination in mental health treatment centres) were also found to be significantly higher, in terms of outpatient mental health and medication costs, than the cost of usual care. 233
No significant impact on costs was reported in nine papers (seven studies). 195,209,217,231,232,235,240,241,306 The patient groups and services included dermatology, neurology, general populations, older adults, asthma, COPD, cardiac and diabetes mellitus. Initiatives included integrated home care, case management, integrated nursing home care, an accountable care model and multidisciplinary care.
One study reported in two papers196,197 found that, although community care costs increased, there was a reduction overall following a multicomponent community intervention. Another study evaluating a model of care for older adults in the community198,199 also found reduced costs (estimated annual savings in emergency attendance/bed-days/admissions of around US$2M at a cost of US$1M to deliver). In another study reported in two papers,207,208 financial change associated with an accountable care model in primary care led to estimated savings of US$114 per patient, with savings being greater for the most-vulnerable patients. A care manager added to a primary care team was found to have reduced outpatient costs in another study (by US$594 per patient) in patients with depression, poorly controlled diabetes mellitus or coronary heart disease. 225,227 A multicomponent whole-population-based intervention for patients with chronic disease in Germany was reported to have led to a cost saving of 16.9% over 4 years. 219
A multidisciplinary doctor/nurse practitioner and restructuring model in a hospital was estimated to save US$978 per acute patient. 212 A reconfigured hospital discharge programme for patients aged > 18 years was found to have reduced combined hospital utilisation and outpatient costs by a mean of US$412 per patient. 222 A patient-focused ICP in hospital for patients with hip fracture was reported to have reduced treatment costs (US$9685 per patient post intervention vs. US$15,984 per patient pre intervention). 229 An integrated home care unit in a hospital resulted in lower total health-care costs (US$140,000 in intervention patients vs. US$205,000 in control patients). 234 Savings were reported particularly in emergency care and hospital transport.
A non-comparator study from the USA252 discussed the potential impact of a new payment system (as part of a patient-centred medical home approach). From data from two group practices in the pilot study, it was estimated that 23% of inpatient admissions would be potentially avoidable, leading to a saving of US$522,564 in averted medical costs.
Strength of evidence across studies
We adopted the four-item rating scale described in Chapter 2, Risk of bias and strength of evidence across studies, to evaluate the quantity and consistency of evidence for the service delivery outcomes and system-wide impacts reported in the included studies. The ratings were discussed at a whole-team meeting to establish consensus. We assigned individual ratings for the three groups of studies (UK studies, systematic reviews and international comparator studies) before providing an overall rating. We considered each of the groups separately as we were particularly interested in the similarities and differences between the UK evidence and studies from other countries.
The completed ratings are provided in Table 3. The outcomes reported by included studies are detailed in the first column, and each individual study is represented by either a plus symbol (+), meaning that the study reported an increase for this outcome, a plus/minus sign (±), meaning that the study reported no change for this outcome, or a minus sign (–), meaning that the study reported a reduction for this outcome. It is important to note that in order to maintain consistency across the data, we have reported increase versus reduction rather than positive versus negative outcomes. For some outcomes, an increase may be positive (e.g. increased patient satisfaction), while for others an increase may be negative (e.g. increased cost or increased length of wait), and for some outcomes it is uncertain whether an increase is positive or negative (e.g. increased access to services may be positive for patients but negative for cost of delivery).
Outcomes | UK literature | Systematic reviews | Non-UK comparator studies | Non-UK other studies | Overall rating | ||||
---|---|---|---|---|---|---|---|---|---|
Results | Strength-of-evidence rating | Results | Strength-of-evidence rating | Results | Strength-of-evidence rating | Results | Strength-of-evidence rating | ||
Resource usage | |||||||||
Number of clinician contacts | + – | Inconsistent evidence | + + – – – | Inconsistent evidence | – | Very limited evidence | Inconsistent evidence | ||
Number of GP appointments | + – – | Inconsistent evidence | Inconsistent UK-only evidence | ||||||
Length of stay | + ± ± ± ± ± – – – – – – – – – – – – – – – | Inconsistent evidence | – – – – – – ± ± ± ± | Inconsistent evidence | – – ± ± | Inconsistent evidence | + – – | Inconsistent evidence | Inconsistent evidence indicating no difference or reduced; one weak UK-only study reporting increase |
Number of unscheduled admissions | + + + + ± ± ± ± – – – – – – – | Inconsistent evidence | ± – – | Inconsistent evidence | – | Very limited evidence | – | Very limited evidence | Inconsistent evidence; UK-only evidence reporting increase |
Numbers of admissions/ inpatients | + + + + + ± ± – – – – – – – – – | Inconsistent evidence | – – – – ± | Stronger evidence | ± ± ± ± ± ± – – – – – – – – – | Inconsistent evidence | – – ± | Inconsistent evidence | Inconsistent evidence; UK-only evidence of an increase |
Readmission | + + ± ± ± | Inconsistent evidence | – – ± | Inconsistent evidence | ± ± ± ± ± ± ± ± | Stronger evidence | Inconsistent evidence; UK-only evidence of increase | ||
A&E attendance | – – ± | Inconsistent evidence | – ± | Inconsistent | + + ± ± ± ± ± ± ± ± – – – – – | Inconsistent evidence | – | Very limited evidence | Inconsistent evidence; less evidence of an increase |
Number of outpatient appointments | – – – – – – | Stronger evidence | ± | Very limited evidence | ± | Very limited evidence | Inconsistent evidence; UK stronger evidence of reduction | ||
Prescribing rates | ± – | Inconsistent evidence | – | Very limited evidence | – | Very limited evidence | Very limited | ||
Access to other resources | + (UK only) | Very limited evidence | Very limited UK-only evidence | ||||||
Quality of care | |||||||||
Perceived quality | + + + + | Stronger evidence of perceived increase among staff | + + + ± | Stronger evidence regarding perceived quality staff/patients | + – | Inconsistent evidence | Stronger evidence of perceived improved quality | ||
Quality standards | + – ± | Inconsistent evidence | Inconsistent evidence | ||||||
Patient satisfaction | + + + + + + + + + ± – QQQQ (increased) QQQ (mixed) | Stronger evidence | + + + + + + + + + + + | Stronger evidence | + + + + ± ± | Inconsistent evidence | Stronger evidence of improved patient satisfaction | ||
Patient preferences met | + + + | Weaker evidence | UK-only weaker evidence of positive effect | ||||||
Time in A&E | ± | Very limited evidence | – | Very limited evidence | – | Very limited evidence | Very limited evidence | ||
Number of incidents/complaints | – – | Very limited evidence | Very limited evidence | ||||||
Length of wait (contact, diagnosis, investigation and treatment) | – – – – – QQQQQ (reduced) | Stronger evidence | ± – | Inconsistent | – ± | Inconsistent evidence | + – | Inconsistent evidence | Inconsistent |
Access to services | + + + + + Q improved | Stronger evidence | + + | Weaker evidence | + + + + + | Stronger evidence of improved access | + – | Very limited evidence | Stronger evidence of improved access |
Unmet need identified | QQ (increased) | Very limited evidence | + | Very limited evidence | + | Very limited evidence | Very limited evidence | ||
Staff working experience | + + (UK only) | Weaker evidence | ± | Very limited evidence | + | Very limited evidence | Inconsistent evidence | ||
System impact | |||||||||
Cost of provision | + ± ± ± ± ± – – – – Q (no change) Q (reduced) | Inconsistent evidence | – – – – ± ± ± ± ± ± | Inconsistent evidence | + + ± ± ± ± ± ± ± ± ± – – – – – – – | Inconsistent evidence | – | Very limited evidence | Inconsistent evidence |
Community care activity | + + – Q (increased) | Inconsistent evidence | – + | Inconsistent evidence | + + ± – – | Inconsistent evidence | + | Very limited evidence | Inconsistent evidence |
Secondary care activity | – QQ (reduced) | Very limited evidence | – – ± mixed | Inconsistent evidence | – | Very limited evidence | – | Very limited evidence | Inconsistent evidence |
Health-care utilisation | ± ± (unclear effect) | Inconsistent evidence |
An example of how to read the table is as follows. The first row indicates that two studies from the UK44,87 reported the outcome ‘number of clinician contacts’: one reported an increase in the number of contacts and one reported a reduction in the number of contacts. Both studies are represented in green font indicating that they had a comparator group design. The evidence was therefore rated as being inconsistent. There were no systematic reviews that reported this outcome. Five non-UK studies of comparator design reported this outcome,203,204,206,217,218 two found an increase204,218 and three found a decrease;203,206,217 therefore, the evidence is rated as being inconsistent. One international non-comparator study249 reported a reduction; therefore, this evidence was rated as being very limited. Overall, the evidence was rated as being inconsistent.
The table depicts an extensive range of outcomes reported in the literature that are associated with new models of integrated care. The evidence was stronger for three outcomes: (1) that integrated care may lead to an increase in patient satisfaction, (2) that quality of care is perceived to improve and (3) that new models of integrated care may increase access. There was stronger UK evidence but inconsistent overall evidence regarding a positive effect on waiting times and outpatient appointments. We found weaker UK-only evidence that there was an increase in care meeting patient preferences.
Evidence regarding effect on the following outcomes was rated as inconsistent: number of clinician contacts; number of GP appointments; length of stay (little evidence of an increase); number of unscheduled admissions; number of admissions (the systematic reviews alone indicated stronger evidence of a reduction); number of elective admissions; readmissions (the international comparator studies alone indicated stronger evidence of no effect); attendance at A&E departments; quality-of-care standards; staff working experience (no evidence of a negative effect); cost of provision; community care activity; secondary care activity (no studies reported an increase); and overall health-care utilisation (systematic review studies only).
The rating of ‘very limited evidence’ (an insufficient number of studies) was given to the following outcomes: prescribing rates, access to other resources, time spent in A&E department, number of incidents/complaints, identification of unmet need and staff working experience.
We also identified strength-of-evidence ratings for the process change data and the influencing factors. (Tables 4 and 5). Regarding process changes, there were no elements in which we rated the evidence as stronger. We rated the evidence as being weaker regarding an increase in discharge planning, increase in timeliness/flow/co-ordination of care, increase in practitioner time, increase in knowledge sharing and reduction in variance in practice. We rated the evidence as inconsistent with regard to continuity of care, shared information, accuracy/completeness of recording of information, and organisational systems and processes. We rated the evidence as very limited with regard to patient relationship with health-care provider, patient understanding of treatment and duplication.
Process change element | UK literature | Systematic reviews | Non-UK comparator studies | Non-UK other studies | Overall rating | ||||
---|---|---|---|---|---|---|---|---|---|
Results | Strength-of-evidence rating | Results | Strength-of-evidence rating | Results | Strength-of-evidence rating | Results | Strength-of-evidence rating | ||
Service users | |||||||||
Relationship with health-care provider | Q (mixed) | Very limited evidence | Very limited evidence | ||||||
Patient understanding of treatment | Q (improved) | Very limited evidence | QQ (+) | Weaker evidence | Very limited evidence | ||||
Care provision | |||||||||
Discharge planning | + + | Weaker evidence of improved discharge planning | + | Very limited evidence | Weaker evidence of improved discharge planning | ||||
Timeliness/flow/co-ordination of care | + QQQQQQQQQ | Weaker evidence of improved flow | + + | + ± | Inconsistent | + Q | Weaker evidence of improved flow | Weaker evidence of improved flow of care | |
Continuity of care | + | Very limited evidence | + – – – | Inconsistent evidence | + + + | Stronger evidence | Inconsistent evidence | ||
Variance in practice | – – – Q (improved) QQ (issues) | Inconsistent evidence | – – | Weaker evidence | Weaker evidence of reduced variance | ||||
Duplication | QQ (reduced) | Very limited evidence | – | Very limited evidence | Very limited evidence | ||||
Practitioner time | + Q (reduced) QQQQQQQQQ (increased) | Weaker evidence of increase | + | Very limited evidence | – – Q (increased) | Inconsistent | Weaker evidence of increase | ||
Information | |||||||||
Shared information | + QQQ (increased) QQQQ (limited) | Inconsistent evidence | Difficult | Very limited evidence | ± QQ (issues) | Inconsistent evidence | |||
Accuracy/completeness of recording of information | + + + Q (concerns) | Inconsistent evidence | + + – | Inconsistent | + | Very limited evidence | Inconsistent evidence | ||
Shared knowledge | QQQQ (increased) | Weaker evidence of increase | + | Very limited evidence | Q(+) | Weaker evidence of increase | |||
Organisational systems and processes | QQQQQQ (challenges) | Weaker evidence of difficulties in achieving change | ± ± | Weaker evidence of no change in funding processes | ± ± + + | ± | Very limited evidence of no change in systems | Inconsistent evidence regarding change in organisation |
Outcomes | UK literature | Systematic reviews | Non-UK comparator studies | Non-UK other studies | Overall rating | ||||
---|---|---|---|---|---|---|---|---|---|
Results | Strength-of-evidence rating | Results | Strength-of-evidence rating | Results | Strength-of-evidence rating | Results | Strength-of-evidence rating | ||
Patient related | |||||||||
Focus | QQQQQQQ | Stronger evidence | Stronger UK-only evidence | ||||||
Patient engagement | QQQQQQQQQQ | Stronger evidence | SR | Very limited evidence | + | Stronger evidence | |||
Concerns with sharing of patient data | QQQ | Weaker evidence | Very limited evidence | ||||||
Workforce related | |||||||||
Emotional response to change | Q | Very limited evidence | + | Very limited evidence | Very limited evidence | ||||
Professional identity | QQQQQQQ | Stronger evidence | Stronger UK-only evidence | ||||||
Role boundaries | QQQQQQQQQQQQQQQQQ | Stronger evidence | SR | Very limited evidence | Stronger evidence | ||||
Stability of workforce | QQQQ | Stronger evidence | + | Stronger evidence | |||||
Employer | Very limited evidence | Very limited evidence | |||||||
Power and hierarchies | QQQQQQQQQQQQQ | Stronger evidence | SR | Very limited evidence | Stronger evidence | ||||
Prior co-operation | QQQQQ | Stronger evidence | Weaker evidence | ||||||
Training | QQQQQQQQQQ | Stronger evidence | SR SR SR | Weaker evidence | Stronger evidence | ||||
Staff engagement | + QQQQQQQQQQQQQQ | Stronger evidence | + + + + | Stronger evidence | |||||
Management and leadership | |||||||||
Local leaders/champions | QQQQQQQQQ | Stronger evidence | Stronger UK-only evidence | ||||||
Effective leadership | QQQQQQQQQQQQQQQ | Stronger evidence | SR SR SR SR SR | Stronger evidence | Stronger evidence | ||||
Support for innovation | QQQ | Weaker evidence | Very limited evidence | ||||||
Organisation and system | |||||||||
Resources | QQQQQQQQQQQQ | Stronger evidence | Stronger UK-only evidence | ||||||
IT | QQQQQQQQQQQQQQ | Stronger evidence | SR SR SR | Weaker evidence | Stronger evidence | ||||
Clarity of vision | QQQQQQ | Stronger evidence | SR | Very limited evidence | Stronger evidence | ||||
Organisational culture | QQQQQQQQQQQQ | Stronger evidence | SR SR SR SR SR SR | Stronger evidence | Q | Very limited evidence | Stronger evidence | ||
Policy context | QQQQQQQ | Stronger evidence | SR | Very limited evidence | Stronger evidence | ||||
Audit/evaluation | SR SR SR | Weaker evidence | |||||||
Finance and governance | |||||||||
Commissioning | QQQQQQQ | Stronger evidence | SR SR | Weaker evidence | Stronger evidence | ||||
Governance | + + QQQQQQ | Stronger evidence | SR SR SR | Weaker evidence | Stronger evidence | ||||
Budgets | + QQQ | Weaker evidence | SR | Very limited evidence | Weaker evidence |
With regard to influencing factors, the evidence overall was consistent. The evidence was stronger regarding focus, patient engagement, professional identity, role boundaries, stability of workforce, power and hierarchies, training, staff engagement, local leaders/champions, effective leadership, resources, IT, clarity of vision, organisational culture, policy context, commissioning and governance. There was weaker evidence regarding previous relationships and budgets, and limited evidence regarding concerns with sharing of patient data, emotional response to change, employer and support for innovation.
We incorporated the strength-of-evidence findings into the logic model to develop a data visualisation portraying the elements of the pathway and associated level of evidence (Figure 5).
Applicability of the evidence
As part of the work, we aimed to develop an applicability framework, which would provide a structure for reporting the findings of the review with particular consideration of factors that may influence implementation and outcomes for new models of integrated care in different health service contexts. As outlined in Chapter 2, we extracted data regarding applicability from the included studies to populate the framework and then used this to scrutinise the data. Box 1 outlines the seven main applicability questions, and the full framework containing all the items can be found in Appendix 3.
-
How do the findings apply to different types of patients and populations?
-
What organisations and systems is the evidence applicable to?
-
What financial and commissioning processes might influence applicability?
-
What systems leadership elements might influence applicability?
-
What features of service(s) might influence applicability?
-
What features of the workforce might influence applicability?
-
What elements of the initiatives might influence applicability?
In the following sections, we summarise the data from the intervention studies regarding each element of the framework: population factors, organisation and systems factors, financial and commissioning factors, systems leadership, characteristics of health-care services, workforce features and characteristics of the initiatives.
Patients and populations
The first area of applicability we examined related to differences in implementation and outcomes for different types of patients.
Patient types and conditions
We examined the studies for data regarding the effect of new models of integrated care on different conditions and patient types, to explore whether or not the evidence might be more applicable to some patient groups than others. One systematic review170 reported that most studies examined involved the elderly (patient-centred medical home interventions); another189 reported that studies tend to be in broad populations with frailty, chronic illness or high utilisation rather than with clinical conditions (case management interventions); and another191 reported that many patients included in the studies had supportive rather than curative needs. Figure 6 indicates the patient types and conditions (when it was possible to identify them) for the studies included in our review. The largest group was studies of older adults, with only small numbers of studies examining other groups.
There were only limited data regarding the comparison of outcomes for different patient groups. A review by Tieman et al. 191 concluded that patients with poorly controlled diabetes mellitus may be priorities for targeting. A mixed-methods UK evaluation of health and social care integration147 found that it did not reduce length of stay for coronary heart disease patients, but there was a reduction for patients with diabetes mellitus and those with COPD. A UK before-and-after study59 evaluating ICPs in a teaching hospital and a district hospital found that there were statistically significant reductions in length of stay for only about half of the conditions analysed, with highly significant reductions for varicose vein operations, transurethral prostatectomy and gynaecological abdominal surgery. A positive effect of ICPs on hospital length of stay for hip, but not knee, arthroplasty was found in another UK intervention study. 65
Interdisciplinary primary care was found to be effective in reducing costs for heart patients only (2/14 studies) in a systematic review. 158 In contrast, a greater reduction in use of hospital services was found for COPD rather than heart failure in an international comparator study. 200 Relational co-ordination and integrated care delivery were significantly higher in integrated geriatric units than in other locations (p < 0.001) in another international comparator study. 254
Several studies indicated that targeting interventions to particular populations may lead to positive outcomes. Authors of an international comparator study205 reported that care co-ordination programmes can reduce the need for hospitalisation in targeted (chronic conditions) populations. Four of the 11 programmes reduced hospitalisations by 8–33% in high-risk groups (other programmes did not achieve this and one increased expenditure). Similarly, there was a reduction in hospital use in targeted high-risk groups only in a US study of a co-ordinated Medicare intervention. 205 Another study in an Accountable Care Organization in the USA207 estimated that there were greater cost savings from interventions targeting vulnerable patients. An international comparator study210 reported that in patients at high risk of hospitalisation, costs were reduced in the third year following implementation (but not earlier), suggesting that savings were made in the longer term.
A further international (non-comparator) study252 described chronic illness as offering the greatest opportunity for benefit, in terms of cost reduction from improved service co-ordination via a medical home intervention. Dorr et al. 211 found that a subgroup of patients with diabetes mellitus and complex morbidities who received a care management intervention had significantly fewer hospitalisations than usual care patients. The authors highlighted that care for diabetes mellitus may have been affected differently by the intervention than care for the other chronically ill older patients. A study from Australia195 described how changing recruitment criteria so that patients at risk of hospitalisation were included (alongside those with COPD, diabetes mellitus or cardiac conditions) changed a deficit (when compared with usual care) into a benefit in terms of saving on hospitalisation costs. The authors concluded that this indicated that appropriate targeting of interventions was important.
A review of care co-ordination191 also suggested that interventions may need to be tailored to particular populations and diseases, and may only be required in more-complex conditions. It was suggested in another review153 that care pathways may be better for more predictable conditions such as acute care, in which care trajectories are more predictable, but were not flexible enough for diverse needs. Tucker et al. 84 (a UK intervention study) concluded that ICPs led to integrated care to a greater extent for long-term conditions than for episodic care such as emergency services. In contrast to these recommendations for particular interventions for targeted groups, a study of integrated home care217 concluded that the approach was suitable for all patient groups, settings and organisations.
We examined the data regarding outcomes and impacts for studies in the two largest subgroups of studies (older adults and populations described as having complex needs) compared with the outcomes and impacts for all studies (Tables 6 and 7). The most-commonly reported outcomes in this set of studies were costs, emergency admissions, other admissions, patient satisfaction, length of stay, access and service usage.
Outcomes | Overall strength of evidence | Older adult studies | ||
---|---|---|---|---|
UK results | International comparator results | Strength of evidence | ||
Number of admissions | Inconsistent | – – – – – – ± ± | ± ± ± ± – – – | Inconsistent |
Number of emergency admissions | Inconsistent | + – – ± ± | ± – | Inconsistent |
Length of stay | Inconsistent | – – | ± | Inconsistent |
Patient contacts/service usage | Inconsistent | – – | + – – – – – ± | Inconsistent |
Patient satisfaction | Stronger evidence of increase | – + | Inconsistent | |
Access | Stronger evidence of increase | + + | Weaker evidence of increase | |
Costs | Inconsistent | ± + | – ± + | Inconsistent |
Outcomes | Overall strength of evidence | Studies of patients with complex needs | ||
---|---|---|---|---|
UK results | International comparator results | Strength of evidence | ||
Number of admissions | Inconsistent | – – – – | Stronger evidence of reduction | |
Number of emergency admissions | Inconsistent | ± | – | Inconsistent |
Length of stay | Inconsistent | – | – | Weaker evidence of reduction |
ED use | Inconsistent | – | – – | Stronger evidence of reduction |
Patient satisfaction | Stronger evidence of increase | + | + | Weaker evidence of increase |
Costs | Inconsistent | – | ± | Inconsistent |
As Table 6 indicates, there were no clear signals regarding the effect of integrated care initiatives in older adult populations. There was weaker evidence of increased access (as with the strength of evidence across all studies), and only two studies223,237 reported patient satisfaction, with opposing findings in contrast to the wider literature, indicating stronger evidence of increased satisfaction.
As Table 7 indicates, the evidence for new models of integrated care for patients described as having ‘complex needs’ suggests stronger evidence for positive outcomes in terms of reduced numbers of admissions and ED use, and weaker evidence regarding reduced length of stay in contrast to the wider evidence base. It is important to note that the UK findings are from only two studies using non-comparator designs,37,46 and the eight non-UK studies195,199,211,214,216,221,236,237 were all identified as having potential sources of bias, including variable patient selection and contamination between groups,195 self-selected groups,199 differences in group characteristics,211 high dropout rates,214 incomplete data216 and issues of participant selection. 221
As numbers were small for the other population groups, we briefly describe the types of intervention used and the outcomes. There were nine studies evaluating interventions in children and young people. 35,44,66,68,75,200,214,226,238 The interventions were (1) ICPs, (2) MDTs, (3) case co-ordinator and (4) multielement. The studies evaluating pathways35,44,68 reported (1) positive outcomes of improved communication, (2) staff-perceived improved quality and (3) improved patient satisfaction; however, increased staff contact; no difference in ED attendance or admissions; and variable outcomes regarding length of stay (reduced or no difference). The studies evaluating MDTs66,75,226 reported reduced A&E usage, reduced length of stay, improved patient satisfaction and reduced waiting times, but no difference in staff-perceived quality of care. Multielement interventions200,214 were reported to have reduced A&E attendance, number of admissions and length of stay and to have increased patient satisfaction. A case co-ordinator was found to have improved co-ordination between services. 238 Three of these studies44,68,200 were in children with respiratory difficulties. There were four other studies36,72,194,195 of patients with respiratory conditions (all COPD). For the group of studies of patients with respiratory conditions, the studies evaluating service redesign or multielement interventions36,72,200 reported reduced length of stay and A&E use, and inconsistent findings regarding admissions and costs. The studies reporting introduction of care pathways44,68 found perceived improved quality, but no difference in length of stay, use of A&E or admissions. A home-based case management intervention also reported improved perceived quality, but no effect on A&E use. 194 In other studies, ICPs did not have an effect on length of stay or admissions,80,85 and case co-ordination had no significant effect on care or costs. 85
We could identify few data regarding outcomes specific to patients with diabetes mellitus. A number of studies included this condition alongside other patient characteristics (such as older adults with or without diabetes mellitus) or patients with or without cardiac conditions or mental health problems. Studies evaluating service redesign for this patient group51,77 indicated improved satisfaction and reduced numbers of admissions. There were a small number of studies of other identifiable clinical groups such as patients with heart failure or people with mental health difficulties. The available evidence related to a mix of interventions and outcomes, with no clearly identifiable trends.
Level of severity of conditions
Many of the studies of older adults described the patient group as ‘frail elderly’. One of these studies48 also described the patients as being ‘high risk’. A study evaluating an intervention for patients with COPD36 described many patients requiring home support. The interventions in palliative care settings were often in patients with advanced disease. 52,53,56,57,79,194,204,234
Data regarding differential effects according to disease severity were provided by four studies (five papers). 42,195–197,222 A UK intervention study of integrated assessment42 reported that for the frailest individuals, there was an indication that the intervention may increase costs. An international study222 found that the intervention (a hospital discharge programme) was only effective in reducing admissions in those who were particularly at risk of hospitalisation. In contrast, another international comparator study196,197 found that reduced cost was more likely in patients with severe chronic diseases, with several disabilities or who were living alone. The total duration of hospital stays reduced for patients with the highest number of functional disabilities.
Levels of deprivation
The literature included a number of studies that described their populations as living in areas of deprivation. 43,48,51,63,76,81,82,87,210 Outcomes for this group of studies varied, with some finding positive effects such as reduction in admissions or service usage,63,87 whereas other found a lack of effect,48,81,82 increased community services51 or increased costs. 43,210 Three additional studies38,80,195 provided data related to levels of deprivation. One study of the North West London Integrated Care Pilot80 reported that emergency admission rates were higher among patients from deprived areas. An international study195 reported that patients who were perceived by staff to have benefited most were those who lived in difficult circumstances. A UK study of a hospital ED redesign38 examined the potential effect of adjusting for deprivation on admissions, but found that this made no difference to the data.
Socioeconomic diversity
There are few data regarding socioeconomic diversity in the included literature. A study from the UK87 noted that there were few patients from black and minority ethnicity backgrounds in the interventions that were evaluated, and that those involved in the projects tended to be older or recently retired and well educated with generally good health. In contrast, another UK study50 described the participants as socioeconomically diverse. One international study196 highlighted that risk profiles of patients included in the study varied according to the society and health system in which the patients lived.
Rural populations compared with urban populations
The included UK literature encompassed studies carried out in urban areas,38–40,50,52,67,72,73,78–80,82 rural or semi-rural areas43,47 and varied areas. 202,203 One43 of the only two studies carried out in a rural area notes that ‘moves to consolidate acute services in larger urban units will require new care models for rural and sparsely populated areas’.
Population density
The authors of one study50 described their intervention as being carried out in a compact high-density region (north-west London). Reports of initiatives in London were reported in a sizeable number of papers. 38–40,50,52,67,72,73,78–80 One of the systematic reviews190 noted that having geographical coverage of a population in an area was a central element of successfully integrated health care; however, ‘this may only be achieved in areas of more dense population and is not possible in rural or remote areas’.
Level of health needs
Only two studies (both UK)43,54 described the level of health needs in their study populations. The first of these43 reported taking place in a population that was relatively healthy and long-lived. The second study54 reported that unplanned admission rates in the area were higher than the national average.
Prevalence of condition
Two studies50,72 described a higher prevalence than the national average of disease in the population that was included. One reported a high prevalence of diabetes mellitus,50 and the other reported a high prevalence of COPD. 72
Other patient characteristics
One international study198,199 reported that the length of stay following the intervention had increased for older patients but had reduced in younger patients. In contrast, another international comparator study211 found that although higher hospitalisation rates were associated with higher age for the first year of the study, an evaluation of the second 2 years indicated that the difference was no longer significant. This study211 reported that higher hospitalisation rates were associated with multiple morbidities.
Features of organisations
Size of organisations
The size of organisations involved in new model initiatives varied greatly; for example, interventions could develop at a day unit,204 general practice46,47 or hospital. 44 At the other extreme, initiatives spanned large organisations that were delivering a specific type of care, for example end-of-life care. 34 The size of populations served by organisations, when reported, also varied; for example, a general practice might deliver care for just over 10,000 patients,46,47 whereas a hospital ED could deliver care to nearly 70,000 patients per year. 38 A study by Harris et al. 50 included a range of organisations with a sum patient population of 500,000.
Ways of reporting organisations and their size differ between included papers (with population size reported as total caseload, those with specific conditions, number of attendances, etc.), making applicability comparisons a challenge. However, it is clear that some interventions were small scale and local, whereas others were national or regional with associated funding.
Number and type of organisations
Initiatives were reported to involve from 1 to 59 organisations, with some studies not detailing specific numbers. Many included similar types of organisations working together, for example general practices54,214 and primary care consortia. 198–200 Harris et al. 50 included a large number of different organisations in their study, encompassing primary care, mental health and community health trusts, local authorities and voluntary sector organisations. A broad range of services was provided in organisations, covering populations from children to the frail elderly, as well as some specific conditions and situations.
Historical relationships between organisations
Several studies reported historical relationships between organisations, for example by having links through being part of the same consortium,198–200 units providing care within the same hospital39,40,44 or previous referral links. 67 Clearly any previous or existing relationship would provide a basis on which to build collaborations, which could be an advantage.
Geographical proximity of organisations
Organisations in reported initiatives could share the same hospital44 or be located within the same city,54,78,194,218 county,58,59 state195,206,213 or country. 205 Geographical proximity could have an impact on co-ordination of care as well as structural differences across regions.
Baseline performance of the study organisations compared with the national average
The standard of performance that organisations start at prior to the new model being implemented can also have a bearing on effectiveness results. For example, in one included study,221 four of the included nursing homes were rated as having a higher level of hospital and emergency care admissions than the national average. This could mean that effectiveness (i.e. measure of hospital admission outcomes) might appear greater following the intervention than in sites where baseline rates are not so high.
The policy environment at the time of the introduction of the initiative
Many initiatives were developed at times when local or national policies were calling for a reduction in health-care costs,47 waiting times309 and numbers of hospital admissions,46,63 as well as improved patient access,51 choice,41 streamlined care delivery50,72 and care that is local195 and/or organised around the patient. 58,216
Some policies were based on system ideals whereas others were in response to issues raised by patients/carer or health-care professionals; for example, increasing numbers of people want to die at home34 and there is a lack of clinical support in nursing homes. 221 A number of initiatives were based on more-specific guidance around a particular condition such as incontinence55 and pelvic surgery. 61
Other changes being made concurrently
Although authors of some included papers described evolving interventions, few reported concurrent activity within the organisation(s) being studied. Two studies66,196,197 included reports of funding change, one being budget cuts;66 the other study (reported in two papers)196,197 described an increase in the home care budget. A Spanish study228 was carried out in the context of a new advanced practice nurse role being established in general practices across the south of the country. Another study reported that work had already begun in the direction of the intervention before it was evaluated. 235 Concurrent change could have an impact on intervention plans, processes or ultimate effects, particularly if the changes are not allowed for in intervention design.
Particular elements of infrastructure or services
One study evaluating a merger between two units in the same hospital38,40 took place after the necessary extension on the building had been erected. Funding system change had been implemented prior to another initiative being started. 235 One study had a complex care team in place prior to evaluation. 46 There could be differences in rates of progress and costs between interventions that are starting up without any prior infrastructure or previous service delivery that aligns with the new model.
Particular admission routes
Some interventions involved changing existing referral/admission routes. For example, prior to the ICP being implemented, one study55 reported that primary care referrals were made to a gynaecology department without work-up. Owing to inconsistencies in this process, the initiative changed this to a direct-access nurse-led assessment. 55 Two studies (reported in three papers)39,40,67 included a change in admission and specialty assessment through the new service. Routes of admission and changes to them could then have an impact on hospital staff and infrastructure/financial planning, particularly if admission rates are likely to alter.
Financial and commissioning processes
Financial and commissioning (planning, monitoring and purchasing of health services) processes differ widely between health systems and between types of organisation (e.g. hospitals and primary care). The UK NHS provides services that are free at the point of use and paid for out of general taxation. Other health systems involve various combinations of insurance (private and not-for-profit) and out-of-pocket payments. Funding of health services is therefore a major factor to be considered in any assessment of the applicability of evidence across different contexts. Financial mechanisms may not transfer easily from one type of health system to another. For example, patient and health professional behaviour may be different in a taxpayer-funded system compared with one funded by insurance with or without co-payments.
Since the 1980s, the NHS has operated on the basis of a ‘purchaser–provider split’, with separate organisations being responsible for the commissioning of health care at different levels. Providers such as hospitals enter into contracts with the commissioners to deliver health services. The Health and Social Care Act 2012 replaced the previous system of commissioning by PCTs and strategic health authorities in England with a system of local Clinical Commissioning Groups (CCGs) supported by a national body (NHS England) with responsibility for specialist services. Responsibility for health is a devolved matter in the UK, which means that Scotland, Wales and Northern Ireland have some differences to the English system. A further complication when considering integration of health and social care is that social care in England is funded by local authorities and is means tested, unlike care provided by the NHS. The existence of separate budgets for health and social care represents a major barrier to efforts to improve integration between the two systems.
Sources of funding
We attempted to assess the source(s) of funding for the initiatives included in the review (UK studies and international comparative studies). This was not always clearly reported and, in particular, it was not always clear what the funding source was for the underlying initiative, as distinct from the funding for the research project evaluating that initiative.
Sources of funding may be classified as national, regional or local. In the UK, some initiatives were supported by the Department of Health and Social Care42,46 or a NHS National Coordinating Centre. 47 In Scotland, an initiative to develop and implement ICPs in Lanarkshire was supported by the Scottish Executive and subsequently developed by NHS Lanarkshire. 59 As an example of a regional initiative, an ICP working across primary and secondary care was supported by a Trent NHS research and development grant. 58
At the local level, several initiatives were supported by PCTs, either alone67,77 or in combination with a provider (a NHS trust). 76 One initiative was supported at the level of the general practice, which funded a practice nurse, whereas funding for a social worker was provided by the practice, social services department and local health authority. 63
A similar pattern of national, regional and local funding sources was seen in the non-UK comparative studies. In the USA, national organisations involved included the National Institutes of Health and foundations such as the Robert Wood Johnson Foundation and the Commonwealth Fund. Several initiatives were supported by multiple funding sources. 194,204
Commissioning and budget arrangements
As noted in Commissioning, commissioning arrangements in the NHS in England changed during the period covered by this review. A few studies91,103,223 mentioned the impact of commissioning and budget arrangements on the implementation of new initiatives. Julian et al. 58 commented that practice-based commissioning (a precursor to the subsequent system of commissioning by CCGs) and payment by results will change relationships between primary and secondary care. The success of one pilot initiative led the local PCT to incorporate it into its plans for the following year. 67 Other studies acknowledged support from PCTs47 or CCGs. 66
Commissioning and budget arrangements reported for non-UK initiatives were often based on fee-for-service205 or per capita models or a combination of the two. 215,220 Battersby et al. 195 reported that funding changed from a fee-for service to an outcomes-based model, whereas another study196,197 reported that per capita funding was planned but found to be too complex to deliver.
Availability and ring fencing of resources to support initiatives
The UK experienced an increase in spending on the NHS under the Labour Government, from 1997 through to the financial system crisis in 2008/9. This was reflected in some of the UK studies included in the review. 46,63,66 Two studies77,82 refer to the availability of Local Enhanced Service (LES) and Directed Enhanced Service (DES) payments. However, two studies35,51 reported that no additional funds were available to support their initiatives. Two studies66,82 highlighted that the extra resources were not sufficient to cover implementation of the initiative. With regard to ‘ring fencing’ of resources, one UK study63 stated that there was no formal ring fencing, but that it was expected that spending on health care for the older population would match their 10% share of the social services budget.
Three of the non-UK studies196,213,235 reported that resources were increased around the time of the introduction of a new initiative. One paper did not provide any further details,196 whereas the other two mentioned additional bonus payments213 or fees. 235 One other study209 noted that resources were not increased, but the authors suggested ways of sourcing extra funding. Incentives for participating in new initiatives are discussed in Incentives.
Incentives
In the UK, payments to general practices under schemes such as DES and LES encourage participation in the provision of new services. For example, Stokes et al. 82 noted the use of DES payments to support case management. By contrast, a clinical assessment service incorporated disincentives for GPs not using the initiative for > 90% of referrals. 67 Non-financial incentives may also be important, for example, GPs with a special interest in a particular area may be more likely to support new services in that area.
Non-UK studies reported incentives in the form of fees or bonus payments. In an Australian study of a change from a funding-based to an outcome-based model of service delivery,195 incentives were provided to GPs but not to hospitals, which continued to be funded on a throughput basis. In two US studies, bonuses were paid based on quality of care213 or meeting performance targets. 235
Commissioning and finance arrangements are key to the introduction of new integrated care initiatives, but having the appropriate financial arrangements in place does not guarantee success. For example, in a study reported earlier,213 practices were paid bonuses according to care quality, care co-ordination and call centre management. In practice, ≤ 10% of patient lists enrolled to the initiative, so bonus payments were relatively small.
Systems leadership
Dedicated project manager/managerial leadership roles
Some studies reported that individuals in managerial positions had assumed leadership roles in the initiatives,46,47,195 whereas others reported that specialists,70,72 a specialist nurse,66 a nurse consultant,221 a medical consultant55 or a GP with a special interest in the service67 had taken on leadership for a model.
Leadership in some studies was associated with involvement in the recognition for change and involvement in motivating others. In other cases, it was difficult to identify at which point different managerial roles came into place and, therefore, how much leadership was delegated from those who developed the intervention to those carrying it out.
Managerial or clinical leadership
When reported (see Dedicated project manager/managerial leadership roles), most managers and leads held a clinical role, and in some cases leadership was shared between a senior nurse and medical officer. 66,72 Graffy et al. 47 reported that a non-clinical manager led the project.
Project champions
Occasionally, a champion was mentioned in respect of driving the intervention forward. Kent and Chalmers59 reviewed a number of local ICPs as part of their study and found that initiatives were championed by a particular individual. The role of champion could overlap with managerial and lead roles; for example, in one study46 the GP lead was also described as the project champion. There could be more than one champion involved in multisite interventions; for example, in one study53 each nursing home had its own champion, and authors reporting an intervention that included a number of general practices213 anecdotally stated that champions were present within some of the practices.
Awareness of the initiative among patients
As intervention studies that include patient-specific data require informed consent, patients would have been made aware of the initiative through an invitation to participate and information provided as part of the consent process. However, it is not clear from the reported studies to what extent patients understood the reason for, or the process of, the initiatives that they were consented to, given the complexity of many interventions. As described shortly, some studies included patient surveys that may have raised patient awareness of the intervention.
Support for the initiative among patients
As well as consenting patients to interventions, a number of studies sought the views/satisfaction rating of patients (and sometimes carers) about the care they had received during the intervention evaluation. Reported satisfaction survey results were all ‘positive’ to ‘high’. 51,58,59,66,67,78,237,238 One study,195 however, reported that patient engagement in the intervention was affected by a range of barriers such as access, cost, understanding and motivation.
Types of services
We examined the included intervention studies to explore whether or not there were particular features of the services evaluated which might determine the applicability of the evidence to other health-care services.
The location of the initiative
Although there were large numbers of studies from both primary and acute care, the larger group was of initiatives implemented solely in primary care settings, with 21 of the UK intervention studies35,37,42,45,47,48,51,53–55,58,63,66,67,72,75,76,79,81,86,87 and 14 non-UK studies201,206,207,209,211,213–215,223,227,228,230,233,237 evaluating new models in this context. Fifteen UK intervention studies39–41,44,49,59,61,62,64,65,68,70,71,78,85 were carried out solely in hospital settings, together with nine non-UK studies. 204,212,216,222,228,229,238–240 One UK intervention study53 and one non-UK intervention study231 were carried out solely in nursing homes. The other included studies examined initiatives across a range of contexts, or the setting was unclear. Collaborations typically developed between primary care/community/social services (horizontal integration)34,42,52,71,194,196,197,204,209–211,214,221,223,224,230,237,241 or primary care/community/hospital specialists (vertical integration)43,198–200,217,218,236 to address complex care needs. Such integration required co-ordination, usually by a named individual (a care co-ordinator or case manager). A common theme was the usefulness of colocation of staff, which allowed face-to-face communication initiatives across locations. Integration between different services clearly poses a challenge to achieving in-person contact.
Some initiatives were implemented in contexts described as a regional centre68 or a specialist unit. 70,204 Authors did not make links between the context and the initiatives, apart from one study233 that highlighted that usual care in trial centres may not be representative of normal practice.
We found little reference in the literature to particular elements of new models of care, making it difficult to transfer findings between contexts (although in-home interventions obviously could only be delivered in that setting). Elements such as ICPs, case management and MDTs were introduced in a variety of contexts, including emergency care, other acute services, primary care and social care, with no reports of a specific context having a positive or adverse effect, apart from issues regarding training and retention of staff in social care53 and the benefits of physical colocation of services. 46
We explored whether or not there were any particular trends in the data in terms of outcomes for initiatives delivered in particular settings, and found variable findings for each context. For example, in relation to the new models implemented only in a hospital setting, three studies reported in five papers38–41,71 evaluated whole-service reconfigurations and found reduced numbers of admissions and reduced length of stay. Nine studies44,59,61,64,65,68,70,85,229 evaluated ICPs with varying outcomes, including reduction or no difference in length of stay, and no difference in admissions or costs. MDTs were reported to have varying impacts on costs and reduced time to readmission. 212,239,240 Care co-ordination initiatives were reported to reduce length of stay but also to have varying impacts on costs, admissions and ED visits. 216,222,228,238
As a second example, nine non-UK intervention studies (11 papers)194,201–203,205,210,214,234,237,242,306 examined initiatives that were delivered to patients in their own home. There were no clear signals regarding effectiveness. With regard to A&E usage, the evidence was inconsistent: two studies found a reduction in A&E use210,237 and one found no difference in A&E use. 194 Two studies reported a reduction in overall hospital use 242,306 and two reported a reduction in overall costs. 234,242 One study reported no effect on admissions. 210 Two studies reported increased patient satisfaction. 214,237 One included UK paper also reports a home-centred initiative (for patients at the end of life). 34 The study found no significant effect on costs or health-care usage.
We looked for any patterns regarding delivery of particular interventions in particular settings. ICPs were developed with the aim of optimising sequential clinical decision-making for a particular condition. Ideally, an ICP is developed and used by a range of clinicians representing the various stages of the pathway. However, included studies show a tendency for ICPs to be tested and reported by representatives from one location (usually either acute services or primary care) with reference to other pathway services. For example, an ICP may be developed in primary care in order to optimise referral decisions,55,58 whereas, in hospital settings, an ICP would assist in treatment decision-making. 59,61,62,64,229 ICPs could span health-care service boundaries35 or focus on one setting; for example, in EDs,44,70 preoperative and postoperative care65 or end-of-life care in nursing homes. 53 All studies evaluating ICPs (except for one229) were published in the UK.
Multidisciplinary teams were reported to draw on a pool of clinicians who could provide the range of expertise required to address complex conditions. The teams practised across a range of locations, for example, at home,36,47,234,237 in nursing homes,46 in community clinics,66 in social services centres,75 in primary care,82,201–203 in hospitals212,240 or in a range of settings. 239 There were no data specifically linking outcomes to location.
Alignment with other initiatives
We searched the included documents for information regarding whether or not other initiatives had already been introduced in the setting being researched, and, if so, whether or not there was alignment between initiatives. We identified few data regarding this in the studies. Offredy et al. 67 describe a pilot having been introduced in the 3 months prior to the main service reconfiguration. Another study46 describes the introduction of the in-home service model being during a period of significant other developments. Two international papers195,235 mentioned recent changes to funding prior to the initiative.
Standard of existing care
The context in terms of standards of the existing service was reported by a limited number of studies, with some reporting that the new model of care was introduced in services that had particularly high existing standards,52,227 or were of variable quality and in need of improvement. 55,71,110,128 Initiatives were reported to have been inspired by lack of choice,34 prolonged waiting times,55,67 inconsistent referrals195 and suboptimal care221 for patients. On the other hand, a UK nursing-home-based initiative showed that study results were positively associated with previous rankings of good care standards. 53
Authors of a systematic review153 concluded that care pathways may be less effective when good practice is already being followed and there are existing MDTs, and it may be better to introduce them where there are deficiencies in services. This view was echoed by another review,189 which suggested that case management might be most effective in settings that need changing, such as poorer-quality community services. The success of an ICP in palliative care was described as depending on aspects of the care structure that were already in place in one UK qualitative study. 125
Features of the workforce
We examined characteristics of the staff who were involved in implementing the new models of integrated care, in order to explore whether or not there were elements that may influence applicability in other service locations with differing workforces.
Levels of motivation and support
The level of motivation and support for the initiative was not explicitly reported within included papers; rather it was implied, for example, by statements that staff had developed the initiative themselves70,229 or had been previously concerned about care quality,221 or that the initiative was being championed46,53 or had been adopted early by some GPs, a group of whom were committed to the intervention. 237 In some cases, levels of support for the initiative differed between roles. For example, in an initiative that employed mental health nurses in an ED,78 it was reported that the ED staff were supportive but the mental health nurses were less supportive. In three studies,55,195,235 it was reported that GPs or specialists were difficult to engage or needed reminders to comply with the protocol. The qualitative literature provided further evidence of lack of engagement, in particular among GPs, and levels of engagement formed one of the factors identified in the logic model, with stronger evidence of its influence.
Employment conditions
Changes in employment conditions for the staff involved in the research were implied by initiatives that were funded differently, creating altered incentives. 235 There were reports of a move from one department to another,40 changed or extended roles and responsibilities46,195,216,227 or changed staff rotas. 41 Such details were generally reported only briefly, although it is clear that implementing new models of care will usually entail a degree of change in this area. One UK study63 detailed that staff had specifically been employed for the project.
Working location
The colocation of staff was a reported element of integrated care, with the aim of bringing together staff from a range of services in order to case manage or co-ordinate care in one place. 40,41,46,196,197 Close communication between non-colocated providers was also mentioned. 236 For each study/intervention, interpretation of the concept of colocation might differ among the settings and aims of the intervention. For example, Boyle et al. 40 reported the closure of a medical admissions unit, colocating the staff with those in the ED. The aim of colocation in this intervention was shared responsibility for workload in the ED, whereas colocating social workers, district nurses, GPs and administrators in primary-care-based case management interventions was used to ensure effective communication. 46,196,197
Specialist staff
There was frequent mention of specialised staff having an important role in initiatives, with the requirement for these staff to be available and to receive additional training when required. Reconfiguration involving additional specialism is also reliant of staff being willing to take on new roles. A UK study examining role change137 highlighted that the intervention relied on the willingness of GPs and practices to take an interest in cardiology. In cases in which initiatives were condition- or population-specific, specialists in these conditions (e.g. respiratory, dermatology, diabetes mellitus, end-of-life care, geriatric medicine and mental health) were reported to be members of MDTs. 34,36,46,53,77,78,199,236,237,241 Advanced practitioners and clinicians with a special interest (nurses and GPs) were also involved. 46,54,58,63,66,67,221 In addition, descriptions of new models of care include new roles that facilitate integration and co-ordination, such as case managers,216,218,236 care co-ordinator,205 family support specialists,214 discharge community link nurse34 and home care team. 217,234 New models that were based around particular conditions were therefore more likely to include specialists in those conditions, and integrating care was likely to require new roles to co-ordinate care from different sectors. For example, a study evaluating a specialist home care unit in Sweden234 highlighted that a range of specialist staff was required in order to operate the new model of care.
Professions involved
The types of profession involved in the included studies varied widely across models, although there was a tendency to bring together professionals from the community, social care, general practice and secondary care in a range of configurations to best improve integration of assessment, treatment, discharge (when applicable) and home/nursing home care. These professionals included consultants, physicians and GPs, district nurses, nurses, nurse assistants, pharmacists, physiotherapists, occupational therapists, dieticians, social workers and practice managers. Some articles did not provide specific details of the professions involved, but used more-generic terms such as ‘team of experts’. 240 A comparator study from the USA212 emphasised that costs of providing new models of integrated care could differ according to the personnel profile, potentially influencing the cost-effectiveness of interventions.
Size of staff group
The size of teams associated with new models was often difficult to ascertain because of limitations in reporting. However, when information was available, teams ranged in size depending on the aims of the model and the setting. Models that aimed to integrate a higher number of professional groups or settings included larger groups. Some models involved linking with other services rather than working with them. There was no indication in the literature regarding optimal numbers of staff required to deliver new models of care; however, one UK evaluation of an ICP reported that a large MDT was required to deliver the service. 65
Staff training
The need for staff training to enable them to deliver new models of integrated care was emphasised by nine UK studies34,35,44,46,51,53,54,64,72 and 10 non-UK studies. 194–197,202,203,214,216,221,227 The training and induction outlined was mainly for and/or by nurses and related to new roles or model protocols. Not all included papers reported details of training, although this does not mean that it was not a part of the intervention. One study53 described the challenge of providing training to nursing home staff because of high levels of staff turnover. Systematic reviews161,181,183 echoed the importance of providing joint training in how to work effectively together. One162 emphasised that well-integrated teams tended to have ongoing support or training, rather than training at periodic intervals.
Although reported workforce factors may have played a part in effectiveness of interventions, it is difficult to pinpoint a particular approach that leads to improved outcomes. This is because each effect is embedded within a broader framework of change, elements of which might also have an effect.
Features of the initiatives
We scrutinised the initiatives that were evaluated in the included literature to explore whether or not there were particular features that might influence applicability in terms of implementation and outcomes.
Complexity of initiatives
As outlined in Elements of the interventions, the included literature encompassed both single-element initiatives, such as the introduction of an integrated pathway, and multielement whole-service transformations. We examined the data to explore whether or not there may be any difference in applicability of the findings for ‘simple’ versus ‘complex’ interventions. Table 8 summarises the strength of evidence for ‘simple’ (single-component) and ‘complex’ (multicomponent) initiatives based on UK studies and non-UK comparative intervention studies.
Outcome | Overall strength of evidence | UK studies results | International comparator studies results | Strength of evidence by intervention type | |||
---|---|---|---|---|---|---|---|
Simple | Complex | Simple | Complex | Simple | Complex | ||
Number of admissions | Inconsistent | ± ± – – | – – + | ± | ± ± ± ± ± + – – – | Inconsistent | Inconsistent |
Number of emergency admissions | Inconsistent | – ± ± | + + ± ± – – – – – | – | Inconsistent | Inconsistent | |
Length of stay | Inconsistent | ± ± – – – | – – – – – – ± – | ± ± – – | Inconsistent | Inconsistent | |
Patient contacts | Inconsistent | + | + – – – – ± ± | ± ± ± – – | Very limited | Inconsistent | |
Patient satisfaction | Stronger evidence of increase | + + | + + + + + + ± | + | + + ± ± ± ± | Weaker evidence of increase | Inconsistent |
Access to services | Weaker evidence of increase | + | + + | + + – ± | Very limited | Inconsistent | |
Costs | Inconsistent | – | – – – – – + ± | ± | ± ± ± ± ± ± + + + – – – – | Very limited | Inconsistent |
The evidence for both types of initiative was generally inconsistent and the strength of evidence was similar to that for the whole group of included studies. Differences were mainly caused by lower strength-of-evidence ratings for the ‘simple’ initiatives, reflecting the small number of studies in this group.
Full integration compared with partial integration
Full integration refers to the creation of a new organisational model, whereas partial integration is the co-ordination or integration of different sectors or organisations via less formal non-binding links. The majority of UK and international studies evaluated some form of partial integration. There are few examples of initiatives involving full integration in the UK apart from the extensively evaluated project in north-west London. 50,73,79,80 Other settings included Greater Manchester72 and the Isle of Wight. 75 The limited number of studies of full integration and the diverse outcomes reported in them means that it is not feasible to explore any potential difference in applicability of the evidence between full and partial integration initiatives.
Breadth of reach
Another fundamental characteristic of initiatives is whether or not they are aimed at a broad population or aim to integrate services for a specific, possibly small, patient group. Although some of the initiatives included in this review clearly fell into one group or the other, many were difficult to categorise in this way. In general, ICPs tended to be aimed at managing specific patient groups whose numbers may be either large or small. For example, a pathway developed at a specialist centre for children with pleural empyema was used for just 13 and 18 patients, respectively, in 2 years of evaluation. 68 By contrast, Johnstone et al. 56,57 developed and evaluated a pathway to improve quality of end-of-life care across the whole of Wales.
Longevity of the initiative
Evaluations of initiatives may produce different findings depending on the stage at which the evaluation takes place and the length of the evaluation. New initiatives are likely to need time to overcome any initial operating problems, whereas an evaluation that is too short may be unable to detect any beneficial effects that exist. 195 Reports were often unclear in reporting how long the initiative had been operating at the start of the study, and if and how programmes were evolving from earlier changes.
Other specific requirements of initiatives
We looked for comments in study reports that identified other features or conditions required for initiatives to operate effectively. When these were mentioned, they were often very general; for example, ‘ability to cross health sector boundaries and use behaviour change skills’195 and ‘ability to target high risk patients’;205 a receptive/supportive context;49,53 presence of local ‘champions’;59,70 and the need for ‘pump-priming’ and, if necessary, the decommissioning of existing services to allow the development of new ones. 87
Chapter 4 Discussion
Summary of evidence
Our systematic review of new models of integrated care identified 267 documents that met our inclusion criteria, and these were examined and synthesised. The literature included studies from the UK, other European countries and North America, and encompassed empirical work with a range of designs including quantitative evaluations of interventions/initiatives, and qualitative data related to staff or patient perceptions of change and barriers/enablers influencing outcomes.
We identified multiple elements that constituted new models of integrated care, which we grouped into four categories: (1) those relating directly to patient care, (2) those with a focus on intervening at an organisational or system level, (3) those with a focus on changing the way that staff are employed or work together and (4) those that have a focus on reconfiguring financial, commissioning or governance aspects of health service provision. Within this typology, a wide range of different types of interventions were included: joint assessment, ICPs, agreed referral criteria, care/case co-ordination, joint review/discharge, integrated IT systems and patient record sharing, new services, new care groupings or units, transferring services from hospitals to the community, MDTs and team meetings, relocating staff, changing employment conditions, joint commissioning, financial integration, and organisational integration. Identifying exactly which changes to delivery had been made as part of a new model of integrated care could be challenging, as many studies examined multicomponent, highly complex initiatives that were only described briefly by authors. Few studies identified cause–effect or associational linkages between the individual elements of new models of care and the service outcomes and system impacts.
Diverse anticipated targets for change were described: changing the model of care to one that was more holistic and/or patient-centred; changing the organisational culture; developing shared values, beliefs and priorities among staff and organisations; changing relationships between staff; changing relationships between organisations; developing trust; developing mutual understanding; and, finally, improving communication between staff. Although these areas were commonly described as targets/aims, few quantitative evaluations measured any change in these target areas. Qualitative data similarly provided evidence of the perceived importance of these changes for improved working practice and service delivery, but limited and mixed evidence regarding whether or not interventions had led to improvements in these areas.
The elements of new models of integrated care offered the potential to effect change at multiple levels: at a process level (changing the way services were delivered); at a service outcomes level (changing resource usage, quality of care, or staff satisfaction); or at a system impact level (changing use of hospital-based services, use of primary care and community services, overall use of health-care services and cost of delivering health care). We therefore adopted a systems approach, using a logic model to analyse the data related to these elements of the pathway, from the interventions reported to process changes, service outcomes and system-wide impacts.
We grouped the process changes described in the literature into the categories of service user change, change in care provision, change in information and organisational change. We examined data from the UK and international intervention studies for evidence of effect on processes, and examined the qualitative literature for data related to perceptions of change. We also examined the literature for reports of relationships or associations between particular intervention elements and specific outcomes.
There was surprisingly little UK or international evidence regarding the effect of integrated care initiatives on service user involvement or relationships. The very limited evidence available provided inconsistent findings regarding whether or not service users perceived any change or had greater knowledge of, or involvement in, services.
With regard to our second grouping, change in care provision, there was weaker evidence that discharge planning may be improved (possibly associated with elements of case conferences and/or joint/streamlined assessment), and there was also weaker evidence that there may be an improvement in the timeliness/flow of care provided (possibly associated with elements of ICPs and/or joint/streamlined assessment). There was also weaker evidence regarding interventions leading to reduced variance in practice (possibly associated with ICPs), and a negative impact in terms of adding to practitioner time burden. There was inconsistent evidence regarding the effect of initiatives on continuity of care.
There was inconsistent evidence regarding any effect on information sharing between staff/services and with regard to the accuracy/completeness of record keeping. There was some weaker indication of increased knowledge sharing between staff. The evidence regarding change in organisational processes as a result of new models of integrated care was also inconsistent, with reporting of considerable challenges achieving system change beyond improvement in relationships.
We identified numerous diverse factors that were reported to influence the implementation and outcomes of new models of integrated care. The factors were grouped into (1) those that were patient related, (2) those that were workforce related, (3) those that were management and leadership related and (4) those that were organisation and system related. The factors that appeared to have stronger potential influence were the perceived focus of the initiative (patient focused versus organisational or financially motivated), the level of patient engagement, staff professional identity, staff professional role boundaries, power and hierarchies within staff groups, stability of the workforce, the provision of training to staff, levels of engagement among staff (particularly GPs), having local leaders/champions and effective leadership; resources available, the compatibility of IT systems, the clarity of the vision for change, the prevailing organisational culture, the policy context in which the initiative was sited, and commissioning and governance policies and procedures.
We identified diverse outcomes for new models of integrated care reported in the included literature. We categorised these outcomes into those related to use of resources, those related to quality of care, and outcomes for staff working experience. We identified three outcomes in which there appeared to be stronger evidence of effect: first, that new models of integrated care may lead to increased patient satisfaction; second, that new models are perceived by staff to increase the quality of care; and, third, that new models of care may increase patient access to services. There was stronger evidence of effect from the UK only that patient waiting times and outpatient appointments may be reduced (although the evidence as a whole was inconsistent), and that patient wishes at the end of life are met. The evidence was inconsistent with regard to the effect on the number of clinician contacts, the number of GP appointments, length of stay in hospital, unscheduled admissions, scheduled admissions, readmissions, attendance at an A&E department and staff working experience. There was very limited evidence available regarding prescribing, access to resources across services, time spent in A&E departments, numbers of incidents or complaints or the identification of unmet need.
The evidence was inconsistent regarding system-wide impacts on hospital services, primary care and community services, overall health-care usage and cost of health-care provision. As we had identified the potential for limited effect on outcomes of single-element ICPs or MDT interventions, we removed these from the strength-of-evidence tables, but found that this did not change the grading assigned to any of the outcomes.
What can be learned from the international literature regarding the key elements of implementation and the potential impacts of new models of care?
The review confirms the high level of complexity that is inherent in implementing and evaluating new models of integrated care. A systems perspective is needed to best understand the elements of implementation and how these individual elements may lead to outcomes and impacts. The review identified numerous factors that may influence implementation and outcomes. In Table 9, we set out how and where these need to be considered during the design and implementation of new models of integrated care. Two influencing factors we identified (external policy context and capacity/resources available) present particular potential threats to implementation and outcomes outside the control of the initiatives.
Stage | Areas of influence identified in the literature | Specific barriers/enablers |
---|---|---|
Planning | Engagement with patients |
|
Staff engagement |
|
|
Shared understanding/vision between organisations |
|
|
Concerns regarding patient information confidentiality |
|
|
Adequacy of systems for sharing information |
|
|
Equitability of incentives across organisations/services |
|
|
Patient groups to target |
|
|
Elements of interventions to introduce |
|
|
Implementation | Staff response to change |
|
Professional identity |
|
|
Professional role boundaries |
|
|
Stability of workforce |
|
|
Effective leadership |
|
|
Sustainability | Promotion of initiatives and achievements |
|
Ongoing audit and feedback |
|
|
Maintaining working practice change |
|
|
Maintaining staff engagement |
|
|
Maintaining patient engagement |
|
|
Evaluation | Evaluation of change in interpersonal/cultural elements |
|
Evaluation of process changes alongside service outcomes |
|
|
Evaluation of patient-related outcomes |
|
As previously outlined, the international literature indicates that evidence is inconsistent with regard to impacts from new models of integrated care. There is no clear evidence regarding whether new models of integrated care are cost neutral, increase costs or reduce costs. The impact on community and hospital-based services was also unclear, although there was no evidence to suggest that new models of care increase use of secondary care.
What are the reported mechanisms of change and the outcomes and impacts associated with new models of care?
We used the data from included studies to develop an evidence-based logic model, which sets out the elements of the pathway from interventions to impacts. The logic model highlights the key role of interpersonal/relational factors in the change pathway, with these aspects being commonly described, but little evaluated, in the literature. The areas of care model, organisational culture, beliefs/values/priorities, relationships between staff and organisations, mutual understanding and communication are important mechanisms of change that may be central to new models of integrated care, and they represent an overall cultural change in health-care delivery.
The review has identified numerous different elements that are subsumed within the label of new models of integrated care, and often form multiple and interlinked initiatives within complex interventions. Included studies highlighted the challenges in identifying cause–effect relationships between new models of integrated care and service delivery impacts. 154,165,198–200 We scrutinised the UK and international interventions literature for evidence of reported associations or cause–effect relationships for specific elements of the model, with the aim of indicating relationships on the pathway model. Although effects of initiatives on service outcomes were widely reported, few studies described which elements of the initiatives may be ‘key ingredients’ that effected change, or they attempted to attribute associational outcomes to different types of integrated care. The intervention elements with more clearly reported associations tended to be ICPs and MDTs (predominantly reported in the UK literature), and case management interventions (predominantly reported in the international literature). Overall, there were very limited data regarding which elements of new models of care lead to which outcomes, and we consider the reported associations outlined in Table 10 to be hypothetical. The studies reporting each association are referenced. Many associations were only made by authors of a single included study.
Element of integrated care | Associated with | |||
---|---|---|---|---|
Improved staff relationships | → | Increased patient satisfaction8 | ||
Increased staff trust | → | New integrated identity90 | ||
Increased staff shared understanding | → | Increased speed of delivery121 | ||
Integration | → | Improved communication181,183,184,191,193 | ||
Improved communication | → | Improved team performance | → | Enhanced care161,171,179 |
Higher levels of GP engagement in initiative | → | Greater effect on outcomes (cost)77 | ||
ICP | → | Reduced variance in practice56,59,109,184 | ||
ICP | → | Increased patient involvement279 | ||
ICP | → | Increased patient understanding of treatment59 | ||
ICP | → | Increased communication35,65 | ||
ICP | → | Increased speed of delivery/flow55,85 | ||
ICP | → | Improved decision-making70 | ||
ICP | → | Improved documentation56,68 | ||
ICP | → | Increased continuity of care64 | ||
ICP | → | Improved patient understanding of care59 | ||
ICP | → | Reduced waiting time for assessment55,85 | ||
MDT | → | Increased communication73 | ||
MDT | → | Restructured care | → | Reduced costs212 |
Multidisciplinary meetings | → | Improved relationships/communication60,61 | ||
Case conferences | → | Improved discharge planning191 | ||
Integrated community team | → | Improved patient knowledge and engagement273 | ||
Joint/streamlined assessment | → | Increased accuracy of assessment | → | Reduced admission42 |
Joint/streamlined assessment | → | Increased efficiency | → | Increased quality of care;38 earlier discharge63 |
Case management | → | Continuity of patient contact/care | → | Reduced hospital use199,204,205,238 |
Case manager | → | Familiarity with local services230 | ||
Case manager | → | Shift from unplanned to planned hospital admissions236 |
The associations described in the literature seem to suggest that the elements of new models of integrated care may act in different ways to create change; for example, ICPs may be particularly effective in reducing variance in practice, improving documentation and improving decision-making, and thus reducing waiting times by improving the flow and/or speed of service delivery. ICPs may improve patient understanding of treatment, MDT meetings may be the most effective element for improving relationships and communication, and case management initiatives may particularly act on the outcome of increased continuity of care. However, there is currently little empirical evidence regarding any potential associations between elements, which represents a gap in the literature that is hindering the identification of key ingredients of change in new models of integrated care.
How applicable are the findings from the international literature to different local populations and contexts, and how might mechanisms of change operate differently in different local services?
In our consideration of data regarding applicability, we highlighted the potential for interventions to work in varying ways in different local populations. The data suggest differential effectiveness for new models within different population subgroups, with several studies suggesting the need to target new models on particular groups. Unfortunately, the evidence is unclear regarding which conditions should be targeted, with diabetes mellitus, chronic conditions or ‘at-risk’ patient groups being suggested, although evidence regarding each of these is limited. One particular group that may offer potential for positive outcomes is those described as having ‘complex needs’. The evidence regarding interventions in populations of older adults generally, however, is more variable, with no strong signal regarding the effectiveness of targeting this group. There was limited evidence suggesting that targeting those with the most-severe conditions may be beneficial, and a limited body of work in specific clinical conditions.
The evidence regarding targeting deprived communities was unclear, with reports ranging from describing positive outcomes to reporting a lack of significant effect or increased costs. Studies from non-UK countries were often carried out in diverse communities, and there was little evidence related to potential influences of socioeconomic diversity on outcomes. In the UK, there was a large body of evidence from large urban and densely populated areas, and there was reference to the potential challenges of implementing integration in rural areas. Several of the non-UK studies compared different geographical areas where the same intervention was implemented, with little identification of differences between these sites.
We noted in Chapter 3, Elements of the interventions, that the elements of interventions varied between the UK and non-UK literature, with a greater proportion of UK interventions focusing on ICPs, and a larger proportion of international studies reporting case management-type interventions. Intervention elements were largely similar across the body of work, apart from financial transformation in systems that operate very differently to the UK. The international literature included more references to whole-population-level interventions, rather than services for particular clinical conditions. Initiatives such as MDTs, ICPs and case management were introduced in a variety of contexts including emergency care, other acute services, primary care and social care, with no reports of a specific context having a positive or adverse effect, thus suggesting that these elements are applicable to varying contexts. In-home interventions and social care settings were described as having some particular challenges, for example, retention and training of staff, availability of specialist staff and local practices having different structures. The assessment of strength of evidence for the UK literature compared with the international literature was different for only one outcome. The evidence of a reduction in length of stay was rated as being stronger for UK studies, whereas it was rated as being weaker across all studies.
The extensive range of influencing factors identified in the review are likely to have considerable importance in the way that mechanisms of change operate in different local contexts. There was evidence of influencing factors related to patients, the workforce, the organisation and systems, together with finance and governance. It is these factors that are likely to influence local differences in implementation and outcomes. Simpler single-element interventions (such as ICPs) may face fewer obstacles to implementation than larger, complex interventions, which are likely to encounter a greater number of influencing factors. However, the evidence indicated that simpler interventions may only have an effect on care process outcomes (rather than service delivery outcomes), and combining them as part of complex multielement initiatives may be required for service and system change.
Limitations
We highlight the challenges inherent in definitions of new models of care, with a lack of agreed definition and clear boundaries to the term. A limitation of this review may be excluding work that could have been of relevance due to lack of clarity of definition. We endeavoured to distinguish between new models of care that are integrated/co-ordinated from those that are not, and this distinction was challenging during the screening and inclusion process. The term ‘integration’ could be used in a variety of ways, including to describe interventions that related to enhanced care or quality assurance but did not include staff working in new ways. We also drew a distinction between outcomes related to service delivery outcomes and those related to clinical (health) outcomes. We included studies that reported both clinical and service delivery outcomes, but prioritised extracting the service delivery data. We believe that our search terms enabled relevant citations to be retrieved; however, we recognise that indexing can be imperfect and that we may not have identified work of relevance. As a priority topic, work on models of integrated care is constantly being published; however, our searches were required to have a cut-off point in order to complete an extensive extraction and analysis process within the time allocated.
We recognise that a particular limitation relates to the lack of statistical summary (meta-analysis), although we would argue not only that the heterogeneity of interventions and outcomes preclude this type of analysis, but that, in order to understand the complexity of the area, a systems approach is required. The process of summarising strength of evidence used in this review is open to debate and potential criticism. As highlighted in Chapter 2, Risk of bias in individual studies, reporting strength by number of studies may be challenged, as this indicates primarily the areas in which there has been research activity. In exploring consistency as well as quantity in our strength-of-evidence assessment, we have sought to overcome this limitation to some extent. Linked to this is a potential issue of reporting bias, with studies reporting less-positive outcomes that are potentially under-represented in the review. We highlight the fact that many of the initiatives outlined may only show the true effects after several years of implementation, and there are considerable challenges for studies in endeavouring to measure multidimensional outcomes and impacts. Our attempt to group and evaluate particular outcomes across studies with highly complex interventions and potentially long-term impacts may be viewed as overly simplistic, and we emphasise the need for a systems view of influences and impacts.
Our categorisations and typologies are open to differing interpretations. We are aware of the extensive literature in the field of organisational research, which offers frameworks and theories for classifying different elements of organisational systems. In developing the logic model, we have been influenced by this existing literature, but have endeavoured to be data-driven, working from a bottom-up process rather than imposing a pre-existing framework. We recognise that we have not explored theoretical mechanisms underpinning the change pathway, which may have offered additional insights. The review has focused instead on bringing together what is known about outcomes that may be associated with new models of integrated care, and evidence regarding the process of change.
Unlike many previous reviews, we did not focus on a particular patient group or a particular type of intervention. The approach could therefore be criticised for prioritising breadth over depth. Although we recognise that our conclusions regarding outcomes and impacts are based on highly heterogeneous studies, we would argue that our broad inclusion enabled us to have a sufficient number of studies to examine a wide range of individual outcomes. Having this greater number of included studies has also enabled the study to ‘slice’ the data in varying ways, such as by differing population features, intervention elements and organisational factors, to explore patterns and trends in the data. It has also provided the opportunity to make comparisons and contrasts between new models of integrated care across these different groupings.
Chapter 5 Conclusions
The term ‘new models of care’ has been used to refer to a wide range of interventions aiming to address issues of integration across health care and between health and social care. The varying composition of these interventions presents considerable challenges in comparing implementation and outcomes. Our systematic review has described the diverse elements that may be included in new models, and has explored the evidence underlying the outcomes and impacts that may result from the models. We have identified positive changes to the process of care delivery that may result from new models of integrated care: improved discharge planning, improved flow of care, reduced variance in practice and improved sharing of knowledge between practitioners; however, the evidence of change is considered to be weaker rather than stronger. The review also found weaker evidence of a potentially negative impact on practitioner time. The review highlights that the implementation of, and outcomes from, new models of care will be influenced by multiple factors, which may contribute to local variation. We have identified the need to consider and, when possible, address these factors at all stages of planning, implementation, sustainability and evaluation, and we have highlighted the influence of external factors such as resources and policy context.
It is interesting that our review included 11 years of research, with the factors reported as enabling or blocking new models changing little between the earlier and later reported work. Influencing elements reported in the international and UK literature shared many similarities, although it was notable that the UK literature included far greater discussion of influencing factors related to the workforce.
We identified three outcomes in which there appeared to be stronger evidence of effect: first, that new models of integrated care may lead to increased patient satisfaction; second, that new models may increase perceived quality of care (the UK evidence relates to staff perceptions of improved quality only); and, third, that new models of care may increase patient access to services. There was weaker evidence of effect that patient waiting time may be reduced, and weaker UK evidence indicating that patient wishes at the end of life are met more often. These outcomes appear to be interlinked, with improved perceived quality, access and waiting times potentially explaining the reported increased level of patient satisfaction. The evidence was inconsistent or limited regarding other outcomes at a service level, with a lack of clear signals regarding system-wide impacts on hospital services, primary care and community services, overall health-care usage and cost of health-care provision.
Evaluating outcomes and impacts from new models of integrated care presents challenges in determining what a ‘good’ outcome may be. We highlighted that outcomes may be perceived differently by different stakeholders, and that there may be contradictions in incentives for achieving change. A key contradiction at an organisational level relates to drivers of competition versus drivers of integration. Several studies highlighted that reducing activity in one sector of health care means a reduction in income, and that increased activity may not be followed by transfer of monies between organisations. Another tension relates to cost saving versus providing improved quality of care. Some studies reported that new models of care may improve quality but result in increased spending. A key influencing factor was reported to be the perceived aim of the initiative among staff and patients (whether to enhance care, change for organisational purposes or to reduce costs), and the need to be clear about the vision for the change was also highlighted. This clearly presents a considerable dilemma as a key aim may be cost saving, but the evidence indicates that this may make it less successful because of a lack of engagement among staff. On the other hand, increasing quality of care for patients may come at increased cost for services that are already under financial pressure. There is currently no strong signal that new models of integration may be cost saving, and increasing ease of access may have a detrimental effect on costs and capacity.
Currently, it is problematic to draw linkages between different elements of new models of integrated care and specific outcomes, which may contribute to a perceived lack of clarity regarding the vision for change. There was some suggestion that ‘simpler’ initiatives, which tended to be more frequently reported in the UK literature, may affect care processes but not service delivery outcomes. Including multiple elements within new models makes identification of the key/active ingredients problematic. However, it seems important to pursue further work in this area to examine which elements of change are or are not required, as reports of increased practitioner time mean that new models are potentially not without adverse effects. Currently, the UK literature in particular has an emphasis on integrating specific care services for patients, whereas reports from the international literature more often include an emphasis on patient education or whole-population health improvement.
A further contradiction noted in the literature concerns notions of patient-centred care versus standardisation of care. Initiatives such as ICPs were reported to reduce variation in practice, although there was also reported caution that they could lead to rigidity. There was surprisingly little UK or international evidence regarding the effect of integrated care initiatives on service users. The very limited evidence available provided inconsistent findings regarding whether service users perceived any change or had greater knowledge of or involvement in services.
Implications for health care
-
There is stronger evidence that new models of integrated care may increase patient satisfaction and may lead to improvements in the accessibility of services and the length of patient waiting times.
-
The evidence is currently unclear regarding other outcomes for services, or for the health-care system as a whole, including the impact on service usage and costs.
-
Numerous factors are reported to influence the implementation and outcomes from new models of integrated care, which need to be considered at all stages of the introduction of new models.
-
New models of integrated care act at a whole-system level, with a need to consider incentives and outcomes across the whole system.
-
There is currently little evidence regarding the impact of new models of care on patients beyond the evaluation of patient satisfaction and some suggestion that outcomes for carers may differ from those for patients.
-
New models of integrated care may offer more potential in certain patient populations, such as those described as having ‘complex needs’, although there is limited evidence related to differential effects in specific clinical conditions.
-
There is some evidence that interventions, such as ICPs, MDTs and case management, implemented in isolation, may achieve change in processes, but that they should form part of multicomponent initiatives in order to improve service delivery outcomes.
Recommendations for future research
-
We have identified a need for research to more clearly link particular elements of new models of care to outcomes.
-
We have documented a need for further research to examine outcomes at a process level to add to the knowledge regarding intervention–outcome links. In particular, further research could include greater examination regarding process change outcomes in terms of patients (such as understanding of treatment), practice (flow of care and duplication), exchange of information (extent and accuracy), time spent engaged in tasks and organisational changes (commissioning, governance and budgetary).
-
Currently, much of the research evidence comes from studies of older adults. There is a need for further research to explore the potential for new models of care to have an impact on the care of other patient groups.
-
The UK literature in particular demonstrates a focus on outcomes and influences related to the workforce, whereas the potential effects on patient and carer experiences (beyond levels of satisfaction) is under-researched.
-
We have identified a need to further explore particular challenges of new models of care in rural areas, and potential differential effects in varying subgroups such as those from socioeconomically deprived communities.
-
We have developed and used an applicability framework to guide analysis of the findings of the review. This framework has potential to be a useful tool for other studies to guide analysis of results.
Acknowledgements
Contributions of authors
Susan Baxter (Senior Research Fellow, University of Sheffield) led the study and took the lead in analysis and report writing.
Maxine Johnson (Research Fellow, University of Sheffield) contributed to the review processes.
Duncan Chambers (Research Fellow, University of Sheffield) contributed to the review processes.
Anthea Sutton (Information Specialist, University of Sheffield) developed the search strategy and carried out electronic database searching.
Elizabeth Goyder (Professor of Public Health, University of Sheffield) provided health service research expertise and advice during the study.
Andrew Booth (Reader, University of Sheffield) provided methodological input to the review processes.
Data-sharing statement
All data requests should be submitted to the corresponding author for consideration. Access to anonymised data may be granted following review.
Disclaimers
This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care.
References
- Five Year Forward View. London: NHS England; 2014.
- Shortell SM, Addicott R, Walsh N, Ham C. The NHS Five Year Forward View: lessons from the United States in developing new care models. BMJ 2015;350. https://doi.org/10.1136/bmj.h2005.
- Long T, Khan AM, Chana N. Achieving better value: primary care must lead on population health. Postgrad Med J 2015;91:59-60. https://doi.org/10.1136/postgradmedj-2015-133264.
- Ham C, Muray R. Implementing the NHS Five Year Forward View: Aligning Policies with the Plan. London: The King’s Fund; 2015.
- Ahmed F, Mays N, Ahmed N, Bisognano M, Gottlieb G. Can the Accountable Care Organization model facilitate integrated care in England?. J Health Serv Res Policy 2015;20:261-4. https://doi.org/10.1177/1355819615590845.
- Robertson H. Integration of Health and Social Care: A Review of Literature and Models – Implications for Scotland. Edinburgh: Royal College of Nursing; 2011.
- The Forward View into Action: New Care Models: Update and Initial Support. London: NHS England; 2015.
- Powell Davies G, Williams AM, Larsen K, Perkins D, Roland M, Harris MF. Coordinating primary health care: an analysis of the outcomes of a systematic review. Med J Aust 2008;188:65-8.
- Shaw S, Rosen R, Rumbold B. What is Integrated Care? An Overview of Integrated Care in the NHS. London: Nuffield Trust; 2011.
- Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 2011. www.cochrane-handbook.org (accessed 20 June 2017).
- Quality Assessment of Systematic Reviews and Meta-Analyses. Bethesda, MD: National Institute of Health; 2014.
- Critical Appraisal Skills Programme . CASP Qualitative Studies Checklist 2017. www.casp-uk.net/checklists (accessed 20 June 2017).
- Glenton C, Colvin CJ, Carlsen B, Swartz A, Lewin S, Noyes J, et al. Barriers and facilitators to the implementation of lay health worker programmes to improve access to maternal and child health: qualitative evidence synthesis. Cochrane Database Syst Rev 2013;10. https://doi.org/10.1002/14651858.CD010414.pub2.
- Hoogendoorn WE, van Poppel MN, Bongers PM, Koes BW, Bouter LM. Physical load during work and leisure time as risk factors for back pain. Scand J Work Environ Health 1999;25:387-403. https://doi.org/10.5271/sjweh.451.
- Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: rating the quality of evidence–indirectness. J Clin Epidemiol 2011;64:1303-10. https://doi.org/10.1016/j.jclinepi.2011.04.014.
- Baxter SK, Blank L, Woods HB, Payne N, Rimmer M, Goyder E. Using logic model methods in systematic review synthesis: describing complex pathways in referral management interventions. BMC Med Res Methodol 2014;14. https://doi.org/10.1186/1471-2288-14-62.
- Rees K, Bennett P, West R, Davey Smith G, Ebrahim S. Psychological interventions for coronary heart disease. Cochrane Database Syst Rev 2004;8. https://doi.org/10.1002/14651858.CD002902.pub2.
- Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 2008;8. https://doi.org/10.1186/1471-2288-8-45.
- Baxter S, Killoran A, Kelly MP, Goyder E. Synthesizing diverse evidence: the use of primary qualitative data analysis methods and logic models in public health reviews. Public Health 2010;124:99-106. https://doi.org/10.1016/j.puhe.2010.01.002.
- Allmark P, Baxter S, Goyder E, Guillaume L, Crofton-Martin G. Assessing the health benefits of advice services: using research evidence and logic model methods to explore complex pathways. Health Soc Care Community 2013;21:59-68. https://doi.org/10.1111/j.1365-2524.2012.01087.x.
- Blank L, Baxter S, Woods HB, Goyder E, Lee A, Payne N, et al. Referral interventions from primary to specialist care: a systematic review of international evidence. Br J Gen Pract 2014;64:e765-74. https://doi.org/10.3399/bjgp14X682837.
- Anderson LM, Petticrew M, Rehfuess E, Armstrong R, Ueffing E, Baker P, et al. Using logic models to capture complexity in systematic reviews. Res Synth Methods 2011;2:33-42. https://doi.org/10.1002/jrsm.32.
- Weiss CH. Theory-based evaluation: past present and future. New Dir Eval 1997;76:41-55. https://doi.org/10.1002/ev.1086.
- Dyson A, Todd L. Dealing with complexity: theory of change evaluation and the full service extended schools initiative. Int J Res Meth Ed 2010;33:119-34. https://doi.org/10.1080/1743727X.2010.484606.
- Hansen FH. Choosing evaluation models: a discussion on evaluation design. Evaluation 2005;11:447-62. https://doi.org/10.1177/1356389005060265.
- Rossi PH, Lipsey MW, Freeman HE. Evaluation: A Systematic Approach. Thousand Oaks, CA: Sage Publications Ltd; 2004.
- Handler A, Issel M, Turnock B. A conceptual framework to measure performance of the public health system. Am J Public Health 2001;91:1235-9. https://doi.org/10.2105/AJPH.91.8.1235.
- Joly BM, Polyak G, Davis MV, Brewster J, Tremain B, Raevsky C, et al. Linking accreditation and public health outcomes: a logic model approach. J Public Health Manag Pract 2007;13:349-56. https://doi.org/10.1097/01.PHH.0000278027.56820.7e.
- Burford B, Lewin S, Welch V, Rehfuess E, Waters E. Assessing the applicability of findings in systematic reviews of complex interventions can enhance the utility of reviews for decision making. J Clin Epidemiol 2013;66:1251-61. https://doi.org/10.1016/j.jclinepi.2013.06.017.
- Burchett HE, Mayhew SH, Lavis JN, Dobrow MJ. When can research from one setting be useful in another? Understanding perceptions of the applicability and transferability of research. Health Promot Int 2013;28:418-30. https://doi.org/10.1093/heapro/das026.
- Wang S, Moss JR, Hiller JE. Applicability and transferability of interventions in evidence-based public health. Health Promot Int 2006;21:76-83. https://doi.org/10.1093/heapro/dai025.
- Burchett H, Umoquit M, Dobrow M. How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability frameworks. J Health Serv Res Policy 2011;16:238-44. https://doi.org/10.1258/jhsrp.2011.010124.
- Buffet C, Ciliska D, Thomas H. Can I Use this Evidence in my Program Decision? Assessing Applicability and Transferability of Evidence. Hamilton, ON: National Collaborating Centre for Methods and Tools; 2007.
- Addicott R, Dewar S. Improving Choice at the End of Life: A Descriptive Analysis of the Impact and Costs of the Marie Curie Delivering Choice Programme in Lincolnshire. London: The King’s Fund; 2008.
- Ahmad F, Roy A, Brady S, Belgeonne S, Dunn L, Pitts J. Care pathway initiative for people with intellectual disabilities: impact evaluation. J Nurs Manag 2007;15:700-2. https://doi.org/10.1111/j.1365-2934.2006.00734.x.
- Bakerly ND, Davies C, Dyer M, Dhillon P. Cost analysis of an integrated care model in the management of acute exacerbations of chronic obstructive pulmonary disease. Chron Respir Dis 2009;6:201-8. https://doi.org/10.1177/1479972309104279.
- Beacon A. Practice-integrated care teams – learning for a better future. J Integr Care 2015;23:74-87. https://doi.org/10.1108/JICA-01-2015-0005.
- Boyle A, Fuld J, Ahmed V, Bennett T, Robinson S. Does integrated emergency care reduce mortality and non-elective admissions? A retrospective analysis. Emerg Med J 2012;29:208-12. https://doi.org/10.1136/emj.2010.108142.
- Boyle AA, Ahmed V, Palmer CR, Bennett TJ, Robinson SM. Reductions in hospital admissions and mortality rates observed after integrating emergency care: a natural experiment. BMJ Open 2012;2. https://doi.org/10.1136/bmjopen-2012-000930.
- Boyle AA, Robinson SM, Whitwell D, Myers S, Bennett TJ, Hall N, et al. Integrated hospital emergency care improves efficiency. Emerg Med J 2008;25:78-82. https://doi.org/10.1136/emj.2007.051037.
- Choo T, Deb S, Wilkins J, Atiomo W. Evaluating the impact of the reconfiguration of gynaecology services at a university hospital NHS trust in the United Kingdom. BMC Health Serv Res 2014;14. https://doi.org/10.1186/1472-6963-14-428.
- Clarkson P, Brand C, Hughes J, Challis D. Integrating assessments of older people: examining evidence and impact from a randomised controlled trial. Age Ageing 2011;40:388-91. https://doi.org/10.1093/ageing/afr015.
- Coupe M. Integrated care in Herefordshire: a case study. J Integr Care 2013;21:198-207. https://doi.org/10.1108/JICA-01-2013-0001.
- Cunningham S, Logan C, Lockerbie L, Dunn MJ, McMurray A, Prescott RJ. Effect of an integrated care pathway on acute asthma/wheeze in children attending hospital: cluster randomized trial. J Pediatr 2008;152:315-20. https://doi.org/10.1016/j.jpeds.2007.09.033.
- National Evaluation of the Department of Health’s Integrated Care Pilots. London: DHSC; 2012.
- Dodd J, Taylor CE, Bunyan P, White PM, Thomas SM, Upton D. A service model for delivering care closer to home. Prim Health Care Res Dev 2011;12:95-111. https://doi.org/10.1017/S1463423610000356.
- Graffy J, Grande M, Campbell J. Case management for elderly patients at risk of hospital admission: a team approach. Prim Health Care Res Devel 2008;9:7-13. https://doi.org/10.1017/S1463423607000564.
- Gravelle H, Dusheiko M, Sheaff R, Sargent P, Boaden R, Pickard S, et al. Impact of case management (Evercare) on frail elderly patients: controlled before and after analysis of quantitative outcome data. BMJ 2007;334. https://doi.org/10.1136/bmj.39020.413310.55.
- Ham C. Working Together for Health: Achievements and Challenges in the Kaiser NHS Beacon Sites Programme. Birmingham: Health Services Management Centre, University of Birmingham; 2010.
- Harris M, Greaves F, Gunn L, Patterson S, Greenfield G, Car J, et al. Multidisciplinary group performance-measuring integration intensity in the context of the North West London Integrated Care Pilot. Int J Integr Care 2013;13. https://doi.org/10.5334/ijic.996.
- Hawthorne G, Grzebalski DK. Service redesign: the experience of Newcastle Diabetes Service 2001–2007. Pract Diabetes Int 2009;26:19-22. https://doi.org/10.1002/pdi.1323.
- Higginson IJ, Bausewein C, Reilly CC, Gao W, Gysels M, Dzingina M, et al. An integrated palliative and respiratory care service for patients with advanced disease and refractory breathlessness: a randomised controlled trial. Lancet Respir Med 2014;2:979-87. https://doi.org/10.1016/S2213-2600(14)70226-7.
- Hockley J, Watson J, Oxenham D, Murray SA. The integrated implementation of two end-of-life care tools in nursing care homes in the UK: an in-depth evaluation. Palliat Med 2010;24:828-38. https://doi.org/10.1177/0269216310373162.
- Huws DW, Cashmore D, Newcombe RG, Roberts C, Vincent J, Elwyn G. Impact of case management by advanced practice nurses in primary care on unplanned hospital admissions: a controlled intervention study. BMC Health Serv Res 2008;8. https://doi.org/10.1186/1472-6963-8-115.
- Jha S, Moran P, Blackwell A, Greenham H. Integrated care pathways: the way forward for continence services?. Eur J Obstet Gynecol Reprod Biol 2007;134:120-5. https://doi.org/10.1016/j.ejogrb.2006.07.028.
- Johnstone R, Jones A, Fowell A. Welsh Collaborative Care Pathway Project; 10 years experience of implementing and maintaining a care pathway for the last days of life. Int J Care Pathways 2011;15:39-43. https://doi.org/10.1258/jicp.2011.010030.
- Johnstone RP, Jones AR, Burton C, Fowell A. Evaluation of the Welsh integrated care pathway (ICP) for the last days of life. Palliat Med 2012;26.
- Julian S, Naftalin NJ, Clark M, Szczepura A, Rashid A, Baker R, et al. An integrated care pathway for menorrhagia across the primary-secondary interface: patients’ experience, clinical outcomes, and service utilisation. Qual Saf Health Care 2007;16:110-15. https://doi.org/10.1136/qshc.2005.016782.
- Kent P, Chalmers Y. A decade on: has the use of integrated care pathways made a difference in Lanarkshire?. J Nurs Manag 2006;14:508-20. https://doi.org/10.1111/j.1365-2934.2006.00707.x.
- Lamb BW, Jalil RT, Sevdalis N, Vincent C, Green JS. Strategies to improve the efficiency and utility of multidisciplinary team meetings in urology cancer care: a survey study. BMC Health Serv Res 2014;14. https://doi.org/10.1186/1472-6963-14-377.
- Letton C, Cheung C, Nordin A. Does an enhanced recovery integrated care pathway (ICP) encourage adherence to prescribing guidelines, accelerate postoperative recovery and reduce the length of stay for gynaecological oncology patients?. J Obs Gynae 2013;33:296-7. https://doi.org/10.3109/01443615.2012.758693.
- Levelt E, Thwaites B, Yadegarfar G. Integrated care pathway for acute coronary syndromes: does it help?. J Integr Care Pathways 2008;12:5-9. https://doi.org/10.1258/jicp.2008.008001.
- Lyon D, Miller J, Pine K. The Castlefields Integrated Care Model: the evidence summarised. J Integr Care 2006;14:7-12. https://doi.org/10.1108/14769018200600003.
- MacLean A, Fuller RM, Jaffrey EG, Hay AJ, Ho-Yen DO. Integrated care pathway for clostridium difficile helps patient management. Brit J Infect Contr 2008;9:15-7. https://doi.org/10.1177/1469044608098324.
- Mertes SC, Raut S, Khanduja V. Integrated care pathways in lower-limb arthroplasty: are they effective in reducing length of hospital stay?. Int Orthop 2013;37:1157-63. https://doi.org/10.1007/s00264-013-1829-1.
- Ng SM, Mariguddi S, Coward S, Middleton H. Paediatric community home nursing: a model of acute care. Br J Nurs 2014;23:209-12. https://doi.org/10.12968/bjon.2014.23.4.209.
- Offredy M, Cleary M, Bland A, Donovan A, Kelshiker A. Improving health and care for patients by redesigning services: the development and implementation of a clinical assessment service in Harrow Primary Care Trust. Qual Prim Care 2008;16:95-102.
- Paize F, White E, Heaf LJ, Baillie C, Kenny S, Couriel JM, et al. An integrated care pathway for optimizing the investigation and management of pediatric pleural empyema. J Pediatr Infect Dis 2007;2:77-82.
- Pearson V, Chant S. Different models of health and social care in Devon – observations and implications for commissioners and providers. J Integr Care 2011;19:22-6. https://doi.org/10.1108/14769011111164269.
- Pettie JM, Dow MA, Sandilands EA, Thanacoody HK, Bateman DN. An integrated care pathway improves the management of paracetamol poisoning. Emerg Med J 2012;29:482-6. https://doi.org/10.1136/emj.2011.111922.
- Richings C, Cook R, Roy A. Service evaluation of an integrated assessment and treatment service for people with intellectual disability with behavioural and mental health problems. J Intellect Disabil 2011;15:7-19. https://doi.org/10.1177/1744629511403563.
- Roberts JA, Maslin TK, Bakerly ND. Development of an integrated chronic obstructive pulmonary disease service model in an inner-city region in the UK: initial findings and 12-month results. Prim Care Respir J 2010;19:390-7. https://doi.org/10.4104/pcrj.2010.00071.
- Roberts S, Unadkat N, Chandok R, Sawtell T. Learning from the integrated care pilot in West London. London J Prim Care 2012;5:59-62. https://doi.org/10.1080/17571472.2013.11493376.
- Roland M, Lewis R, Steventon A, Abel G, Adams J, Bardsley M, et al. Case management for at-risk elderly patients in the English integrated care pilots: observational study of staff and patient experience and secondary care utilisation. Int J Integr Care 2012;12. https://doi.org/10.5334/ijic.850.
- Rowlandson PH, Smith C. An interagency service delivery model for autistic spectrum disorders and attention deficit hyperactivity disorder. Child Care Health Dev 2009;35:681-90. https://doi.org/10.1111/j.1365-2214.2009.00956.x.
- Ryan T, Hatfield B, Sharma I. Outcomes of referrals over a six-month period to a mental health gateway team. J Psychiatr Ment Health Nurs 2007;14:527-34. https://doi.org/10.1111/j.1365-2850.2007.01122.x.
- Simmons D, Yu D, Wenzel H. Changes in hospital admissions and inpatient tariff associated with a Diabetes Integrated Care Initiative: preliminary findings. J Diabetes 2014;6:81-9. https://doi.org/10.1111/1753-0407.12071.
- Sinclair L, Hunter R, Hagen S, Nelson D, Hunt J. A&E Mental Health Study Group . How effective are mental health nurses in A&E departments?. Emerg Med J 2006;23:687-92. https://doi.org/10.1136/emj.2005.033175.
- Smith C, Hough L, Cheung CC, Millington-Sanders C, Sutton E, Ross JR, et al. Coordinate My Care: a clinical service that coordinates care, giving patients choice and improving quality of life. BMJ Support Palliat Care 2012;2:301-7. https://doi.org/10.1136/bmjspcare-2012-000265.
- Soljak M, Cecil E, Gunn L, Broddle A, Hamilton S, Tahir A, et al. Quality of Care and Health Outcomes. North West London Integrated Care Pilot Evaluation: Report on Work Programme 3. London: Imperial College London; 2013.
- Steventon A, Bardsley M, Billings J, Georghiou T, Lewis G. An Evaluation of the Impact of Community-based Interventions on Hospital Use. A Case Study of Eight Partnership for Older People Projects (POPP). London: Nuffield Trust; 2011.
- Stokes J, Kristensen SR, Checkland K, Bower P. Effectiveness of multidisciplinary team case management: difference-in-differences analysis. BMJ Open 2016;6. https://doi.org/10.1136/bmjopen-2015-010468.
- Tucker H, Burgis M. Patients set the agenda on integrating community services in Norfolk. J Integr Care 2012;20:232-40. https://doi.org/10.1108/14769011211255267.
- Tucker S, Baldwin R, Hughes J, Benbow SM, Barker A, Burns A, et al. Integrating mental health services for older people in England – from rhetoric to reality. J Interprof Care 2009;23:341-54. https://doi.org/10.1080/13561820902739833.
- Waller SL, Delaney S, Strachan MW. Does an integrated care pathway enhance the management of diabetic ketoacidosis?. Diabet Med 2007;24:359-63. https://doi.org/10.1111/j.1464-5491.2007.02102.x.
- Wilberforce M, Tucker S, Brand C, Abendstern M, Jasper R, Challis D. Is integrated care associated with service costs and admission rates to institutional settings? An observational study of community mental health teams for older people in England. Int J Geriatr Psychiatry 2016;31:1208-16. https://doi.org/10.1002/gps.4424.
- Windle K, Wagland R, Forder J, D’Amico F, Janssen D, Wistow G. National Evaluation of Partnerships for Older People Projects. Final Report. London: Personal Social Services Research Unit; 2009.
- Allan HT, Brearley S, Byng R, Christian S, Clayton J, Mackintosh M, et al. People and teams matter in organizational change: professionals’ and managers’ experiences of changing governance and incentives in primary care. Health Serv Res 2014;49:93-112. https://doi.org/10.1111/1475-6773.12084.
- Amador S, Goodman C, Mathie E, Nicholson C. Evaluation of an organisational intervention to promote integrated working between health services and care homes in the delivery of end-of-life care for people with dementia: understanding the change process using a social identity approach. Int J Integr Care 2016;16. https://doi.org/10.5334/ijic.2426.
- Anderson ES, Pollard L, Conroy S, Clague-Baker N. Forming a new clinical team for frail older people: can a group development model help?. J Interprof Care 2014;28:163-5. https://doi.org/10.3109/13561820.2013.853653.
- Bachmann MO, O’Brien M, Husbands C, Shreeve A, Jones N, Watson J, et al. Integrating children’s services in England: national evaluation of children’s trusts. Child Care Health Develop 2009;35:257-65. https://doi.org/10.1111/j.1365-2214.2008.00928.x.
- Baillie L, Gallini A, Corser R, Elworthy G, Scotcher A, Barrand A. Care transitions for frail, older people from acute hospital wards within an integrated healthcare system in England: a qualitative case study. Int J Integr Care 2014;14. https://doi.org/10.5334/ijic.1175.
- Barnett J, Vasileiou K, Djemil F, Brooks L, Young T. Understanding innovators’ experiences of barriers and facilitators in implementation and diffusion of healthcare service innovations: a qualitative study. BMC Health Serv Res 2011;11. https://doi.org/10.1186/1472-6963-11-342.
- Belling R, Whittock M, McLaren S, Burns T, Catty J, Jones IR, et al. Achieving continuity of care: facilitators and barriers in community mental health teams. Implement Sci 2011;6. https://doi.org/10.1186/1748-5908-6-23.
- Bouamrane MM, Mair FS. Implementation of an integrated preoperative care pathway and regional electronic clinical portal for preoperative assessment. BMC Med Inform Decis Mak 2014;14. https://doi.org/10.1186/1472-6947-14-93.
- Bridges J, Fitzgerald L, Meyer J. New workforce roles in health care: exploring the longer-term journey of organisational innovations. J Health Organ Manag 2007;21:381-92. https://doi.org/10.1108/jhom.2007.02521daa.001.
- Cheyne H, Abhyankar P, McCourt C. Empowering change: realist evaluation of a Scottish Government programme to support normal birth. Midwifery 2013;29:1110-21. https://doi.org/10.1016/j.midw.2013.07.018.
- Cleland J, Moffat M, Small I. A qualitative study of stakeholder views of a community-based anticipatory care service for patients with COPD. Prim Care Respir J 2012;21:255-60. https://doi.org/10.4104/pcrj.2012.00008.
- Collins F, McCray J. Partnership working in services for children: use of the common assessment framework. J Interprof Care 2012;26:134-40. https://doi.org/10.3109/13561820.2011.630111.
- Curry N, Harris M, Gunn LH, Pappas Y, Blunt I, Soljak M, et al. Integrated care pilot in north-west London: a mixed methods evaluation. Int J Integr Care 2013;13. https://doi.org/10.5334/ijic.1149.
- Dattée B, Barlow J. Complexity and whole-system change programmes. J Health Serv Res Policy 2010;15:19-25. https://doi.org/10.1258/jhsrp.2009.009097.
- Dent M, Tutt D. Electronic patient information systems and care pathways: the organisational challenges of implementation and integration. Health Informatics J 2014;20:176-88. https://doi.org/10.1177/1460458213518545.
- Dickinson H, Glasby J. How effective is joint commissioning? A study of five English localities. J Integr Care 2013;21:221-32. https://doi.org/10.1108/JICA-04-2013-0012.
- Dodds S, Chamberlain C, Williamson GR. Modernising chronic obstructive pulmonary disease admissions to improve patient care: local outcomes from implementing the Ideal Design of Emergency Access project. Accid Emerg Nurs 2006;14:141-7. https://doi.org/10.1016/j.aaen.2006.03.006.
- Erens B, Wistow G, Mounier-Jack S, Douglas N, Jones L, Manacorda T, et al. Early Evaluation of the Integrated Care and Support Pioneers Programme: Interim Report. London: London School of Hygiene and Tropical Medicine; 2015.
- Evans S, Means R, Powell J. Making care homes part of the community? An evaluation of the Gloucestershire Partnerships for Older People Project. Qual Ageing Older Adults 2013;14:66-74. https://doi.org/10.1108/14717791311311120.
- Farrington C, Clare IC, Holland AJ, Barrett M, Oborn E. Knowledge exchange and integrated services: experiences from an integrated community intellectual (learning) disability service for adults. J Intellect Disabil Res 2015;59:238-47. https://doi.org/10.1111/jir.12131.
- Freeman T, Peck E. Evaluating partnerships: a case study of integrated specialist mental health services. Health Soc Care Community 2006;14:408-17. https://doi.org/10.1111/j.1365-2524.2006.00658.x.
- Gambles M, Stirzaker S, Jack BA, Ellershaw JE. The Liverpool Care Pathway in hospices: an exploratory study of doctor and nurse perceptions. Int J Palliat Nurs 2006;12:414-21. https://doi.org/10.12968/ijpn.2006.12.9.21869.
- Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R. How do you modernize a health service? A realist evaluation of whole-scale transformation in London. Milbank Q 2009;87:391-416. https://doi.org/10.1111/j.1468-0009.2009.00562.x.
- Greenhalgh T, Macfarlane F, Barton-Sweeney C, Woodard F. ‘If we build it, will it stay?’ A case study of the sustainability of whole-system change in London. Milbank Q 2012;90:516-47. https://doi.org/10.1111/j.1468-0009.2012.00673.x.
- Griffiths C, Gerressu M, French RS. One-Stop Shop Evaluation Team . Are one-stop shops acceptable? Community perspectives on one-stop shop models of sexual health service provision in the UK. Sex Transm Infect 2008;84:395-9. https://doi.org/10.1136/sti.2008.030833.
- Haddow G, O’Donnell CA, Heaney D. Stakeholder perspectives on new ways of delivering unscheduled health care: the role of ownership and organizational identity. J Eval Clin Pract 2007;13:179-85. https://doi.org/10.1111/j.1365-2753.2006.00667.x.
- Hamilton S, Huby G, Tierney A, Powell A, Kielmann T, Sheikh A, et al. Mind the gap between policy imperatives and service provision: a qualitative study of the process of respiratory service development in England and Wales. BMC Health Serv Res 2008;8. https://doi.org/10.1186/1472-6963-8-248.
- Harris M, Greaves F, Gunn L, Patterson S, Greenfield G, Car J, et al. Multidisciplinary integration in the context of integrated care – results from the North West London Integrated Care Pilot. Int J Integr Care 2013;13. https://doi.org/10.5334/ijic.1146.
- Heenan D, Birrell D. The integration of health and social care: the lessons from Northern Ireland. Soc Pol Administrat 2006;40:47-66. https://doi.org/10.1111/j.1467-9515.2006.00476.x.
- Hendy J, Barlow J. The role of the organizational champion in achieving health system change. Soc Sci Med 2012;74:348-55. https://doi.org/10.1016/j.socscimed.2011.02.009.
- Hewison A, Lord L, Bailey C. ‘It’s been quite a challenge’: redesigning end-of-life care in acute hospitals. Palliat Support Care 2015;13:609-18. https://doi.org/10.1017/S1478951514000170.
- Hu M. The impact of an integrated care service on service users: the service users’ perspective. J Health Organ Manag 2014;28:495-510. https://doi.org/10.1108/JHOM-04-2012-0074.
- Huby G, Harris FM, Powell AE, Kielman T, Sheikh A, Williams S, et al. Beyond professional boundaries: relationships and resources in health services’ modernisation in England and Wales. Sociol Health Illn 2014;36:400-15. https://doi.org/10.1111/1467-9566.12067.
- Hudson B. Integrated team working: you can get it if you really want it: part I. J Integr Care 2006;14:13-21. https://doi.org/10.1108/14769018200600004.
- Ignatowicz A, Greenfield G, Pappas Y, Car J, Majeed A, Harris M. Achieving provider engagement: providers’ perceptions of implementing and delivering integrated care. Qual Health Res 2014;24:1711-20. https://doi.org/10.1177/1049732314549024.
- Kassianos AP, Ignatowicz A, Greenfield G, Majeed A, Car J, Pappas Y. ‘Partners rather than just providers . . .’: a qualitative study on health care professionals’ views on implementation of multidisciplinary group meetings in the North West London Integrated Care Pilot. Int J Integr Care 2015;15. https://doi.org/10.5334/ijic.2019.
- Knowles SE, Chew-Graham C, Coupe N, Adeyemi I, Keyworth C, Thampy H, et al. Better together? A naturalistic qualitative study of inter-professional working in collaborative care for co-morbid depression and physical health problems. Implement Sci 2013;8. https://doi.org/10.1186/1748-5908-8-110.
- Lhussier M, Carr SM, Wilcockson J. The evaluation of an end-of-life integrated care pathway. Int J Palliat Nurs 2007;13:74-81. https://doi.org/10.12968/ijpn.2007.13.2.23072.
- Ling T, Brereton L, Conklin A, Newbould J, Roland M. Barriers and facilitators to integrating care: experiences from the English Integrated Care Pilots. Int J Integr Care 2012;12. https://doi.org/10.5334/ijic.982.
- Lunts P. Change management in integrated care: what helps and hinders middle managers – a case study. J Integr Care 2012;20:246-54. https://doi.org/10.1108/14769011211255285.
- Macfarlane F, Greenhalgh T, Humphrey C, Hughes J, Butler C, Pawson R. A new workforce in the making? A case study of strategic human resource management in a whole-system change effort in healthcare. J Health Organ Manag 2011;25:55-72. https://doi.org/10.1108/14777261111116824.
- Manley K, Martin A, Jackson C, Wright T. Using systems thinking to identify workforce enablers for a whole systems approach to urgent and emergency care delivery: a multiple case study. BMC Health Serv Res 2016;16. https://doi.org/10.1186/s12913-016-1616-y.
- McDowell JR, McPhail K, Halyburton G, Brown M, Lindsay G. Perceptions of a service redesign by adults living with type 2 diabetes. J Adv Nurs 2009;65:1432-41. https://doi.org/10.1111/j.1365-2648.2009.05003.x.
- McKenna H, Richey R, Keeney S, Hasson F, Sinclair M, Poulton B. The introduction of innovative nursing and midwifery roles: the perspective of healthcare managers. J Adv Nurs 2006;56:553-62. https://doi.org/10.1111/j.1365-2648.2006.04047.x.
- Oborn E, Dawson S. Knowledge and practice in multidisciplinary teams: struggle, accommodation and privilege. Human Relations 2010;63:1835-57. https://doi.org/10.1177/0018726710371237.
- Ovseiko PV, Melham K, Fowler J, Buchan AM. Organisational culture and post-merger integration in an academic health centre: a mixed-methods study. BMC Health Serv Res 2015;15. https://doi.org/10.1186/s12913-014-0673-3.
- Mastellos N, Gunn L, Harris M, Majeed A, Car J, Pappas Y. Assessing patients’ experience of integrated care: a survey of patient views in the North West London Integrated Care Pilot. Int J Integr Care 2014;14. https://doi.org/10.5334/ijic.1453.
- Petch A, Cook A, Miller E. Partnership working and outcomes: do health and social care partnerships deliver for users and carers?. Health Soc Care Community 2013;21:623-33. https://doi.org/10.1111/hsc.12050.
- Pinnock H, Huby G, Tierney A, Hamilton S, Powell A, Kielmann T, et al. Is multidisciplinary teamwork the key? A qualitative study of the development of respiratory services in the UK. J R Soc Med 2009;102:378-90. https://doi.org/10.1258/jrsm.2009.080356.
- Pollard L, Rogers S, Shribman J, Sprigings D, Sinfield P. A study of role expansion: a new GP role in cardiology care. BMC Health Serv Res 2014;14. https://doi.org/10.1186/1472-6963-14-205.
- Pollard LC, Graves H, Scott DL, Kingsley GH, Lempp H. Perceived barriers to integrated care in rheumatoid arthritis: views of recipients and providers of care in an inner-city setting. BMC Musculoskelet Disord 2011;12. https://doi.org/10.1186/1471-2474-12-19.
- Roberts L, Cameron G. Evaluation of the Integrated Care Communities 2 Programme (Incorporating the Integration Discovery Community). London: OPM Group; 2014.
- Rothera I, Jones R, Harwood R, Avery AJ, Fisher K, James V, et al. An evaluation of a specialist multiagency home support service for older people with dementia using qualitative methods. Int J Geriatr Psychiatry 2008;23:65-72. https://doi.org/10.1002/gps.1841.
- Scragg T. An evaluation of integrated team management. J Integr Care 2006;14:39-48. https://doi.org/10.1108/14769018200600023.
- Sheaff R, Boaden R, Sargent P, Pickard S, Gravelle H, Parker S, et al. Impacts of case management for frail elderly people: a qualitative study. J Health Serv Res Policy 2009;14:88-95. https://doi.org/10.1258/jhsrp.2008.007142.
- Sleeman KE, Koffman J, Bristowe K, Rumble C, Burman R, Leonard S, et al. ‘It doesn’t do the care for you’: a qualitative study of health care professionals’ perceptions of the benefits and harms of integrated care pathways for end of life care. BMJ Open 2015;5. https://doi.org/10.1136/bmjopen-2015-008242.
- Smith N, Barnes M. New jobs old roles – working for prevention in a whole-system model of health and social care for older people. Health Soc Care Community 2013;21:79-87. https://doi.org/10.1111/j.1365-2524.2012.01089.x.
- Stuart K. Collaborative agency to support integrated care for children, young people and families: an action research study. Int J Integr Care 2014;14. https://doi.org/10.5334/ijic.1171.
- Syson G, Bond J. Integrating health and social care teams in Salford. J Integr Care 2010;18:17-24. https://doi.org/10.5042/jic.2010.0132.
- Thiel V, Sanola L, Goodwin N, Kodner D. Developing Community Resource Teams in Pembrokeshire, Wales. Integration of Health and Social Care in Progress. London: The King’s Fund; 2013.
- Tucker H. Discovering integrated care in community hospitals. J Integr Care 2013;21:336-46. https://doi.org/10.1108/JICA-10-2013-0038.
- Waterson P, Eason K, Tutt D, Dent M. Using HIT to deliver integrated care for the frail elderly in the UK: current barriers and future challenges. Work 2012;41:4490-3. https://doi.org/10.3233/WOR-2012-0750-4490.
- Wilson R, Baines S, Cornford J, Martin M. ‘Trying to do a jigsaw without the picture on the box’: understanding the challenges of care integration in the context of single assessment for older people in England. Int J Integr Care 2007;7. https://doi.org/10.5334/ijic.186.
- Holder H, Gaskins M, Wistow G, Smith J. Putting Integrated Care into Practice: The North West London Experience. London: Nuffield Trust; 2015.
- Alexander JA, Bae D. Does the patient-centred medical home work? A critical synthesis of research on patient-centred medical homes and patient-related outcomes. Health Serv Manage Res 2012;25:51-9. https://doi.org/10.1258/hsmr.2012.012001.
- Allen D, Gillen E, Rixson L. Systematic review of the effectiveness of integrated care pathways: what works, for whom, in which circumstances?. Int J Evid Based Healthc 2009;7:61-74. https://doi.org/10.1111/j.1744-1609.2009.00127.x.
- Allen D, Rixson L. How has the impact of ‘care pathway technologies’ on service integration in stroke care been measured and what is the strength of the evidence to support their effectiveness in this respect?. Int J Evid Based Healthc 2008;6:78-110.
- Béland F, Hollander MJ. Integrated models of care delivery for the frail elderly: international perspectives. Gac Sanit 2011;25:138-46. https://doi.org/10.1016/j.gaceta.2011.09.003.
- Bélanger E, Rodríguez C. More than the sum of its parts? A qualitative research synthesis on multi-disciplinary primary care teams. J Interprof Care 2008;22:587-97. https://doi.org/10.1080/13561820802380035.
- Best A, Greenhalgh T, Lewis S, Saul JE, Carroll S, Bitz J. Large-system transformation in health care: a realist review. Milbank Q 2012;90:421-56. https://doi.org/10.1111/j.1468-0009.2012.00670.x.
- Boult C, Green AF, Boult LB, Pacala JT, Snyder C, Leff B. Successful models of comprehensive care for older adults with chronic conditions: evidence for the Institute of Medicine’s ‘Retooling for an aging America’ report. J Amer Geriatr Soc 2009;57:2328-37. https://doi.org/10.1111/j.1532-5415.2009.02571.x.
- Butler M, Kane RL, McAlpine D, Kathol RG, Fu SS, Hagedorn H, et al. Integration of mental health/substance abuse and primary care. Evid Rep Technol Assess 2008;173:1-362.
- Cameron A, Bostock L, Lart R. Service user and carers perspectives of joint and integrated working between health and social care. J Integr Care 2014;22:62-70. https://doi.org/10.1108/JICA-10-2013-0042.
- Cameron A, Lart R, Bostock L, Coomber C. Factors that promote and hinder joint and integrated working between health and social care services: a review of research literature. Health Soc Care Community 2014;22:225-33. https://doi.org/10.1111/hsc.12057.
- Davies SL, Goodman C, Bunn F, Victor C, Dickinson A, Iliffe S, et al. A systematic review of integrated working between care homes and health care services. BMC Health Serv Res 2011;11. https://doi.org/10.1186/1472-6963-11-320.
- de Bruin SR, Versnel N, Lemmens LC, Molema CC, Schellevis FG, Nijpels G, et al. Comprehensive care programs for patients with multiple chronic conditions: a systematic literature review. Health Policy 2012;107:108-45. https://doi.org/10.1016/j.healthpol.2012.06.006.
- Eklund K, Wilhelmson K. Outcomes of coordinated and integrated interventions targeting frail elderly people: a systematic review of randomised controlled trials. Health Soc Care Community 2009;17:447-58. https://doi.org/10.1111/j.1365-2524.2009.00844.x.
- Footman K, Garthwaite K, Bambra C, McKee M. Quality check: does it matter for quality how you organize and pay for health care? A review of the international evidence. Int J Health Serv 2014;44:479-505. https://doi.org/10.2190/HS.44.3.d.
- Genet N, Boerma WG, Kringos DS, Bouman A, Francke AL, Fagerström C, et al. Home care in Europe: a systematic literature review. BMC Health Serv Res 2011;11. https://doi.org/10.1186/1472-6963-11-207.
- Homer CJ, Klatka K, Romm D, Kuhlthau K, Bloom S, Newacheck P, et al. A review of the evidence for the medical home for children with special health care needs. Pediatrics 2008;122:e922-37. https://doi.org/10.1542/peds.2007-3762.
- Huntley AL, Thomas R, Mann M, Huws D, Elwyn G, Paranjothy S, et al. Is case management effective in reducing the risk of unplanned hospital admissions for older people? A systematic review and meta-analysis. Fam Pract 2013;30:266-75. https://doi.org/10.1093/fampra/cms081.
- Hussain M, Seitz D. Integrated models of care for medical inpatients with psychiatric disorders: a systematic review. Psychosomatics 2014;55:315-25. https://doi.org/10.1016/j.psym.2013.08.003.
- Jackson G, Powers BJ, Chatterjee R, Bettger JP, Kemper AR, Hasselblad V, et al. The patient-centred medical home: a systematic review. Ann Intern Med 2013;158:169-78. https://doi.org/10.7326/0003-4819-158-3-201302050-00579.
- Johansson G, Eklund K, Gosman-Hedström G. Multidisciplinary team, working with elderly persons living in the community: a systematic literature review. Scand J Occup Ther 2010;17:101-16. https://doi.org/10.1080/11038120902978096.
- Kammerlander C, Roth T, Friedman SM, Suhm N, Luger TJ, Kammerlander-Knauer U, et al. Ortho-geriatric service – a literature review comparing different models. Osteoporosis Int 2010;21:S637-46. https://doi.org/10.1007/s00198-010-1396-x.
- Kinley J, Froggatt K, Bennett MI. The effect of policy on end-of-life care practice within nursing care homes: a systematic review. Palliat Med 2013;27:209-20. https://doi.org/10.1177/0269216311432899.
- Kuhlmann AS, Gavin L, Galavotti C. The integration of family planning with other health services: a literature review. Int Perspect Sex Reprod Health 2010;36:189-96. https://doi.org/10.1363/3618910.
- Laver K, Lannin NA, Bragge P, Hunter P, Holland AE, Tavender E, et al. Organising health care services for people with an acquired brain injury: an overview of systematic reviews and randomised controlled trials. BMC Health Serv Res 2014;14. https://doi.org/10.1186/1472-6963-14-397.
- Loader BD, Hardey M, Keeble L. Health informatics for older people: a review of ICT facilitated integrated care for older people. Int J Soc Welfare 2008;17:46-53. https://doi.org/10.1111/j.1468-2397.2007.00489.x.
- Low LF, Yap M, Brodaty H. A systematic review of different models of home and community care services for older persons. BMC Health Serv Res 2011;11. https://doi.org/10.1186/1472-6963-11-93.
- MacAdam M. Frameworks of Integrated Care for the Elderly: A Systematic Review. Ottowa, ON: Canadian Policy Research Network; 2008.
- Mackie S, Darvill A. Factors enabling implementation of integrated health and social care: a systematic review. Br J Community Nurs 2016;21:82-7. https://doi.org/10.12968/bjcn.2016.21.2.82.
- Martínez-González NA, Berchtold P, Ullman K, Busato A, Egger M. Integrated care programmes for adults with chronic conditions: a meta-review. Int J Qual Health Care 2014;26:561-70. https://doi.org/10.1093/intqhc/mzu071.
- Maslin-Prothero SE, Bennion AE. Integrated team working: a literature review. Int J Integr Care 2010;10.
- Mason A, Goddard M, Weatherly H, Chalkley M. Integrating funds for health and social care: an evidence review. J Health Serv Res Policy 2015;20:177-88. https://doi.org/10.1177/1355819614566832.
- McConnell T, O’Halloran P, Porter S, Donnelly M. Systematic realist review of key factors affecting the successful implementation and sustainability of the Liverpool Care Pathway for the dying patient. Worldviews Evid Based Nurs 2013;10:218-37. https://doi.org/10.1111/wvn.12003.
- Myors KA, Schmied V, Johnson M, Cleary M. Collaboration and integrated services for perinatal mental health: an integrative review. Child Adol Ment Health 2013;18:1-10. https://doi.org/10.1111/j.1475-3588.2011.00639.x.
- Nicholson C, Jackson C, Marley J. A governance model for integrated primary/secondary care for the health-reforming first world – results of a systematic review. BMC Health Serv Res 2013;13. https://doi.org/10.1186/1472-6963-13-528.
- Peikes D, Chen A, Schore J, Brown R. Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. JAMA 2009;301:603-18. https://doi.org/10.1001/jama.2009.126.
- Smith E, Ross FM. Service user involvement and integrated care pathways. Int J Health Care Qual Assur 2007;20:195-214. https://doi.org/10.1108/09526860710743345.
- Stewart MJ, Georgiou A, Westbrook JI. Successfully integrating aged care services: a review of the evidence and tools emerging from a long-term care program. Int J Integr Care 2013;13. https://doi.org/10.5334/ijic.963.
- Stokes J, Panagioti M, Alam R, Checkland K, Cheraghi-Sohi S, Bower P. Effectiveness of case management for ‘at risk’ patients in primary care: a systematic review and meta-analysis. PLoS One 2015;10. https://doi.org/10.1371/journal.pone.0132340.
- Suter E, Oelke ND, Adair CE, Armitage GD. Ten key principles for successful health systems integration. Healthc Q 2009;13:16-23. https://doi.org/10.12927/hcq.2009.21092.
- Tieman J, Mitchell G, Shelby-James T, Currow D, Fazekas B, O’Doherty L, et al. Integration, coordination and multidisciplinary care: what can these approaches offer to Australian primary health care?. Aust J Prim Health 2007;13:56-65. https://doi.org/10.1071/PY07024.
- Trivedi D, Goodman C, Gage H, Baron N, Scheibl F, Iliffe S, et al. The effectiveness of inter-professional working for older people living in the community: a systematic review. Health Soc Care Community 2013;21:113-28. https://doi.org/10.1111/j.1365-2524.2012.01067.x.
- Xyrichis A, Lowton K. What fosters or prevents interprofessional teamworking in primary and community care? A literature review. Int J Nurs Stud 2008;45:140-53. https://doi.org/10.1016/j.ijnurstu.2007.01.015.
- Aiken LS, Butner J, Lockhart CA, Volk-Craft BE, Hamilton G, Williams FG. Outcome evaluation of a randomized trial of the PhoenixCare intervention: program of case management and coordinated care for the seriously chronically ill. J Palliat Med 2006;9:111-26. https://doi.org/10.1089/jpm.2006.9.111.
- Battersby M, Harvey P, Mills PD, Kalucy E, Pols RG, Frith PA, et al. SA HealthPlus: a controlled trial of a statewide application of a generic model of chronic illness care. Milbank Q 2007;85:37-6. https://doi.org/10.1111/j.1468-0009.2007.00476.x.
- Béland F, Bergman H, Lebel P, Clarfield AM, Tousignant P, Contandriopoulos AP, et al. A system of integrated care for older persons with disabilities in Canada: results from a randomized controlled trial. J Gerontol A Biol Sci Med Sci 2006;61:367-73. https://doi.org/10.1093/gerona/61.4.367.
- Béland F, Bergman H, Lebel P, Dallaire L, Fletcher J, Contandriopoulos AP, et al. Integrated services for frail elders (SIPA): a trial of a model for Canada. Can J Aging 2006;25:5-42. https://doi.org/10.1353/cja.2006.0018.
- Bird S, Noronha M, Sinnott H. An integrated care facilitation model improves quality of life and reduces use of hospital resources by patients with chronic obstructive pulmonary disease and chronic heart failure. Aust J Prim Health 2010;16:326-33. https://doi.org/10.1071/PY10007.
- Bird SR, Kurowski W, Dickman GK, Kronborg I. Integrated care facilitation for older patients with complex health care needs reduces hospital demand. Aust Health Rev 2007;31:451-61. https://doi.org/10.1071/AH070451.
- Bird SR, Noronha M, Kurowski W, Orkin C, Sinnott H. Integrated care facilitation model reduces use of hospital resources by patients with pediatric asthma. J Healthc Qual 2012;34:25-33. https://doi.org/10.1111/j.1945-1474.2011.00143.x.
- Boult C, Leff B, Boyd CM, Wolff JL, Marsteller JA, Frick KD, et al. A matched-pair cluster-randomized trial of guided care for high-risk older patients. J Gen Intern Med 2013;28:612-21. https://doi.org/10.1007/s11606-012-2287-y.
- Boult C, Reider L, Frey K, Leff B, Boyd CM, Wolff JL, et al. Early effects of ‘Guided Care’ on the quality of health care for multimorbid older persons: a cluster-randomized controlled trial. J Gerontol A Biol Sci Med Sci 2008;63:321-7. https://doi.org/10.1093/gerona/63.3.321.
- Boult C, Reider L, Leff B, Frick KD, Boyd CM, Wolff JL, et al. The effect of guided care teams on the use of health services: results from a cluster-randomized controlled trial. Arch Intern Med 2011;171:460-6. https://doi.org/10.1001/archinternmed.2010.540.
- Brännström M, Boman K. Effects of person-centred and integrated chronic heart failure and palliative home care. PREFER: a randomized controlled study. Eur J Heart Fail 2014;16:1142-51. https://doi.org/10.1002/ejhf.151.
- Brown RS, Peikes D, Peterson G, Schore J, Razafindrakoto CM. Six features of Medicare coordinated care demonstration programs that cut hospital admissions of high-risk patients. Health Aff 2012;31:1156-66. https://doi.org/10.1377/hlthaff.2012.0393.
- Callahan CM, Boustani MA, Unverzagt FW, Austrom MG, Damush TM, Perkins AJ, et al. Effectiveness of collaborative care for older adults with Alzheimer disease in primary care: a randomized controlled trial. JAMA 2006;295:2148-57. https://doi.org/10.1001/jama.295.18.2148.
- Colla CH, Lewis VA, Kao LS, O’Malley AJ, Chang CH, Fisher ES. Association between Medicare Accountable Care Organization implementation and spending among clinically vulnerable beneficiaries. JAMA Intern Med 2016;176:1167-75. https://doi.org/10.1001/jamainternmed.2016.2827.
- Colla CH, Wennberg DE, Meara E, Skinner JS, Gottlieb D, Lewis VA, et al. Spending differences associated with the Medicare Physician Group Practice Demonstration. JAMA 2012;308:1015-23. https://doi.org/10.1001/2012.jama.10812.
- Counsell SR, Callahan CM, Clark DO, Tu W, Buttar AB, Stump TE, et al. Geriatric care management for low-income seniors: a randomized controlled trial. JAMA 2007;298:2623-33. https://doi.org/10.1001/jama.298.22.2623.
- Counsell SR, Callahan CM, Tu W, Stump TE, Arling GW. Cost analysis of the Geriatric Resources for Assessment and Care of Elders care management intervention. J Am Geriatr Soc 2009;57:1420-6. https://doi.org/10.1111/j.1532-5415.2009.02383.x.
- Dorr DA, Wilcox AB, Brunker CP, Burdon RE, Donnelly SM. The effect of technology-supported, multidisease care management on the mortality and hospitalization of seniors. J Am Geriatr Soc 2008;56:2195-202. https://doi.org/10.1111/j.1532-5415.2008.02005.x.
- Ettner SL, Kotlerman J, Afifi A, Vazirani S, Hays RD, Shapiro M, et al. An alternative approach to reducing the costs of patient care? A controlled trial of the multi-disciplinary doctor-nurse practitioner (MDNP) model. Med Decis Making 2006;26:9-17. https://doi.org/10.1177/0272989X05284107.
- Fagan P, Schuster A, Boyd C, Marsteller J, Griswold M, Murphy S. Chronic care improvement in primary care: evaluation of an integrated pay-for-performance and practice-based care coordination program among elderly patients with diabetes. Health Serv Res 2010;45:1763-82. https://doi.org/10.1111/j.1475-6773.2010.01166.x.
- Farmer JE, Clark MJ, Drewel EH, Swenson TM, Ge B. Consultative care coordination through the medical home for CSHCN: a randomized controlled trial. Matern Child Health J 2011;15:1110-18. https://doi.org/10.1007/s10995-010-0658-8.
- Gray D, Armstrong CD, Dahrouge S, Hogg W, Zhang W. Cost-effectiveness of anticipatory and preventive multidisciplinary team care for complex patients: evidence from a randomized controlled trial. Can Fam Physician 2010;56:e20-9.
- Hajewski CJ, Shirey MR. Care coordination: a model for the acute care hospital setting. J Nurs Adm 2014;44:577-85. https://doi.org/10.1097/NNA.0000000000000129.
- Hammar T, Rissanen P, Perälä ML. The cost-effectiveness of integrated home care and discharge practice for home care patients. Health Policy 2009;92:10-2. https://doi.org/10.1016/j.healthpol.2009.02.002.
- Hébert R, Raîche M, Dubois MF, Gueye NR, Dubuc N, Tousignant M. PRISMA Group . Impact of PRISMA, a coordination-type integrated service delivery system for frail older people in Quebec (Canada): a quasi-experimental study. J Gerontol B Psychol Sci Soc Sci 2010;65B:107-18. https://doi.org/10.1093/geronb/gbp027.
- Hildebrandt H, Schulte T, Stunder B. Triple Aim in Kinzigtal, Germany: improving population health, integrating health care and reducing costs of care – lessons for the UK?. J Integr Care 2012;20:205-22. https://doi.org/10.1108/14769011211255249.
- Hogg W, Lemelin J, Dahrouge S, Liddy C, Armstrong CD, Legault F, et al. Randomized controlled trial of anticipatory and preventive multidisciplinary team care: for complex patients in a community-based primary care setting. Can Fam Physician 2009;55:e76-85.
- Hullick C, Conway J, Higgins I, Hewitt J, Dilworth S, Holliday E, et al. Emergency department transfers and hospital admissions from residential aged care facilities: a controlled pre–post design study. BMC Geriatr 2016;16. https://doi.org/10.1186/s12877-016-0279-1.
- Jack BW, Chetty VK, Anthony D, Greenwald JL, Sanchez GM, Johnson AE, et al. A reengineered hospital discharge program to decrease rehospitalization: a randomized trial. Ann Intern Med 2009;150:178-87. https://doi.org/10.7326/0003-4819-150-3-200902030-00007.
- Janse B, Huijsman R, de Kuyper RD, Fabbricotti IN. The effects of an integrated care intervention for the frail elderly on informal caregivers: a quasi-experimental study. BMC Geriatr 2014;14. https://doi.org/10.1186/1471-2318-14-58.
- Janse B, Huijsman R, Fabbricotti IN. A quasi-experimental study of the effects of an integrated care intervention for the frail elderly on informal caregivers’ satisfaction with care and support. BMC Health Serv Res 2014;14. https://doi.org/10.1186/1472-6963-14-140.
- Katon W, Russo J, Lin EH, Schmittdiel J, Ciechanowski P, Ludman E, et al. Cost-effectiveness of a multicondition collaborative care intervention: a randomized controlled trial. Arch Gen Psychiatry 2012;69:506-14. https://doi.org/10.1001/archgenpsychiatry.2011.1548.
- Martinussen M, Adolfsen F, Lauritzen C, Richardsen AM. Improving interprofessional collaboration in a community setting: relationships with burnout, engagement and service quality. J Interprof Care 2012;26:219-25. https://doi.org/10.3109/13561820.2011.647125.
- McGregor M, Lin EH, Katon WJ. TEAMcare: an integrated multicondition collaborative care program for chronic illnesses and depression. J Ambul Care Manage 2011;34:152-62. https://doi.org/10.1097/JAC.0b013e31820ef6a4.
- Morales-Asencio JM, Gonzalo-Jiménez E, Martin-Santos FJ, Morilla-Herrera JC, Celdráan-Mañas M, Carrasco AM, et al. Effectiveness of a nurse-led case management home care model in primary health care. A quasi-experimental, controlled, multi-centre study. BMC Health Serv Res 2008;8. https://doi.org/10.1186/1472-6963-8-193.
- Olsson LE, Hansson E, Ekman I, Karlsson J. A cost-effectiveness study of a patient-centred integrated care pathway. J Adv Nurs 2009;65:1626-35. https://doi.org/10.1111/j.1365-2648.2009.05017.x.
- Parsons M, Senior H, Kerse N, Chen MH, Jacobs S, Vanderhoorn S, et al. Should care managers for older adults be located in primary care? A randomized controlled trial. J Am Geriatr Soc 2012;60:86-92. https://doi.org/10.1111/j.1532-5415.2011.03763.x.
- Paulus AT, van Raak AJ. The impact of integrated care on direct nursing home care. Health Policy 2008;85:45-59. https://doi.org/10.1016/j.healthpol.2007.05.014.
- Paulus AT, van Raak AJ, Maarse HJ. Is integrated nursing home care cheaper than traditional care? A cost comparison. Int J Nurs Stud 2008;45:1764-77. https://doi.org/10.1016/j.ijnurstu.2008.05.005.
- Rosenheck R, Leslie D, Sint K, Lin H, Robinson DG, Schooler NR, et al. Cost-effectiveness of comprehensive, integrated care for first episode psychosis in the NIMH RAISE Early Treatment Program. Schizophr Bull 2016;42:896-90. https://doi.org/10.1093/schbul/sbv224.
- Sahlen KG, Boman K, Brännström M. A cost-effectiveness study of person-centered integrated heart failure and palliative home care: based on a randomized controlled trial. Palliat Med 2016;30:296-302. https://doi.org/10.1177/0269216315618544.
- Salmon RB, Sanderson MI, Walters BA, Kennedy K, Flores RC, Muney AM. A collaborative accountable care model in three practices showed promising early results on costs and quality of care. Health Aff 2012;31:2379-87. https://doi.org/10.1377/hlthaff.2012.0354.
- de Stampa M, Vedel I, Buyck JF, Lapointe L, Bergman H, Béland F, et al. Impact on hospital admissions of an integrated primary care model for very frail elderly patients. Arch Gerontol Geriatr 2014;58:350-5. https://doi.org/10.1016/j.archger.2014.01.005.
- Stewart M, Sangster JF, Ryan BL, Hoch JS, Cohen I, McWilliam CL, et al. Integrating physician services in the home: evaluation of an innovative program. Can Fam Physician 2010;56:1166-74.
- Taylor A, Lizzi M, Marx A, Chilkatowsky M, Trachtenberg SW, Ogle S. Implementing a care coordination program for children with special healthcare needs: partnering with families and providers. J Healthc Qual 2013;35:70-7. https://doi.org/10.1111/j.1945-1474.2012.00215.x.
- Theodoridou A, Hengartner MP, Gairing SK, Jäger M, Ketteler D, Kawohl W, et al. Evaluation of a new person-centered integrated care model in psychiatry. Psychiatr Q 2015;86:153-68. https://doi.org/10.1007/s11126-014-9310-x.
- van der Marck MA, Munneke M, Mulleners W, Hoogerwaard EM, Borm GF, Overeem S, et al. IMPACT study group . Integrated multidisciplinary care in Parkinson’s disease: a non-randomised, controlled trial (IMPACT). Lancet Neurol 2013;12:947-56. https://doi.org/10.1016/S1474-4422(13)70196-0.
- van Gils RF, Bosmans JE, Boot CR, Rustemeyer T, van Mechelen W, van der Valk PG, et al. Economic evaluation of an integrated care programme for patients with hand dermatitis. Contact Derm 2013;69:144-52. https://doi.org/10.1111/cod.12095.
- Wennberg DE, Marr A, Lang L, O’Malley S, Bennett G. A randomized trial of a telephone care-management strategy. N Engl J Med 2010;363:1245-55. https://doi.org/10.1056/NEJMsa0902321.
- Berry LL, Rock BL, Smith Houskamp B, Brueggeman J, Tucker L. Care coordination for patients with complex health profiles in inpatient and outpatient settings. Mayo Clin Proc 2013;88:184-94. https://doi.org/10.1016/j.mayocp.2012.10.016.
- Blewett LA, Owen RA. Accountable care for the poor and underserved: Minnesota’s Hennepin Health model. Am J Public Health 2015;105:622-4. https://doi.org/10.2105/AJPH.2014.302432.
- Brawer PA, Martielli R, Pye PL, Manwaring J, Tierney A. St. Louis Initiative for Integrated Care Excellence (SLI(2)CE): integrated-collaborative care on a large scale model. Fam Syst Health 2010;28:175-87. https://doi.org/10.1037/a0020342.
- Breton M, Pineault R, Levesque J-F, Roberge D, Da Silva RB, Prud’homme A. Reforming healthcare systems on a locally integrated basis: is there a potential for increasing collaborations in primary healthcare?. BMC Health Serv Res 2013;13. https://doi.org/10.1186/1472-6963-13-262.
- Brokel JM, Harrison MI. Redesigning care processes using an electronic health record: a system’s experience. Jt Comm J Qual Patient Saf 2009;35:82-9. https://doi.org/10.1016/S1553-7250(09)35011-4.
- Callaly T, von Treuer K, van Hamond T, Windle K. Forming and sustaining partnerships to provide integrated services for young people: an overview based on the headspace Geelong experience. Early Interv Psychiatry 2011;5:28-33. https://doi.org/10.1111/j.1751-7893.2010.00237.x.
- Chen C, Garrido T, Chock D, Okawa G, Liang L. The Kaiser Permanente Electronic Health Record: transforming and streamlining modalities of care. Health Aff 2009;28:323-33. https://doi.org/10.1377/hlthaff.28.2.323.
- Cohen E, Lacombe-Duncan A, Spalding K, MacInnis J, Nicholas D, Narayanan UG, et al. Integrated complex care coordination for children with medical complexity: a mixed-methods evaluation of tertiary care-community collaboration. BMC Health Serv Res 2012;12. https://doi.org/10.1186/1472-6963-12-366.
- Epstein AM, Jha AK, Orav EJ, Liebman DL, Audet AM, Zezza MA, et al. Analysis of early Accountable Care Organizations defines patient, structural, cost, and quality-of-care characteristics. Health Aff 2014;33:95-102. https://doi.org/10.1377/hlthaff.2013.1063.
- Fuller RL, Clinton S, Goldfield NI, Kelly WP. Building the affordable medical home. J Ambul Care Manage 2010;33:71-80. https://doi.org/10.1097/JAC.0b013e3181c9fca9.
- Guerrero EG, Aarons GA, Palinkas LA. Organizational capacity for service integration in community-based addiction health services. Am J Public Health 2014;104:e40-7. https://doi.org/10.2105/AJPH.2013.301842.
- Hartgerink JM, Cramm JM, Bakker TJ, van Eijsden RA, Mackenbach JP, Nieboer AP. The importance of relational coordination for integrated care delivery to older patients in the hospital. J Nurs Manag 2014;22:248-56. https://doi.org/10.1111/j.1365-2834.2012.01481.x.
- Hébert R, Veil A, Raîche M, Dubois MF, Dubuc N, Tousignant M. Evaluation of the implementation of PRISMA, a coordination-type integrated service delivery system for frail older people in Québec. J Integr Care 2008;16:4-14. https://doi.org/10.1108/14769018200800041.
- Kautz CM, Gittell JH, Weinberg DB, Lusenhop RW, Wright J. Patient benefits from participating in an integrated delivery system: impact on coordination of care. Health Care Manage Rev 2007;32:284-94. https://doi.org/10.1097/01.HMR.0000281629.30149.b1.
- Khanna N, Shaya F, Chirikov V, Steffen B, Sharp D. Dissemination and adoption of the advanced primary care model in the Maryland multi-payer patient centered medical home program. J Health Care Poor Underserved 2014;25:122-38. https://doi.org/10.1353/hpu.2014.0066.
- Ouwens MM, Hermens RR, Termeer RA, Vonk-Okhuijsen SY, Tjan-Heijnen VC, Verhagen AF, et al. Quality of integrated care for patients with nonsmall cell lung cancer: variations and determinants of care. Cancer 2007;110:1782-90. https://doi.org/10.1002/cncr.22986.
- Pineault R, Borgès Da Silva R, Prud’homme A, Fournier M, Couture A, Provost S, et al. Impact of Québec’s healthcare reforms on the organization of primary healthcare (PHC): a 2003–2010 follow-up. BMC Health Serv Res 2014;14. https://doi.org/10.1186/1472-6963-14-229.
- Veerbeek L, van Zuylen L, Swart SJ, van der Maas PJ, de Vogel-Voogt E, van der Rijt CC, et al. The effect of the Liverpool Care Pathway for the dying: a multi-centre study. Palliat Med 2008;22:145-51. https://doi.org/10.1177/0269216307087164.
- Weaver FM, Hickey EC, Hughes SL, Parker V, Fortunato D, Rose J, et al. Providing all-inclusive care for frail elderly veterans: evaluation of three models of care. J Am Geriatr Soc 2008;56:345-53. https://doi.org/10.1111/j.1532-5415.2007.01538.x.
- Wedel R, Kalischuk RG, Patterson E, Brown S. Turning vision into reality: successful integration of primary healthcare in Taber, Canada. Healthc Policy 2007;3:80-95. https://doi.org/10.12927/hcpol.2007.19175.
- Ahgren B, Axelsson R. Determinants of integrated health care development: chains of care in Sweden. Int J Health Plann Manage 2007;22:145-57. https://doi.org/10.1002/hpm.870.
- Ahgren B, Nordgren L. Is choice of care compatible with integrated health care? An exploratory study in Sweden. Int J Health Plann Manage 2012;27:e162-72. https://doi.org/10.1002/hpm.2104.
- Alexander JA, Cohen GR, Wise CG, Green LA. The policy context of patient centered medical homes: perspectives of primary care providers. J Gen Intern Med 2013;28:147-53. https://doi.org/10.1007/s11606-012-2135-0.
- Anderson JK, Bonner A, Grootjans J. Collaboration: developing integration in multipurpose services in rural New South Wales, Australia. Rural Remote Health 2011;11:1-7.
- Banfield M, Gardner K, McRae I, Gillespie J, Wells R, Yen L. Unlocking information for coordination of care in Australia: a qualitative study of information continuity in four primary health care models. BMC Fam Pract 2013;14. https://doi.org/10.1186/1471-2296-14-34.
- Berendsen AJ, Benneker WH, Schuling J, Rijkers-Koorn N, Slaets JP, Meyboom-de Jong B. Collaboration with general practitioners: preferences of medical specialists – a qualitative study. BMC Health Serv Res 2006;6. https://doi.org/10.1186/1472-6963-6-155.
- Birken SA, Lee SY, Weiner BJ, Chin MH, Schaefer CT. Improving the effectiveness of health care innovation implementation: middle managers as change agents. Med Care Res Rev 2013;70:29-45. https://doi.org/10.1177/1077558712457427.
- Brousselle A, Lamothe L, Sylvain C, Foro A, Perreault M. Integrating services for patients with mental and substance use disorders: what matters?. Health Care Manage Rev 2010;35:212-23. https://doi.org/10.1097/HMR.0b013e3181d5b11c.
- Camden C, Swaine B, Tétreault S, Carrière M. Going beyond the identification of change facilitators to effectively implement a new model of services: lessons learned from a case example in paediatric rehabilitation. Dev Neurorehabil 2011;14:247-60. https://doi.org/10.3109/17518423.2011.577049.
- Carter WC, Lee SY, Thomas KC, Morrissey J. Managed care, inter-agency linkages, and outpatient substance abuse treatment. Adm Policy Ment Health 2006;33:122-31. https://doi.org/10.1007/s10488-005-0010-6.
- Collinsworth A, Vulimiri M, Snead C, Walton J. Community health workers in primary care practice: redesigning health care delivery systems to extend and improve diabetes care in underserved populations. Health Promot Pract 2014;15:51-6. https://doi.org/10.1177/1524839914539961.
- Costich JF, Scutchfield FD, Ingram RC. Population health, public health, and accountable care: emerging roles and relationships. Am J Public Health 2015;105:846-50. https://doi.org/10.2105/AJPH.2014.302484.
- DeMiglio L, Williams A. Factors enabling shared care with primary healthcare providers in community settings: the experiences of interdisciplinary palliative care teams. J Palliat Care 2012;28:282-9.
- Donnelly C, Brenchley C, Crawford C, Letts L. The integration of occupational therapy into primary care: a multiple case study design. BMC Fam Pract 2013;14. https://doi.org/10.1186/1471-2296-14-60.
- Fagnan LJ, Dorr DA, Davis M, McGinnis P, Mahler J, King MM, et al. Turning on the care coordination switch in rural primary care: voices from the practices – clinician champions, clinician partners, administrators, and nurse care managers. J Ambul Care Manage 2011;34:304-18. https://doi.org/10.1097/JAC.0b013e31821c63ee.
- Gaboury I, Bujold M, Boon H, Moher D. Interprofessional collaboration within Canadian integrative healthcare clinics: key components. Soc Sci Med 2009;69:707-15. https://doi.org/10.1016/j.socscimed.2009.05.048.
- Hadjistavropoulos HD, Dunn-Pierce T, Biem HJ. Provider perceptions of implementation of integrated care pathways for patients with chronic heart conditions. Can J Cardiovasc Nurs 2008;18:20-6.
- Hogan C, Barry M, Burke M, Joyce P. Healthcare professionals’ experiences of the implementation of integrated care pathways. Int J Health Care Qual Assur 2011;24:334-47. https://doi.org/10.1108/09526861111139179.
- Hroscikoski MC, Solberg LI, Sperl-Hillen JM, Harper PG, McGrail MP, Crabtree BF. Challenges of change: a qualitative study of chronic care model implementation. Ann Fam Med 2006;4:317-26. https://doi.org/10.1370/afm.570.
- Jové AM, Fernández A, Hughes C, Guillén-Solà M, Rovira M, Rubio-Valera M. Perceptions of collaboration between general practitioners and community pharmacists: findings from a qualitative study based in Spain. J Interprof Care 2014;28:352-7. https://doi.org/10.3109/13561820.2014.898621.
- Kathol RG, Butler M, McAlpine DD, Kane RL. Barriers to physical and mental condition integrated service delivery. Psychosom Med 2010;72:511-18. https://doi.org/10.1097/PSY.0b013e3181e2c4a0.
- Kilbourne AM, Greenwald DE, Bauer MS, Charns MP, Yano EM. Mental health provider perspectives regarding integrated medical care for patients with serious mental illness. Adm Policy Ment Health 2012;39:448-57. https://doi.org/10.1007/s10488-011-0365-9.
- Kilbourne AM, Irmiter C, Capobianco J, Reynolds K, Milner K, Barry K, et al. Improving integrated general medical and mental health services in community-based practices. Adm Policy Ment Health 2008;35:337-45. https://doi.org/10.1007/s10488-008-0177-8.
- Kreindler SA, Larson BK, Wu FM, Carluzzo KL, Gbemudu JN, Struthers A, et al. Interpretations of integration in early Accountable Care Organizations. Milbank Q 2012;90:457-83. https://doi.org/10.1111/j.1468-0009.2012.00671.x.
- Lewis VA, Colla CH, Schoenherr KE, Shortell SM, Fisher ES. Innovation in the safety net: integrating community health centers through accountable care. J Gen Intern Med 2014;29:1484-90. https://doi.org/10.1007/s11606-014-2911-0.
- Lukas CV, Holmes SK, Cohen AB, Restuccia J, Cramer IE, Shwartz M, et al. Transformational change in health care systems: an organizational model. Health Care Manage Rev 2007;32:309-20. https://doi.org/10.1097/01.HMR.0000296785.29718.5d.
- Lyngsø AM, Godtfredsen NS, Frølich A. Interorganisational integration: healthcare professionals’ perspectives on barriers and facilitators within the Danish healthcare system. Int J Integr Care 2016;16. https://doi.org/10.5334/ijic.2449.
- Minkman MM, Ligthart SA, Huijsman R. Integrated dementia care in the Netherlands: a multiple case study of case management programmes. Health Soc Care Community 2009;17:485-94. https://doi.org/10.1111/j.1365-2524.2009.00850.x.
- Ortiz J, Bushy A, Zhou Y, Zhang H. Accountable Care Organizations: benefits and barriers as perceived by rural health clinic management. Rural Remote Health 2013;13.
- Ottevanger N, Hilbink M, Weenk M, Janssen R, Vrijmoeth T, de Vries A, et al. Oncologic multidisciplinary team meetings: evaluation of quality criteria. J Eval Clin Pract 2013;19:1035-43. https://doi.org/10.1111/jep.12022.
- Ruppert D, Stegbauer C, Bramesfeld A, Bestmann B, Szecsenyi J, Gotz K. ‘Hope dies last . . .’ – cross-sectoral cooperation in integrated care. Psychiatr Prax 2016;9.
- Rosen R, Mountford J, Lewis G, Lewis R, Shand J, Shaw S. Integration in Action: Four International Case Studies. London: Nuffield Trust; 2011.
- Tousijn W. Integrating health and social care: interprofessional relations of multidisciplinary teams in Italy. Current Sociol 2012;60:522-37. https://doi.org/10.1177/0011392112438335.
- Tummers JF, Schrijvers AJ, Visser-Meily JM. A qualitative study of stakeholder views on the effects of provider payment on cooperation, quality of care and cost-containment in integrated stroke care. BMC Health Serv Res 2013;13. https://doi.org/10.1186/1472-6963-13-127.
- Walker KO, Labat A, Choi J, Schmittdiel J, Stewart AL, Grumbach K. Patient perceptions of integrated care: confused by the term, clear on the concept. Int J Integr Care 2013;13.
- Wang MC, Hyun JK, Harrison M, Shortell SM, Fraser I. Redesigning health systems for quality: lessons from emerging practices. Jt Comm J Qual Patient Saf 2006;32:599-611. https://doi.org/10.1016/S1553-7250(06)32078-8.
- Wodskou PM, Høst D, Godtfredsen NS, Frølich A. A qualitative study of integrated care from the perspectives of patients with chronic obstructive pulmonary disease and their relatives. BMC Health Serv Res 2014;14. https://doi.org/10.1186/1472-6963-14-471.
- Addicott R. Challenges of commissioning and contracting for integrated care in the National Health Service (NHS) in England. Aust J Prim Health 2016;22:50-4. https://doi.org/10.1071/PY15067.
- Katon W, Unützer J, Fan MY, Williams JW, Schoenbaum M, Lin EH, et al. Cost-effectiveness and net benefit of enhanced treatment of depression for older adults with diabetes and depression. Diabetes Care 2006;29:265-70. https://doi.org/10.2337/diacare.29.02.06.dc05-1572.
- van Gils RF, Boot CR, Knol DL, Rustemeyer T, van Mechelen W, van der Valk PG, et al. The effectiveness of integrated care for patients with hand eczema: results of a randomized, controlled trial. Contact Derm 2012;66:197-204. https://doi.org/10.1111/j.1600-0536.2011.02024.x.
- Johnston B, Buchanan D, Papadopoulou C, Sandeman G, Lord H. Integrating palliative care in lung cancer: an early feasibility study. Int J Palliat Nurs 2013;19:433-7. https://doi.org/10.12968/ijpn.2013.19.9.433.
- Macfarlane DP, Voigt D, Mackie A, Brennan G. Introduction of an integrated care pathway improves management of diabetic ketoacidosis. Diabetic Med 2011;28.
- Anderson A, Chojnacka I. Benefits of using the Liverpool Care Pathway in end of life care. Nurs Stand 2012;26:42-50. https://doi.org/10.7748/ns2012.04.26.34.42.c9066.
- Brännström M, Boman K. A new model for integrated heart failure and palliative advanced homecare – rationale and design of a prospective randomized study. Eur J Cardiovasc Nurs 2013;12:269-75. https://doi.org/10.1177/1474515112445430.
- Cohen E, Bruce-Barrett C, Kingsnorth S, Keilty K, Cooper A, Daub S. Integrated complex care model: lessons learned from inter-organizational partnership. Healthc Q 2011;14:64-70. https://doi.org/10.12927/hcq.0000.22580.
- Brewah H. Integrated health care for patients with motor neurone disease. Br J Nurs 2013;22:1182-8. https://doi.org/10.12968/bjon.2013.22.20.1182.
- Oliver R, Thakar R, Sultan AH, Phillimore A. Urogynecology triage clinic: a model of healthcare delivery. Int Urogynecol J Pelvic Floor Dysfunct 2009;20:913-17. https://doi.org/10.1007/s00192-009-0878-x.
Appendix 1 Search strategy
MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, Epub Ahead of Print, MEDLINE without Revisions via OvidSP
Search strategy
-
(care adj1 model*).ti,ab.
-
new.ti,ab.
-
1 and 2
-
(model* adj1 service delivery).ti,ab.
-
(model* adj1 (healthcare or health care or health-care)).ti,ab.
-
(transform* adj1 (service* or care)).ti,ab.
-
(“integration of care” or integrated care).ti,ab.
-
(integrated system* and health).ti,ab.
-
(vanguard* and (health or service*)).ti,ab.
-
accountable care.ti,ab.
-
(future proof* or future-proof* or futureproof*).ti,ab.
-
(reform* adj (health or service* or care or healthcare)).ti,ab.
-
(service* adj1 redesign*).ti,ab.
-
“five year forward view”.ti,ab.
-
((health or healthcare or service*) adj reform*).ti.
-
or/3-15
-
*Health Care Reform/
-
*Organizational Innovation/
-
*Quality Improvement/
-
*Health Priorities/
-
*Accountable Care Organizations/
-
*“Delivery of Health Care, Integrated”/
-
(og or td).fs.
-
or/17-22
-
23 and 24
-
(health service* or healthcare or health care or model*).ti,ab.
-
25 and 26
-
16 or 27
-
limit 28 to english language
-
limit 29 to yr=”2006-Current”
Search filters
-
To retrieve systematic reviews, the following search filter was combined with (AND) line 30 of the above search strategy: Scottish Intercollegiate Guidelines Network (SIGN). Search Filters: Systematic Reviews. Available from www.sign.ac.uk/assets/search-filters-systematic-reviews.docx (accessed 5 July 2017).
-
To retrieve UK primary studies, the following terms were combined with (AND) line 30 of the above search strategy: exp Great Britain/ OR (Britain or british or wales or welsh or Scottish or scots or Scotland or England or English or Birmingham or leeds or London or Liverpool or Manchester or Glasgow or Edinburgh or Cardiff or Belfast or UK or GB or aberdeen).ti,ab,in,hw.
EMBASE via OvidSP
Search strategy
-
(care adj1 model*).ti,ab.
-
new.ti,ab.
-
1 and 2
-
(model* adj1 service delivery).ti,ab.
-
(model* adj1 (healthcare or health care or health care)).ti,ab.
-
(transform* adj1 (service* or care)).ti,ab.
-
(“integration of care” or integrated care).ti,ab.
-
(integrated system* and health).ti,ab.
-
(vanguard* and (health or service*)).ti,ab.
-
accountable care.ti,ab.
-
(future proof* or future-proof* or futureproof*).ti,ab.
-
(reform* adj (health or service* or care or healthcare)).ti,ab.
-
(service* adj1 redesign*).ti,ab.
-
“five year forward view”.ti,ab.
-
((health or healthcare or service*) adj reform*).ti.
-
or/3-15
-
*Health Care Reform/
-
*Organizational Innovation/
-
*Quality Improvement/
-
*Health Priorities/
-
*Accountable Care Organizations/
-
*“Delivery of Health Care, Integrated”/
-
(og or td).fs.
-
or/17-22
-
23 and 24
-
(health service* or healthcare or health care or model*).ti,ab.
-
25 and 26
-
16 or 27
-
limit 28 to embase
-
limit 29 to english language
-
limit 30 to yr=”2006-Current”
Search filters
-
To retrieve systematic reviews, the following search filter was combined with (AND) line 31 of the above search strategy: Scottish Intercollegiate Guidelines Network (SIGN). Search Filters: Systematic Reviews. Available from: www.sign.ac.uk/assets/search-filters-systematic-reviews.docx (accessed 5 July 2017).
-
To retrieve UK primary studies, the following terms were combined with (AND) line 31 of the above search strategy: exp Great Britain/ OR (Britain or british or wales or welsh or Scottish or scots or Scotland or England or English or Birmingham or leeds or London or Liverpool or Manchester or Glasgow or Edinburgh or Cardiff or Belfast or UK or GB or aberdeen).ti,ab,in,hw.
Cumulative Index to Nursing and Allied Health Literature via EBSCOhost
Search strategy
-
S1 TI (care n1 model*) OR AB (care n1 model*)
-
S2 TI new OR AB new
-
S3 S1 AND S2
-
S4 TI (model* n1 service delivery) OR AB (model* n1 service delivery)
-
S5 TI ( (model* n1 (healthcare or health care or health-care)) ) OR AB ( (model* n1 (healthcare or health care or health-care)) )
-
S6 TI ( (transform* n1 (service* or care)) ) OR AB ( (transform* n1 (service* or care)) )
-
S7 TI ( (“integration of care” or integrated care) ) OR AB ( (“integration of care” or integrated care) )
-
S8 TI ( (integrated system* and health) ) OR AB ( (integrated system* and health) )
-
S9 TI ( (vanguard* and (health or service*)) ) OR AB ( (vanguard* and (health or service*)) )
-
S10 TI accountable care OR AB accountable care
-
S11 TI ( (future proof* or future-proof* or futureproof*) ) OR AB ( (future proof* or future-proof* or futureproof*) )
-
S12 TI ( (reform* n1 (health or service* or care or healthcare)) ) OR AB ( (reform* n1 (health or service* or care or healthcare)) )
-
S13 TI (service* n1 redesign*) OR AB (service* n1 redesign*)
-
S14 TI “five year forward view” OR AB "five year forward view”
-
S15 S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 OR S11 OR S12 OR S13 OR S14
-
S16 S3 OR S15
-
S17 (MM “Health Care Reform”)
-
S18 (MM “Diffusion of Innovation”) OR (MM “Organizational Change”)
-
S19 (MM “Quality Improvement”)
-
S20 (MM “Accountable Care Organizations”)
-
S21 (MM “Health Care Delivery, Integrated”)
-
S22 MW OG or TD
-
S23 S17 OR S18 OR S19 OR S20 OR S21
-
S24 S22 AND S23
-
S25 TI ( (health service* or healthcare or health care or model*) ) OR AB ( (health service* or healthcare or health care or model*) )
-
S26 S24 AND S25
-
S27 S16 OR S26
Search filters
-
To retrieve systematic reviews, the following search filter was combined with (AND) line S27 of the above search strategy: Scottish Intercollegiate Guidelines Network (SIGN). Search Filters: Systematic Reviews. Available from: www.sign.ac.uk/assets/search-filters-systematic-reviews.docx (accessed 5 July 2017).
-
To retrieve UK primary studies, the following terms were combined with (AND) line S27 of the above search strategy: (MH ‘Great Britain’) OR (Britain or british or wales or welsh or Scottish or scots or Scotland or England or English or Birmingham or leeds or London or Liverpool or Manchester or Glasgow or Edinburgh or Cardiff or Belfast or UK or GB or aberdeen).
-
Search results were refined to English language and 2006–16 publication dates.
PyscINFO via OvidSP
Search strategy
-
(care adj1 model*).ti,ab.
-
new.ti,ab.
-
1 and 2
-
(model* adj1 service delivery).ti,ab.
-
(model* adj1 (healthcare or health care or health-care)).ti,ab.
-
(transform* adj1 (service* or care)).ti,ab.
-
(“integration of care” or integrated care).ti,ab.
-
(integrated system* and health).ti,ab.
-
(vanguard* and (health or service*)).ti,ab.
-
accountable care.ti,ab.
-
(future proof* or future-proof* or futureproof*).ti,ab.
-
(reform* adj (health or service* or care or healthcare)).ti,ab.
-
(service* adj1 redesign*).ti,ab.
-
"five year forward view".ti,ab.
-
((health or healthcare or service*) adj reform*).ti.
-
or/3-15
-
*Health Care Reform
-
*Organizational Innovation/
-
*Integrated Services/
-
17 or 18 or 19
-
(health service* or healthcare or health care or model*).ti,ab.
-
20 and 21
-
16 or 22
-
limit 23 to english language
-
limit 24 to yr=”2006-Current”
Search filters
-
To retrieve systematic reviews, the following search filter was combined with (AND) line 25 of the above search strategy: The University of Texas School of Public Health. Search Filters for Various Databases: Ovid PsycINFO (Systematic reviews and meta-analyses). Available from: http://libguides.sph.uth.tmc.edu/search_filters/ovid_psycinfo_filters [accessed 5 July 2017).
-
To retrieve UK primary studies, the following terms were combined with (AND) line 25 of the above search strategy: (Britain or british or wales or welsh or Scottish or scots or Scotland or England or English or Birmingham or leeds or London or Liverpool or Manchester or Glasgow or Edinburgh or Cardiff or Belfast or UK or GB or aberdeen).ti,ab,in,hw.
The Cochrane Library (Database of Abstracts of Reviews of Effects, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials and Health Technology Assessment)
Search strategy
-
#1 (care near/1 model*):ti,ab,kw (Word variations have been searched)
-
#2 new:ti,ab
-
#3 #1 and #2
-
#4 (model* near/1 service delivery):ti,ab
-
#5 (model* near/1 (healthcare or health care or health-care)):ti,ab
-
#6 (transform* near/1 (service* or care)):ti,ab
-
#7 (“integration of care” or integrated care):ti,ab
-
#8 (integrated system* and health):ti,ab
-
#9 (vanguard* and (health or service*)):ti,ab
-
#10 accountable care:ti,ab
-
#11 (future proof* or future-proof* or futureproof*):ti,ab
-
#12 (reform* next (health or service* or care or healthcare)):ti,ab
-
#13 (service* near/1 redesign*):ti,ab
-
#14 "five year forward view":ti,ab
-
#15 ((health or healthcare or service*) next reform*):ti
-
#16 {or #3-#15}
-
#17 MeSH descriptor: [Health Care Reform] this term only
-
#18 MeSH descriptor: [Organizational Innovation] this term only
-
#19 MeSH descriptor: [Quality Improvement] this term only
-
#20 MeSH descriptor: [Health Priorities] this term only
-
#21 MeSH descriptor: [Accountable Care Organizations] this term only
-
#22 MeSH descriptor: [Delivery of Health Care, Integrated] this term only
-
#23 Any MeSH descriptor with qualifier(s): [Organization & administration - OG]
-
#24 Any MeSH descriptor with qualifier(s): [Trends - TD]
-
#25 {or #17-#22}
-
#26 #24 and #25
-
#27 (health service* or healthcare or health care or model*):ti,ab
-
#28 #26 and #27
-
#29 #16 or #28 Publication Year from 2006 to 2016
-
#30 MeSH descriptor: [Great Britain] explode all trees
-
#31 (Britain or british or wales or welsh or Scottish or scots or Scotland or England or English or Birmingham or leeds or London or Liverpool or Manchester or Glasgow or Edinburgh or Cardiff or Belfast or UK or GB or aberdeen)
-
#32 #30 or #31
-
#33 #29 and #32 – UK primary studies
-
#34 #29 not #32 - Reviews
Search filters
No search filters were applied to The Cochrane Library searches.
Science Citation Index and Social Sciences Citation Index via Web of Science
-
#1 TOPIC: (care near/1 model*)
-
#2 TOPIC: (new)
-
#3 #2 AND #1
-
#4 TOPIC: (model* near/1 "service delivery")
-
#5 TOPIC: (model* near/1 (“healthcare” or "health care” or "health-care"))
-
#6 TOPIC: ((transform* near/1 (service* or care)))
-
#7 TOPIC: ((“integration of care” or integrated care))
-
#8 TOPIC: ((integrated system* and health))
-
#9 TOPIC: ((vanguard* and (health or service*)))
-
#10 TOPIC: (accountable care)
-
#11 TOPIC: ((future proof* or future-proof* or futureproof*))
-
#12 TOPIC: ((reform* near (health or service* or care or healthcare)))
-
#13 TOPIC: ((service* near/1 redesign*))
-
#14 TOPIC: (five year forward view)
-
#15 TITLE: (((health or healthcare or service*) near reform*))
-
#16 #15 OR #14 OR #13 OR #12 OR #11 OR #10 OR #9 OR #8 OR #7 OR #6 OR #5 OR #4 OR #3
Search filters
Methodological search filters are not available for Web of Science; therefore, the following approach was used to identify reviews and UK primary studies:
-
To retrieve systematic reviews, the following terms were combined with (AND) line #16 of the above search strategy TITLE: ((meta analysis) or (systematic review)) and the search results were refined to ‘Review’ for document type and the publication years 2006–16.
-
To retrieve UK primary studies, the following terms were combined with (AND) line #16 of the above search strategy TS=((Britain or british or wales or welsh or Scottish or scots or Scotland or England or English or Birmingham or leeds or London or Liverpool or Manchester or Glasgow or Edinburgh or Cardiff or Belfast or UK or United Kingdom or GB or aberdeen)) and the search results were refined to ‘Article’ for document type and the publication years 2006–16.
Appendix 2 Completed quality appraisal checklists
First author and year | Potential for | Potential reporting bias? (Selective reporting) | Other sources of bias (comment) | ||||
---|---|---|---|---|---|---|---|
Selection bias? | Performance bias? (Blinding of participants and personnel) | Detection bias? (Blinding of outcome assessments) | Attrition bias? (Incomplete outcome data assessments) | ||||
Random-sequence generation | Allocation concealment | ||||||
Boyle 201239 | NA; anonymised data set | NA | NA | N | N | N | Able to describe associations, not causal links |
Clarkson 201142 | NA; reanalysis of data from an earlier trial | ||||||
Cunningham 200844 | N; cluster | N | Y | N/Y | N | N | Many staff failed to complete required documentation |
Department of Health and Social Care 201245 | NA | NA | Y | Y | Y | N | Service data from intervention sites matched to control areas |
Gravelle 200748 | NA | NA | N | Y | N | N | Small sample so low power. Definition of high-risk group may be problematic. Used matched controls from other practices in England |
Higginson 201452 | N | Y | Single blind | Y | N | N | Self-reported primary outcome, short follow-up |
Huws 200854 | N | N | N (not possible) | N (admissions data) | NA | N | Small number of practices and nurses delivering intervention |
Julian 200758 | Y; compares different PCTs | N | Not possible | Y | Y | Y; no discussion of study limitations | Some data self-reported diary, only 32% of eligible GPs used the tool |
Lyon 200663 | Y; compares neighbouring practices | N | Y | Y | N; admissions data | N | Comparative observational study |
Roland 201274 | NA | NA | N (for hospital data) | N | N | N | Data from both a self-reported questionnaire and hospital routine data. Used matched controls from other areas |
Simmons 201477 | N | NA | NA (uses routine data) | N | N | N (only two outcomes) | Only 1 year of data for intervention |
Sinclair 200678 | NA; crossover design compares period of intervention to no intervention | NA | Not possible | N; data mostly routinely collected patient data | N; routine hospital data | N | The intervention period lasted only 12 weeks |
Steventon 201181 | N | NA | Not possible | N | N; routine hospital data | N | Most of the data were self-reported |
Stokes 201682 | NA; anonymised data from a data set, propensity matching, no significant baseline differences | NA | NA | NA | N; data from existing data set | N | Used a difference-in-differences design. Multiple intervention start dates |
Waller 200785 | NA; all patients in given time frame, small sample, some baseline differences | NA | CD | CD | N; retrospective data from case notes | CD; may have been other service changes in time period | Compares group of patients prior to introduction with a separate group of patients after introduction |
Wilberforce 201686 | CD; anonymised patient data randomly selected but no details | NA | Y | Y | Y; data for 877 of 960 patients – no explanation for missing data | N | The study is described as observational with two data time points, but compares two forms of teams and conclusions are based on this comparison |
First author and year | Quality criteria | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Was the study question or objective clearly stated? | Were eligibility/selection criteria for the study population prespecified and clearly described? | Were the participants in the study representative of those who would be eligible for the test/service/intervention in the general or clinical population of interest? | Were all eligible participants who met the prespecified entry criteria enrolled? | Was the sample size sufficiently large to provide confidence in the findings? | Was the test/service/intervention clearly described and delivered consistently across the study population? | Were the outcome measures prespecified, clearly defined, valid, reliable and assessed consistently across all study participants? | Were the people assessing the outcomes blinded to the participants’ exposures/interventions? | Was the loss to follow-up after baseline ≤ 20%? Were those lost to follow-up accounted for in the analysis? | Did the statistical methods examine changes in outcome measures from before to after the intervention? Were statistical tests done that provided p-values for the pre–post changes? | Were outcome measures of interest taken multiple times before the intervention and multiple times after the intervention (i.e. did they use an interrupted time-series design)? | If the intervention was conducted at a group level (e.g. a whole hospital or a community), did the statistical analysis take into account the use of individual-level data to determine effects at the group level? | |
Addicott 200834 | Y | Y | Y | N | Y | Y | Y | N | NA | Y | N | CD |
Ahmad 200735 | N | N | Y | CD | N | CD | N | N | NA | CD | N | N |
Bakerly 200936 | Y | Y | Y | N | CD | Y | Y | N | Y | Y | N | NA |
Beacon 201537 | Y | N | CD | CD | CD | Y | Y | N | NA | N | N | N |
Boyle 200840 | Y | NA | Y | NA | Y | Y | Y | N | NA | Y | N | CD |
Boyle 201238 | Y | NA | Y | NA | Y | Y | Y | N | NA | Y | N | CD |
Choo 201441 | Y | NA | Y | NA | Y | Y | Y | N | NA | Y | N | CD |
Coupe 201343 | N | NA | Y | NA | Y | CD | N | N | CD | N | N | NA |
Dodd 201146 | Y | N | Y | N | N | Y | CD | N | CD | Y | N | CD |
Graffy 200847 | Y | Y | Y | CD | N | Y | N | N | Y | N | N | N |
Ham 201049 | Y | Y | Y | CD | Y | N | N | CD | CD | N | N | N; overview of work at a number of sites |
Harris 201350 | Y | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA | NA |
Hawthorne 200951 | Y | NA | NA | NA | Y | CD | CD | N | NA | N | N | NA |
Hockley 201053 | Y | Y | Y | N | CD | N | Y | N | NA | Y | N | CD |
Jha 200755 | Y | CD | Y | N | N | CD | Y | N | NA | Y | N | N |
Johnstone 201257 | Y | Y | Y | N | Y | CD | Y | N | NA | N | N | NA |
Kent 200659 | Y | NA | Y | NA | Y | CD | Y | N | NA | Y | N | CD |
Letton 201361 | Y | CD | Y | CD | CD | CD | Y | N | NA | Y | N | CD |
Levelt 200862 | Y | Y | Y | N | CD | N | Y | N | NA | Y | N | N |
Mertes 201365 | Y | Y | Y | Y | Y | CD | Y | N | NA | Y | N | N |
Ng 201466 | Y | CD | Y | Y | Y | CD | Y | N | NA | N | N | N |
Paize 200768 | Y | Y | Y | CD | N | N | Y | N | NA | N | N | N |
Pettie 201170 | Y | Y | Y | Y | Y | N | Y | N | NA | Y | N | N |
Richings 201171 | Y | Y | Y | N | N | CD | Y | N | CD | N | N | N |
Roberts 201072 | Y | Y | Y | Y | Y | CD | Y | N | NA | Y | N | N |
Soljak 201380 | Y | Y | Y | CD | Y | CD | Y | Y | CD | Y | N | Y |
Tucker 201283 | Y | NA | Y (patients) | NA | N | CD | Y | N | NA | N | N | NA |
Windle 200987 | Y | Y | Y | N | Y | N | CD | N | Y | Y | N | Y |
First author and year | Quality criteria | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Was the research question clearly stated? | Was the study population clearly specified and defined? | Was the participation rate of eligible persons ≥ 50%? | Were all the subjects selected or recruited from the same or similar populations (including the same time period)? Were inclusion and exclusion criteria for being in the study prespecified and applied uniformly to all participants? | Was a sample size justification, power description, or variance and effect estimates provided? | For exposures that can vary in amount or level, did the study examine different levels of the exposure as related to the outcome? | Were the exposure measures (independent variables) clearly defined, valid, reliable and implemented consistently across all study participants? | Were the outcome measures (dependent variables) clearly defined, valid, reliable and implemented consistently across all study participants? | Were the outcome assessors blinded to the exposure status of participants? | Were key potential confounding variables measured and adjusted statistically for their impact on the relationship between exposure(s) and outcome(s)? | |
Lamb 201460 | Y | Y | Y; 54% | Y | N | NA | NA | NA; survey of staff | NA | NA; non-comparative |
MacLean 200864 | Y | N | N | N | N | NA | NA | NA; staff survey | NA | NA; non-comparative |
Offredy 200867 | N | N | CD | CD | N | CD | CD | CD; limited data | N | N |
Pearson 201169 | Y | N | CD | CD | N | CD | CD | Y | CD | N |
Roberts 201273 | Y | Y | Y | Y | N | NA | N | N | N | N |
Rowlandson 200975 | Y | Y | All eligible | Y | N | NA | NA | Few data on main outcome – timing of referral | NA | NA; non-comparative |
Ryan 200776 | Y | Y | CD | Y | N | NA | NA | Potential for inconsistent recording of information | NA | NA; non-comparative |
Smith 201279 | N | N | CD | N | N | NA | NA | N; missing data | NA | NA; non-comparative |
Tucker 200984 | Y | Y | Y (72%) | Y | NA; all eligible | NA | NA; survey of views | N; no reporting of survey development | NA | NA; non-comparative |
First author and year | Potential for | Potential reporting bias? (Selective reporting) | Other sources of bias (comment) | ||||
---|---|---|---|---|---|---|---|
Selection bias? | Performance bias? (Blinding of participants and personnel) | Detection bias? (Blinding of outcome assessments) | Attrition bias? (Incomplete outcome data assessments) | ||||
Random-sequence generation | Allocation concealment | ||||||
Aiken 2006194 | Y | Y | N | CD | N | N | |
Battersby 2007195 | Y | NA | NA | CD | CD | Y | Some health-care professionals cared for patients in both groups; variable patient selection |
Béland 2006;196 Béland 2006197 | Y | Y | NA | Y | N | CD | Inadequate statistical power (n = 1230, where n = 1270 was calculated to detect differences of 25% hospital and 50% nursing home utilisation; some contamination likely |
Bird 2010;198 Bird 2007199 | N (dummy control) | N | NA | Y | NA | CD | Self-selected groups, cannot establish cause–effect |
Bird 2012200 | N (dummy control) | N | NA | Y | NA | CD | As above |
Boult 2013;201 Boult 2008;202 Boult 2011203 | Y (cluster) | Y | NA | Y | Y | N | Inadequate statistical power; incomplete follow-up (frail elderly population) |
Brännström 2014204 | Y | N | NA | N | N | N | Small samples (n = 36 in each group), in one setting; intervention group had higher mean age than control |
Brown 2012205 | Y | Y | CD | Y | N | N | Inadequate statistical power |
Callahan 2006206 | Y (cluster) | Y | Until recruitment | Y | Y (but not outcomes relevant to our review) | N | Small samples (n = 84 in intervention group, n = 69 in control group) therefore inadequate statistical power |
Colla 2016;207 Colla 2012208 | CD | CD | CD | CD | N | N | High degree of heterogeneity |
Counsell 2007;209 Counsell 2009210 | Y | Y | Researchers blinded | CD | N | N | Multiple outcome measures and testing |
Dorr 2008211 | N | N | N | N | CD | CD | Differences in referral rates among physicians |
Ettner 2006212 | NA | NA | N | CD | N | N | Patients form own control group; self-reported data |
Fagan 2010213 | NA | N | CD | N | Y | N | Lack of data from comparator practices |
Farmer 2011214 | Y | CD | CD | CD | Y | N | Self-reported data; high dropout rate |
Gray 2010;215 Hogg 2009220 | Y | CD | N | N | N | N | Small sample |
Hajewski 2014216 | N | NA | N | N | Y | CD | Some data sets not available at follow-up (incomplete data recording of complex data) |
Hammar 2009217 | Y | Y | Y (low-risk patients); N (higher risk for researchers recruiting) | N | N | N | Higher number of participants recruited to intervention group than control group |
Hébert 2010218 | N | N | N | N | CD | N | Overall sample identified randomly |
Hildebrandt 2012219 | N | N | N | N | CD | Y | Selective and limited reporting |
Hullick 2016221 | N | N | N | CD | CD | N | Selection of participants on basis of number of ED attendances |
Jack 2009222 | Y | Partial | Y | CD | N | N |
Sample may be younger than other studies Self-reported data |
Janse 2014;223 Janse 2014224 | N | N | N | CD | Y | N | Small sample size; large loss to follow-up; large number of statistical tests carried out |
Martinussen 2012226 | N | N | N | N | N | CD | Pretest measures not carried out; course was voluntary; self-reported measures |
McGregor 2011;227 Katon 2010225 | Y | NR | Personnel blinded, blinding of patients not possible | Y | Y | CD/unclear | |
Morales-Asencio 2008228 | N | N | N | N | CD | CD | |
Olsson 2009229 | N | N | N | N | Y | CD | |
Parsons 2012230 | Y (cluster) | N | N | N | CD | CD | Practices randomised before participants identified |
Paulus 2008;231 Paulus 2008232 | N | N | N | N | NA | CD | |
Rosenheck 2016233 | Y (cluster) | N | CD (blinding of patients not possible) | Y | CD | CD | Practices randomised before participants identified |
Sahlen 2016234 | Y | NR | N | N | CD | N | Described as ‘open evaluation’; separate methods/protocol paper lists outcomes |
Salmon 2012235 | N | NA | NA | NA | NA | CD | Study uses mainly administrative data |
de Stampa 2014236 | N | N | N | N | CD | CD | |
Stewart 2010237 | N | N | N | N | N | CD | |
Taylor 2013238 | N | N | NA | N | NA | N | Small number of outcomes |
Theodoridou 2015239 | Y | Y | N (not possible) | N | N | CD | Allocation independent of trial personnel |
van der Marck 2013240 | N | N | CD | CD (wording unclear) | Y | CD | |
van Gils 2013241 | Y | Y | N (not possible) | Y | CD | CD | |
Wennberg 2010242 | Y | Y | Y | CD | CD | N |
First author and year | Quality criteria | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Were the research question and inclusion criteria specified a priori? | Were there at least two data extractors and was there a procedure for disagreements? | Were at least two databases searched? Did the report include years, databases and keywords? Was there consultation and reference list checking? | Did the review include a search for grey literature? | Was a list of included and excluded studies provided? | Were the characteristics of included studies provided in an aggregated form such as a table? | Was the quality of included studies assessed? | Was the quality of included studies accounted for in the conclusions? | Were the methods used to combine data appropriately? Do the authors refer to the extent of heterogeneity? | Was the likelihood of publication bias assessed? | Was the source of support included for the review and each included study? | |
Alexander 2012152 | Y | CD | N | N | N | N | Y | N | N | N | N |
Allen 2009;153 Allen 2008154 | Y | CD | Y | Y | Y | Y | Y | Y | Y | N | N |
Béland 2011155 | Y | Y | Y | CD | N | Y | Y | N | N | N | N |
Bélanger 2008156 | Y | CD | Y | N | N | Y | CD | N | N | N | N |
Best 2012157 | Y | Y | Y | Y | Y | Y | CD | N | N | N | N |
Boult 2009158 | Y | Y | N | CD | N | Y | CD | N | N | N | N |
Cameron 2014;160 Cameron 2014161 | Y | Y | N | CD | N | N | Y | N | N | N | N |
Davies 2011162 | Y | Y | Y | Y | N | Y | Y | Y | Y | N | N |
de Bruin 2012163 | CD | Y | Y | N | N | Y | Y | Y | Y | N | N |
Eklund 2009164 | CD | Y | N | N | N | Y | Y | Y | Y | N | N |
Footman 2014165 | Y | Y | Y | Y | Y | Y | Y | Y | Y | N | N |
Huntley 2013168 | Y | Y | Y | Y | Y | Y | Y | Y | Y | N | N |
Hussain 2014169 | CD | CD | N | N | N | Y | Y | Y | Y | N | N |
Jackson 2013170 | Y | Y | N | N | Y | Y | Y | Y | Y | N | N |
Johansson 2010171 | Y | U | N | N | N | N | N | N | NA | N | N |
Kammerlander 2010172 | Y | Y | Y | N | Y | Y | N | N | Y | N | N |
Kinley 2013173 | Y | Y | Y | Y | N | N | Y | Y | Y | N | N |
Kuhlmann 2010174 | Y | Y | Y | N | N | Y | CD | N | Y | N | N |
Laver 2014175 | Y | Y | Y | N | N | Y | Y | Y | Y | N | N |
Loader 2008176 | Y | CD | CD | CD | N | N | N | N | CD | N | N |
Low 2011177 | Y | Y | Y | N | N | N | N | Y | N | N | N |
MacAdam 2008178 | Y | CD | Y | Y | N | Y | N | N | Y | N | N |
Mackie 2016179 | N | CD | CD | N | N | N | Y | Y | N | N | N |
Martinez-González 2014180 | Y | Y | Y | N | Y | Y | Y | Y | N | N | N |
Maslin-Prothero 2010181 | Y | CD | N | N | N | N | N | N | N | N | N |
Mason 2015182 | Y | Y | Y | Y | N | N | N | N | N | N | N |
McConnell 2013183 | Y | CD | Y | Y | N | N | Y | Y | N | N | N |
Myors 2013184 | N | CD | N | CD | N | Y | N | N | N | N | N |
Nicholson 2013185 | CD | Y | Y | Y | N | Y | Y | N | N | N | N |
Stewart 2013188 | Y | CD | Y | Y | N | Y | N | N | N | N | N |
Stokes 2015189 | Y | Y | Y | N | Y | Y | Y | Y | Y | Y | N |
Suter 2009190 | N | CD | CD | N | N | N | Y | N | N | N | N |
Tieman 2006191 | Y | Y | Y | Y | Y | N | Y | Y | N | N | N |
Trivedi 2013192 | Y | Y | Y | N | N | N | Y | Y | N | N | N |
Xyrichis 2008193 | Y | CD | Y | N | N | N | N | N | NA | N | N |
First author and year | Quality criteria | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Is there a clear statement of the aims? | Is a qualitative methodology appropriate? | Was the research design appropriate to address the aims? | Was the recruitment strategy appropriate? | Were the data collected in a way that addressed the issue? | Was there consideration of the relationship between researcher and participants? | Were ethics issues taken into consideration? | Was data analysis sufficiently rigorous? | Was there a clear statement of findings? | How valuable is the research? (Comments) | |
Allan 201488 | Y | Y | Y | U | Y | N | U | Y | Y | Explores emotional and practical impact of change on staff and managerial perceptions; linked to relevant literature |
Amador 201689 | Y | Y | Y | Y | Y | Y | U | Y | Y | Provides insight into factors involved in integrated working of visiting staff and care home staff; relates to existing literature |
Anderson 201490 | Y | Y | Y | U | Y | N | U | U | Y | Reporting of methods is brief; useful data on experiences of newly formed teams; limited reference to literature |
Bachmann 200991 | Y | Y | Y | U | Y | U | Y | U | Y | Reports evaluation of integrated child services; some details of methods lacking |
Baillie 201492 | Y | Y | Y | Y | Y | Y | Y | Y | Y | Well-reported study assessing staff and patient issues around vertical integration for care transitions; relates findings to existing literature |
Barnett 201193 | Y | Y | Y | Y | Y | Y | Y | Y | Y | Well-reported study assessing innovation diffusion across organisations with links to literature |
Belling 201194 | Y | Y | Y | Y | Y | U | Y | Y | Y | Unpublished survey of continuity of care; lacks quotations to back up themes |
Bouamrane 201495 | Y | Y | Y | Y | Y | U | Y | Y | Y | Detailed and lengthy reporting of e-form portal for integrated pathway; relates findings to previous literature |
Bridges 200796 | Y | Y | Y | U | Y | Y | N | Y | Y | Action participation research examining innovation and role shift; relates findings to theoretical literature |
Cheyne 201397 | Y | Y | Y | Y | Y | U | Y | Y | Y | Study of change in care delivery; no quotations supplied but good analysis representations |
Cleland 201298 | Y | Y | Y | Y | Y | U | Y | Y | Y | Reports on new community service; findings not related to existing literature |
Collins 201299 | Y | Y | Y | Y | Y | U | Y | Y | Y | Reports on partnership working with some reference to previous work |
Curry 2013100 | Y | Y | U | Y | Y | U | U | Y | Y | Mixed-methods study reporting lessons learned from integrated care pilot; lacks quotations from qualitative methods |
Dattée 2010101 | Y | Y | Y | Y | Y | U | N | Y | Y | Identifies issues when change is implemented in complex systems; limited reference to existing literature in discussion |
Dent 2014102 | Y | Y | U | U | Y | N | N | U | Y | Assesses use of electronic ICPs; methods not clear; limited reference to existing literature in discussion |
Dickinson 2013103 | Y | Y | U | U | U | N | U | U | Y | Study of joint commissioning for services; many details of method lacking or brief |
Dodds 2006104 | Y | Y | Y | Y | Y | U | Y | U | Y | Evaluation of new COPD pathway; lacks quotations from qualitative data |
Erens 2015105 | Y | Y | Y | Y | Y | U | Y | Y | Y | No comments |
Evans 2013106 | Y | Y | Y | U | U | N | N | U | Y | Refers to report for details |
Farrington 2015107 | Y | Y | Y | Y | Y | U | Y | Y | Y | Well-reported service evaluation of knowledge transfer issues in learning disability |
Freeman 2006108 | Y | Y | Y | Y | Y | U | Y | Y | Y | Evaluation of partnership intervention |
Gambles 2006109 | Y | Y | Y | U | Y | N | U | Y | Y | Explores staff perceptions of care pathway |
Greenhalgh 2009111 | Y | Y | Y | U | Y | U | N | P | Y | Realist evaluation of modernisation initiative; no quotations but good visuals of analysis |
Greenhalgh 2012110 | Y | Y | Y | Y | Y | Y | Y | Y | Y | Well-reported evaluation of modernisation initiative |
Griffiths 2008112 | Y | Y | Y | U | Y | U | U | Y | Y | Assessment of acceptability of one-stop shop for sexual health services; some details not reported |
Haddow 2007113 | Y | Y | U | Y | Y | U | Y | Y | Y | Evaluation of nurse-led telephone advice service (NHS 24) |
Hamilton 2008114 | Y | Y | Y | Y | Y | U | Y | Y | Y | Well-reported assessment of issues encountered when organisations develop and reconfigure respiratory services |
Harris 2013115 | Y | Y | Y | Y | U | U | U | U | U | Assessed communication regarding multidisciplinary groups; no quotations, not really qualitative |
Heenan 2006116 | Y | Y | Y | Y | U | U | U | U | Y | Study of key issues in health and social care integration; lacks detail in places |
Hendy 2012117 | Y | Y | Y | Y | Y | Y | U | Y | Y | Well-reported study of the role of champions with good reference to literature |
Hewison 2015118 | Y | Y | Y | Y | Y | U | U | Y | Y | Redesign of end-of-life pathway |
Hu 2014119 | Y | Y | Y | U | Y | U | U | Y | Y | Study of integration programme; method detail lacking in places |
Huby 2014120 | Y | Y | Y | Y | Y | U | Y | Y | Y | Assesses professional boundaries associated with health-care change |
Hudson 2006121 | Y | Y | U | Y | Y | U | U | Y | Y | Action research approach |
Ignatowicz 2014122 | Y | Y | Y | U | Y | U | Y | Y | Y | Clear presentation of staff perceptions of their engagement in integrated care |
Kassianos 2015123 | Y | Y | Y | Y | U | Y | Y | Y | Y | Staff views of MDT meetings in integrated care |
Knowles 2013124 | Y | Y | Y | Y | U | Y | U | Y | Y | Study of issues in implementing collaborative care; discussion lacks reference to previous literature |
Lhussier 2007125 | Y | Y | Y | Y | Y | U | U | U | Y | Evaluation of end-of-life pathway; lacking some details in methods section |
Ling 2012126 | Y | Y | Y | U | U | U | U | Y | Y | Evaluation of integrated care pilots; details brief but reader directed to associated paper |
Lunts 2012127 | Y | Y | Y | U | Y | U | U | Y | Y | Assesses middle manager perceptions of integrated care; method section brief |
Macfarlane 2011128 | Y | Y | Y | U | Y | U | U | Y | Y | Realist review of human resource issues in an innovative programme; no quotations but good analysis visuals |
Manley 2016129 | Y | Y | Y | U | Y | U | Y | Y | Y | Reports enablers of delivering patient-centred transitional care |
McDowell 2009130 | Y | Y | Y | Y | Y | U | Y | Y | Y | Unpublished study of new diabetes care service delivery |
McKenna 2006131 | Y | Y | Y | Y | Y | U | Y | Y | Y | Well-reported exploration of manager perceptions of innovative roles |
Oborn 2010132 | Y | Y | Y | U | Y | Y | U | Y | Y | Analyses knowledge translation processes across occupational boundaries; well discussed, lacking some method details |
Ovseiko 2015133 | Y | Y | Y | U | Y | N | Y | Y | Y | Detailed presentation of findings, in particular related to organisational culture; limited discussion of findings in relation to other literature/policy |
Pappas 2012134 | Y | Y | Y | Y | Y | Y | N | Y | Y | Part of London evaluation |
Petch 2013135 | Y | Y | Y | Y | Y | Y | Y | Y | Y | Provides detailed outline of the study findings and implications |
Pinnock 2009136 | Y | Y | Y | Y | Y | Y | U | Y | Y | Interprets the findings with regard to other studies and provides key conclusions |
Pollard 2014137 | Y | Y | Y | U | Y | N | U | N | Y | Mixed-methods study with limited reporting of qualitative data; extensive discussion section |
Pollard 2011138 | Y | Y | Y | U | Y | N | U | Y | Y | The aims of the study were to investigate integrated care; however, most of the data relate to patient care and referral |
Roberts 2014139 | Y | Y | U | U | N | U | U | N | N | Main purpose of the paper was to evaluate the support provided |
Rothera 2008140 | Y | Y | Y | U | Y | U | Y | Y | Y | Evaluation of a new service model compared with usual care |
Scragg 2006141 | Y | Y | Y | U | Y | N | U | N | Y | Explores staff perceptions and provides insight into process of change |
Sheaff 2009142 | Y | Y | Y | Y | U | U | N | N | Y | Described as a case study methodology; limited data presented |
Sleeman 2015143 | Y | Y | Y | U | Y | N | U | Y | Y | Provides a model to summarise the themes; clearly identifies contribution of the study findings |
Smith 2013144 | Y | Y | Y | U | Y | N | U | Y | Y | Evaluates the initiative and describes the implication of the findings |
Stuart 2014145 | Y | Y | Y | U | Y | Y | Y | Y | Y | Provides findings in context of other models and theories |
Syson 2010146 | Y | Y | Y | U | Y | N | U | Y | Y | Brief details regarding methods for this pilot study but clear presentation of data; very brief consideration of other literature or implications |
Thiel 2013147 | Y | Y | Y | Y | Y | U | N | Y | Y | No comments |
Tucker 2013148 | Y | Y | Y | U | Y | N | U | Y | Y | Discusses implications fully |
Waterson 2012149 | Y | Y | Y | U | Y | N | U | U | Y | Brief paper that provides limited detail of methods; discusses study in context of future work needed |
Wilson 2007150 | N | Y | U | U | U | N | U | N | N | Few data provided and limited detail of methodology |
Wistow 2015151 | Y | Y | U | Y | N | U | N | N | Y | Limited presentation of data; predominantly description |
Appendix 3 Applicability reporting framework
Glossary
- Clinical Commissioning Groups
- Clinically led statutory NHS bodies that are responsible for the planning and commissioning of health-care services for their local area.
- Effect size
- A way of quantifying the difference between two groups by calculating the size of the difference. Customarily, an effect size of ≤ 0.2 is considered to be a small effect size, 0.5 is considered to be a medium effect size and ≥ 0.8 is considered to be a large effect size.
- Governance
- The system through which NHS organisations are accountable for continuously improving the quality of their services and safeguarding high standards of care.
- Integrated care pathway
- An outline of anticipated care provided by different professionals, placed in an appropriate time frame, to help a patient with a specific condition or set of symptoms move progressively through clinical care.
- Multidisciplinary team
- A group of health-care workers who are members of different disciplines and who each provide a specific service to a patient.
- p-value
- The probability value is customarily used to indicate whether or not research results are statistically significant. A p-value of < 0.05 means that there is a < 5% chance that the results of the study occurred by chance alone.
- Risk ratio
- The probability of an event taking place.
List of abbreviations
- A&E
- accident and emergency
- CCG
- Clinical Commissioning Group
- CI
- confidence interval
- COPD
- chronic obstructive pulmonary disease
- DES
- Directed Enhanced Service
- ED
- emergency department
- GP
- general practitioner
- ICP
- integrated care pathway
- IT
- information technology
- LES
- Local Enhanced Service
- MDT
- multidisciplinary team
- MeSH
- medical subject heading
- NIHR
- National Institute for Health Research
- PCT
- primary care trust
- POPP
- Partnership for Older People Project
- PRISMA
- Preferred Reporting Items for Systematic Reviews and Meta-Analyses
- RCT
- randomised controlled trial
- SD
- standard deviation