Project objectives

Aim

Explore whether the second translational gap in primary care could be reduced by targeting researchers in the process of developing and evaluating complex interventions to improve health and health care in a primary care context.

Specific objectives

1. To review and summarise the existing literature on the causes of, and effective methods of reducing, the second translational gap in primary care (WP1).
2. To consider appropriate theoretical frameworks for addressing this gap, and current empirical support for these theories (WP2).
3. To identify available toolkits aimed at improving implementation processes, and determine which of these has the strongest empirical and theoretical foundation (WP3).
4. To determine which toolkit is most acceptable to target users (health service and primary care researchers), and determine the feasibility and acceptability of using it (or an appropriately modified version) to optimise the design of complex interventions and / or trials evaluating complex interventions in primary care (WP4).
5. To explore the impact of the selected toolkit on development & design of complex interventions and proposed trial design and methods (WP5).

Changes to project objectives

None.

Brief summary

1. To review and summarise the existing literature on the causes of, and effective methods of reducing, the second translational gap in primary care (WP1).

Brief summary of methods

Review question 1: What are the causes of the second translational gap in primary care?

Review question 2: What are the effectiveness of implementation strategies/ methods of reducing the second translational gap in primary care?

Our systematic review of reviews aimed to identify the available literature relevant to the review questions and to extract and synthesise the evidence in a transparent and reproducible manner.

A comprehensive electronic search was carried out in MEDLINE, EMBASE, CINAHL, PsycINFO and the Cochrane Library, to identify English-language, peer reviewed evidence from inception to December 2013 relevant to the review questions. Citation searches were conducted and reference lists from reviews and relevant articles were screened for additional evidence. Search results (titles and abstracts as well as full text papers) were screened by two reviewers independently, according to predefined inclusion and exclusion criteria. Data extraction was undertaken using extraction forms developed iteratively and tested extensively for this review of reviews. For review question 1, data were synthesised using meta-ethnography techniques, based on an inductive and interpretive approach. Data extraction and synthesis were checked by co-investigators for a purposive sample of 25%. Formal quality assessment was not undertaken; instead a structured approach was used to describe the quality of included reviews (a number of key items which determined quality was identified). For review question 2, owing to the substantial literature relevant to the review topic, a systematic, transparent and rigorous method was developed and applied to allow more effective data management and synthesis, which included sorting papers into EPOC categories, the selection of a benchmark review for each category and the selection of important outcomes. Data on both dichotomous and continuous outcome measures were extracted. Due to the heterogeneity of the included reviews, a meta-analysis was not undertaken and instead, a narrative approach was applied. All benchmark review papers plus two randomly selected subsequent papers for each category was checked by the co-investigators. The PRISMA checklist was used to critically appraise the quality of reporting of the included benchmark reviews papers.

Key findings

Causes of the second translational gap

A total of 70 reviews were included, which encompassed a wide range of topics including guidelines implementation, technology implementation, new role integration, management of care, public health and preventive medicine and prescribing behaviour. All the included reviews described causes in terms of “barriers and facilitators” to implementation perceived by health professionals and staff. The causes of success or failure of complex interventions are complex and can be classified into a four-level model: external context, organisation, professionals and intervention. At the level of external context, we find policies, incentivisation structures, dominant paradigm, stakeholders buy-in, infrastructure, advances in technology and public awareness. Organisational level – culture (including leadership and organisational readiness), available resources, integration with existing processes, relationships, skill mix and involvement affect the likelihood of implementing complex interventions.  Professional level – we find professional role, underlying philosophy of care, attitudes and beliefs and other personal attributes, and competencies. Intervention level – characteristics (e.g. complexity) and implementability of the intervention. The “fit” between the intervention and the context appeared to be critical in determining the success of implementation.

Effective implementation strategies

A total of 91 reviews were included. Strategies that targeted at the professional level were commonly evaluated, including audit and feedback, educational outreach visits, educational meetings and workshops, reminders, local opinion leaders and distribution of printed educational materials. Overall, the use of these single strategies alone can achieve small to modest improvement (median absolute improvement 2-9%) in professional practice. The effects of local opinion leaders were mixed and using passive methods alone such as printed educational materials was not effective. We found relatively little evidence on the effects of implementation strategies which addressed the organisation-level and wider context-level barriers (with the majority of evidence focused on financial payments and structures, which showed positive but variable effects). The effects of multifaceted strategies were variable and they are not necessarily more effective than single strategies alone. A number of features that support and impede good implementation were identified and presented. There appears to be limited cost-effectiveness evidence on these implementation strategies.

2. To consider appropriate theoretical frameworks for addressing this gap, and current empirical support for these theories (WP2).

Brief summary of methods

A residential workshop was held to gather expert opinions from co-applicants.

Key findings

The recognition that using research evidence in clinical practice is not a straightforward, technical process has meant that theories are increasingly complex and multi-layered. The importance of taking a longitudinal perspective and considering the question of routinisation means that they also need to be informed by theories of change. This may lead to them being seen as too difficult to apply, hence the emergence of toolkits that are theoretically informed.

3. To identify available toolkits aimed at improving implementation processes, and determine which of these has the strongest empirical and theoretical foundation (WP3).

Brief summary of methods

A list of available toolkits was created using multiple sources of evidence: 1) toolkits identified from the WP1 review of reviews; 2) published systematic review of toolkits and 3) expert knowledge. A focus systematic research was conducted for all studies evaluating the toolkits identified in MEDLINE, EMBASE and CINAHL. Data were then extracted and summarised. A two-stage shortlisting process was undertaken to select appropriate toolkit(s) for testing in WP4 and 5; stage 1: initial shortlisting by using pre-specified criteria and stage 2: refinement via a two-day expert consensus workshop.

Key findings

Twenty-five toolkits were identified, of which 20 were excluded during initial shortlisting for reasons such as not a toolkit, too specific, not accessible, not easy to use. Five remaining toolkits were worked through and discussed during the consensus workshop. Following the consensus process, three toolkits were selected including the Readiness for Implementation model (RIM), Organizational Change Manager (OCM) and Normalization Process Theory (NPT).

4. To determine which toolkit is most acceptable to target users (health service and primary care researchers), and determine the feasibility and acceptability of using it (or an appropriately modified version) to optimise the design of complex interventions and / or trials evaluating complex interventions in primary care (WP4).

Brief summary of methods

Think aloud observation methods and in-depth interviews were carried out with research teams that implement complex interventions, identified through the contacts of project steering group members and their professional networks, including clinical trial unit, local Primary Care Research Network and Collaborations for Leadership in Applied Health Research and Care (CLAHRC). Research terms were asked to apply the selected toolkits to a project they were working on. Thematic analysis and constant comparison technique were used for data synthesis.

Key findings

Four trial teams were recruited to participate in the study. This study focused on four areas: accessibility, format/ layout, ease of use and salience of the questions/ constructs. The participants thought the use of a toolkit was a useful accompaniment to trial design and conduct; they were concerned about the questions focusing on modifying an intervention to fit existing organisational working practices, which was against the core principles of trial methodology. The NPT, on the whole, thought to be quicker and easier to use with salient questions/ constructs. Thus, it was taken forward to be tested in WP5.

5. To explore the impact of the selected toolkit on development & design of complex interventions and proposed trial design and methods (WP5).

Brief summary of methods

Qualitative methods including observational methods, semi-structured interviews and documentary analysis were used. A purposive sample of research teams were recruited through the professional networks and personal contacts of steering group members, similar to WP4. The analysis of the qualitative case study data (observation field notes, interviews and documents) was iterative using thematic analysis. Interviews were analysed using inductive coding, subsequent theme development and constant comparison.

Key findings

Three trial teams were recruited to participate in the study. Three main themes were identified: engagement and enrolment (linked to recruitment), using the toolkit and salience of the toolkit. Concerns were raised about the potential impact on workload, changing the fidelity of their intervention, and appropriate timing for using a toolkit. As the development of the NPT toolkit is grounded in sociological theory, some participants experienced some difficulties initially to get to grips with the complexity of the language used; this could be overcome by the presence of a NPT “champion” who can explain the toolkit and promote its use within research teams. Once the participants had become familiar with the toolkit, they found it useful as a heuristic tool - helped the development of their intervention, and to work out whether their intervention would work in practice; and they liked the flexibility of the toolkit, i.e. it can be used at different time points as a strategic tool.

Conclusions

Our work emphasises the importance of understanding the context in which an implementation project is to take place and paying attention to the “fit” between the proposed complex intervention and major contextual drivers. In a complex environment such as primary care, it is also important to acknowledge that each primary care organisation (e.g. general practice, CCG) will have specific features which will impede or promote implementation of innovative practices.  Identifying (and addressing) these organisational features may be as important as providing change management strategies aimed at individuals.  Use of a toolkit, such as the NPT toolkit, may help with planning and implementing complex interventions, although further work to improve the usability of this toolkit is required.

Plain English summary

Getting results of research implemented or applied in routine practice is challenging in health care. Much time and resources have been spent to develop and evaluate new ways of delivering health care, despite being shown to be beneficial, are never implemented or applied in the NHS. This problem is often referred to as the evidence to practice gap or the second translational gap. To apply something new in this particular setting is challenging because 1) the NHS is a highly complex organisation with different systems and ways of working; and 2) these new ways of providing health care are usually complex interventions, i.e. consist of multiple components.

This project addresses this problem by carrying out a ‘systematic review of reviews’ which involved us looking at the research that has been done in relation to 1) what causes the evidence-practice gap and 2) what can be done to improve the use of these complex interventions (i.e. implementation strategies). We found that success of implementation of any kinds of complex interventions is dependent upon how well the intervention fits with a) the wider context in which implementation occurs, b) organisational structures/features and c) characteristics of health professionals involved. Context is important when deciding whether to introduce a change in practice. Implementation strategies targeted at professionals, e.g. educational outreach visits, audit & feedback, reminders, educational meetings can achieve small-modest improvement in professional practice. Relatively little research was done to examine strategies targeting at the organisation (e.g. strategies to change organisational culture) and the wider context (financing strategies showed mixed results). It is unclear how cost-effective these strategies are. Use of combination strategies (e.g. educational meeting plus reminders) is not necessarily better than single strategies alone.

We also explored the potential of helping researchers consider issues related to implementation from the beginning of their research by using a toolkit - Normalization Process Theory toolkit, with the goal of ensuring research is relevant to the NHS and has greater potential to benefit patients. We found that the toolkit may enable researchers to think through likely problems during the design phase of their research – either for designing intervention or research procedures.

More research is needed to examine 1) the effects of strategies that operate at organisational and wider context levels and 2) whether a toolkit has any impact on the design and development of complex interventions, as well as later stages of the research process.

Dissemination

Several outputs will be expected, including a detailed full project report, an executive summary report for distribution in a number of events/workshops, and peer-reviewed journal publications. 

Published articles

1. Lau R et al. Addressing the evidence to practice gap for complex interventions in primary care: a systematic review of reviews protocol. BMJ Open. 2014 June 23;4(6):e005548.
   http://bmjopen.bmj.com/content/4/6/e005548.full

Planned articles

1. Conference abstract published in a supplement issue of Implementation Science –May 2015.
2. Achieving change in primary care (1): perceptions of facilitators and barriers to implementing complex interventions – systematic review of reviews.
3. Achieving change in primary care (2): effectiveness of strategies for improving implementation of complex interventions – systematic review of reviews.

Public involvement

The key aim of this project grant is to improve the implementation/use of complex interventions in primary care at the level of organisation and health care professionals and not directly related to patients. At the beginning of the project, two PPIs were invited to our steering group meeting but they felt that the project was too technical and scientific. Therefore, we did not feel it was relevant or appropriate to involve PPIs.

Impact

An all-day project dissemination event was held at the King’s Fund in October 2014, in which clinicians, health services/commissioning managers, policy makers and academics were invited to attend. Professor David Haslam, Chair of NICE and Professional Carl May from University of Southampton were invited discussants for the morning session. Social media (i.e. twitter) was also used throughout the day.

The work had been (or will be) presented at a number of events and national/International conferences, including:

• NSPCR showcase conference, Oxford – September 2014;
• 7th Annual Conference on the Science of Dissemination and Implementation, North Bethesda, Washington, USA – December 2014;
• CLAHRC North Thames Quality Improvement in Primary Care Workshop, UCL – January 2015;
• Workshop on effective leadership in ICT innovation for healthcare, Royal Holloway University – March 2015;
• Global Implementation Conference, Dublin – May 2015.

This project was funded by the National Institute for Health Research School for Primary Care Research (project number 122)

Department of Health Disclaimer

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR School for Primary Care Research, NIHR, NHS or the Department of Health.