Project objectives

The original objectives of this work were to:

1. To re-examine the PINCER trial outcome measures, and prescribing safety indicators developed for the RCGP, and select those most suitable for inclusion as secondary outcome measures in a Phase IV trial;
2. To identify the computer codes and develop the computer queries needed to extract data on each of the selected prescribing safety indicators for a range of GP clinical software systems;
3. Using the QRESEARCH database, to describe secular trends in rates of hazardous prescribing and potentially inadequate blood-test monitoring for these indicators across a large number of practices, along with intra-cluster correlation coefficients to help undertake sample size calculations;
4. To develop further and test methods of extracting pseudo-anonymised data on patients at risk from medication errors that will minimise the input needed from researchers and allow for efficient data extraction for a Phase IV trial;
5. To review the qualitative data from the PINCER trial in order to help refine the intervention;
6. To pilot any changes to the PINCER trial intervention;
7. To bid for a major national Phase IV trial.

Changes to project objectives

The only change to the project is in relation to objective 3 above. As part of the original PINCER trial, we described secular trends in rates of hazardous prescribing and potentially inadequate blood-test monitoring for the PINCER indicators across a large number of practices over the same time period as the trial. We had originally planned to do a similar study as part of this work to allow us to calculate intra-cluster correlation coefficients to help undertake sample size calculations. However, due to the fact that our colleagues from the Medication Safety theme of the Greater Manchester Primary Care Patient Safety Translational Research Centre (GMPSTRC) have a great deal of expertise in undertaking epidemiological work of this type using CPRD (and have ready access to CPRD data), we decided to undertake this work using CPRD data, rather than QResearch data.

Brief summary

By identifying interventions aimed at reducing the prevalence of prescribing errors, this research has potential to improve the quality of care for patients, prevent medication-related harm, and improve the cost-effectiveness of care. This project has laid the foundations for bidding for a major Phase IV trial using methods developed in the PINCER trial. The project addresses an important clinical problem and has had high profile, high quality, outputs.

Having developed an effective complex intervention for improving prescribing safety in a large cluster randomised trial, and having a new set of prescribing safety indicators that are likely to have the support of the GP profession, this funding has enabled us to undertake further studies to evaluate the new prescribing safety indicators. An important part of this work has involved describing rates of potentially hazardous prescribing and inadequate blood-test monitoring as this has been of considerable value in helping to decide which of the indicators are most appropriate for intervention studies and for comparing the safety of prescribing between general practices. In addition, we have built on the innovative methods we developed for extracting pseudo-anonymised data on patients at risk from medication errors.

Methods and findings against objectives

1. The MIQUEST computer queries used in the PINCER trial have been recreated with support from PRIMIS (Primary Care Information Services) (www.primis.nottingham.ac.uk). The queries have been made available from the PRIMIS website from 13 February 2013 so that they can be downloaded by general practices across England and run on their computer systems. Using CHART (Care and Health Analysis in Real Time) analysis tool, GP practices can view the results of the searches in a user-friendly format to identify patients within their practice potentially at risk of prescribing error. By uploading search results from individual GP practices to CHART online, it is now possible for CCGs to view summary data from the GP practices within their cluster.
   We have re-examined the prescribing safety indicators developed for the RCGP along with the evidence-based summaries developed for each of the indicators. Using the Reference Group set up for the RCGP indicators project we selected 15 indicators on the basis of the following factors: strength of support from the RCGP consensus group; importance of the indicators in terms of evidence of patient harm; and likelihood that the indicators can be turned into effective computer queries for routine use.
2. We have worked withPRIMIS to identify the computer codes and developed the computer queries needed to extract data on each of the selected new prescribing safety indicators developed for theRCGP. We have done this for all GP clinical software systems. We have also:
   • Explored different ways of presenting summary data for CCG views;
   • Investigated more efficient ways of extracting pseudo-anonymised data on patients at risk from medication errors.
3. We have undertaken the preparatory work needed to describe secular trends in rates of hazardous prescribing and potentially inadequate blood-test monitoring for these indicators across a large number of practices contributing to the Clinical Practice Research Datalink, along with intra-cluster correlation coefficients to help undertake sample size calculations. We have also undertaken a series of cohort studies in collaboration with colleagues in Epidemiology and Public Health to determine the risks of upper gastrointestinal bleeding while being prescribed NSAIDs or aspirin.
4. We have worked closely with PRIMIS to pilot the acceptability, technical feasibility, reliability, and validity of 15 new prescribing safety indicators (PINCER+ Query Library) and CHART Online views in 15 general practices in one Clinical Commissioning Group (CCG). As part of the Patient Safety Toolkit (PST) Study, we have liaised with colleagues from the University of Nottingham, PRIMIS and other research centres to facilitate the implementation of the prescribing safety indicators in stage 3 of the PST project.
5. In addition to written feedback from the pharmacists involved in the study, and findings from qualitative work, we have conducted a significant amount of work to refine the intervention including: 
   • Conducting reviews of the literature and liaising with experts in the field to identify which indicators represent important safety problems for general practitioners;
   • Working with colleagues at the University of Manchester to apply the prescribing safety indicators in the Clinical Practice Research Datalink (CPRD) to provide estimates of the number of patients we would expect to identify in an ‘average’ practice of 6,000 patients;
   • Writing new computer queries to identify patients at risk of hazardous prescribing according to the prescribing safety indicators.
   The final 11 prescribing safety indicators to be used in the PINCER roll out across the East Midlands relate to the following serious harm outcomes:
   • Gastrointestinal (GI) bleeding;
   • Acute exacerbation of asthma;
   • Thromboembolism;
   • Stroke;
   • Heart failure;
   • Acute kidney injury;
   • A composite of the above outcomes.
   We have also modified the feedback to general practices and CCGs by creating time trend views of the results in CHART Online and functionality to export results to printer-friendly PDF views and Microsoft Excel software.
6. In addition to piloting the acceptability, technical feasibility, reliability, and validity of 15 new prescribing safety indicators (PINCER+ Query Library) and CHART Online views in 15 general practices in one Clinical Commissioning Group (CCG), we have also piloted the final set of prescribing safety indicators in 6 general practices in one Clinical Commissioning Group to ensure that the indicators perform as intended and that our data collection methods are robust.
7. The project has addressed an important clinical problem and has had a number of high profile, high quality outputs. The project has increased the evidence base for primary care practice by:
   • Developing a set of prescribing safety indicators to improve safety of prescribing;
   • Applying these prescribing safety indicators to the Clinical Practice Research Datalink to examine variations in prescribing safety in UK general practice;
   • Undertaking a series of cohort studies in collaboration with colleagues in Epidemiology and Public Health to determine the risks of upper gastrointestinal bleeding while being prescribed NSAIDs or aspirin.

We have now used the information gathered from 1-7 above, along with outputs from the original PINCER trial, to apply for an NIHR Programme Grant for Applied Research:

Avery AJ, Rodgers S et al. Avoiding patient harm through the application of prescribing safety indicators in English general practices (acronym: PRoTeCT). Second stage application submitted to NIHR Programme Grant for Applied Research, October 2015. Amount £2,430,144.

Plain English summary

Medicines prescribed by doctors can sometimes cause harm to patients, including unnecessary admissions to hospital. The main aim of our research is to find effective and affordable ways of supporting doctors working in general practices to prescribe medicines as safely as possible.

One approach to doing this is to search general practice computer records to find patients who have already received medicines which might harm them. A previous study, called PINCER, showed that doctors in general practices using this approach, along with the support of a pharmacist to correct any problems found, can reduce the number of patients put at risk of harm. However we do not know if this approach reduces the numbers of patients seriously harmed, or how well it works when rolled out to large numbers of general practices.

In order to answer these questions, and estimate the likely costs to the NHS, we need to apply for further funding to carry out a much bigger research study. The funding that we have received from the NIHR School for Primary Care Research has allowed us to do the necessary work to apply for an NIHR programme grant which we submitted in October 2015. This has included:

• Undertaking studies using large clinical databases to find out the scale of the problem;
• Improving the searches used in the PINCER trial to identify patients at risk of medication error from general practice computer systems;
• Making these searches available free of charge to all general practices in England;
• Developing new searches to identify “at risk” patients;
• Testing these new searches in volunteer general practices;
• Developing training materials for primary care pharmacists so that they can support general practices to improve patient safety by correcting any errors found;
• Talking to health care professionals (including doctors and pharmacists), patients and members of the public to help us to understand how well the PINCER approach works and how it might be improved and used more widely across the NHS in the future.

Patients and members of the public have been involved in designing this project and have been actively involved throughout by:

• Attending project meetings;
• Helping analyse data;
• Taking part in workshops;
• Communicating the research findings to those people who have taken part, and the wider public.

We have published our findings in journals for researchers and healthcare professionals. It is vitally important that patients have this information too and so our results have been sent to public bodies and organisations/charities concerned with patient safety.

Dissemination

Published articles

1. Stocks SJ, Kontopantelis E, Akbarov A, Rodgers S, Avery AJ, Ashcroft DM. Examining variations in prescribing safety in UK general practice: a cross-sectional study using the Clinical Practice Research Datalink. British Medical Journal 2015;351:h5501.
   http://www.bmj.com/content/351/bmj.h5501
2. Akbarov A, Kontopantelis E, Sperrin M, Stocks SJ, Williams R, Rodgers S, Avery A, Buchan I,  Ashcroft DM. Primary care medication safety surveillance with integrated primary and secondary care electronic health records: a cross-sectional study. Pharmacoepidemiology & Drug Safety 2015;38(7):671-82 DOI:10.1007/s40264-015-0304-x.
   http://www.ncbi.nlm.nih.gov/pubmed/26100143
3. Elliott RA, Putman KD, Franklin M, Annemans L, Verhaeghe N, Eden M, Hayre J, Rodgers S, Sheikh A, Avery AJ (on behalf of the PINCER Team). Cost-effectiveness of a pharmacist-led information technology intervention for reducing rates of clinically important errors in medicines management in general practices (PINCER). PharmacoEconomics 2014;32(6):573-590 doi: 10.1007/s40273-014-0148-8.
   http://www.ncbi.nlm.nih.gov/pubmed/24639038
4. Avery AJ, Rodgers S, Franklin BD, Elliott RA, Howard R, Slight SP, Swanwick G, Knox R, Gookey G, Barber N, Sheikh A. Research into practice: safe prescribing. British Journal of General Practice 2014;64(622):259-61 doi: 10.3399/bjgp14X679895.
   http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4001139/
5. Spencer R, Bell B, Avery AJ, Gookey G, Campbell SM. Identification of an updated set of prescribing safety indicators for GPs British Journal of General Practice 2014;64 (621);e181-e190.
   http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3964450/
6. Sadler S, Rodgers S, Howard R, Morris CJ, Avery AJ (on behalf of the PINCER triallists). Training pharmacists to deliver a complex information technology intervention (PINCER) using the principles of educational outreach and root cause analysis. International Journal of Pharmacy Practice 2014;22:47-58.
   http://www.ncbi.nlm.nih.gov/pubmed/23600928
7. Howard R, Rodgers S, Avery AJ, Sheikh S (on behalf of the PINCER triallists). Description and process evaluation of pharmacists’ interventions in a pharmacist-led information technology-enabled multicentre cluster randomised controlled trial for reducing medication errors in general practice (PINCER trial). International Journal of Pharmacy Practice 2014;22:59-68.
   http://www.ncbi.nlm.nih.gov/pubmed/23718905
8. Cresswell KM, Sadler S, Rodgers S, Avery A, Cantrill J, Murray SA, Sheikh A. An embedded longitudinal multi-faceted qualitative evaluation of a complex cluster randomised controlled trial aiming to reduce clinically important errors in medicines management in general practice. Trials 2012;13:78.
   http://www.ncbi.nlm.nih.gov/pubmed/22682095
9. Avery A, Rodgers S, Cantrill J, Armstrong S, Cresswell K., Eden M, Elliott RA, Howard R, Kendrick D, Morris CJ, Prescott RJ, Swanwick G, Franklin M, Putman K, Boyd  M, Sheikh A. A pharmacist-led information technology intervention for medication errors (Pincer): a multicentre, cluster randomised, controlled trial and cost-effectiveness analysis. The Lancet 2012;379:1310-1319.
   http://www.ncbi.nlm.nih.gov/pubmed/22357106

Commissioned articles

1. Hobbs FDR, Taylor CJ. Academic primary care: at a tipping point? British Journal of General Practice 2014;64: 214-215.
   http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4001133/
2. Rodgers S. New PINCER Query Library tool to support safer prescribing. Prescriber 2013;24(6):11-14.
   http://www.prescriber.co.uk/wp-content/uploads/sites/23/2015/10/New-PINCER-Query-Library-tool-to-support-safer-prescribing.pdf
3. Rodgers S. Five steps to reducing prescribing errors using PINCER. PULSE practice topics, 12 February 2013.
   http://www.pulsetoday.co.uk/your-practice/practice-topics/it/-five-steps-to-reducing-prescribing-errors-using-pincer/20001835.article
4. Rodgers S. Tackling patient safety: is there a pharmacist in the house? (Editorial) Prescriber 2012;23(7):7-8.
   http://onlinelibrary.wiley.com/doi/10.1002/psb.888/pdf
5. Avery T, Rodgers S. How we reduced prescribing errors with pharmacists’ support. PULSE Learning CPD Module (1 hour CPD) 23 May 2012.
   http://www.pulse-learning.co.uk/commissioning-modules/commissioning/how-we-reduced-prescribing-errors-with-pharmacists-support

Published peer reviewed conference abstracts

1. Jeffries M, Phipps D, Howard R, Avery A, Rodgers S, Ashcroft DM. Medication safety implications of a technological intervention in primary care: a realist evaluation of eclipse live. Pharmacoepidemiology and Drug Safety 2015;DOI:10.1002/pds.

Reports to funding bodies

1. Elliott RA, Putman K, Franklin M, VerhaegheN, Annemans L, Eden M, Hayre J, Rodgers S, Cantrill JA, Armstrong S, Cresswell K, Hippisley-Cox J, Howard R, Kendrick D, Morris CJ, Murray SA, Prescott RJ, Swanwick G, Boyd M, Tuersley L, Turner T, Vinogradova Y, Sheikh A, Avery AJ. Economic evaluation of a pharmacist-led IT-based intervention with simple feedback in reducing rates of clinically important errors in medicines management in general practices (PINCER). Report to the Department of Health National Patient Safety Research Portfolio, February 2013.
   http://www.birmingham.ac.uk/Documents/college-mds/haps/projects/cfhep/psrp/finalreports/PINCER-economics-report.pdf

Peer reviewed conference presentations

1. Giles S, Rodgers S, Morris R, Campbell S, Boote J, Beresford J. Moving forwards: sharing and learning from contrasting methods of patient and public involvement (PPI) in the primary care setting. Workshop, 43rd North American Primary Care Research Group Annual Meeting, 24-28 October 2015, Cancun, Mexico.
2. Rodgers S, Avery AJ. Development of prescribing safety indicators for inclusion in a patient safety toolkit for use in general practices. Poster presentation, NIHR School for Primary Care Research Showcase Event, 21 September 2015, Oxford.
3. Stocks SJ, Kontopantelis E, Akbarov A, Rodgers S, Anthony AJ, Ashcroft D. Examining variations in prescribing safety in UK general practice: a cross-sectional study using the Clinical Practice Research Datalink. Oral presentation, Society for Social Medicine 59th Annual Scientific Meeting, 2 September 2015, Dublin.
4. Jeffries M, Phipps D, Howard R, Avery A, Rodgers S, Ashcroft DM. Understanding the Medication Safety Implications of a Technological Intervention in Primary Care: A Realist Evaluation of Eclipse Live. Poster presentation, Health Services Research Network Symposium, July 2015, Nottingham.
5. Bell B, Marsden L, Avery A, Rodgers S, Spencer R, Campbell S, Ricci-Cabello I, Valderas-Martinez J. The development of the Patient Safety Toolkit for general practices in England. Poster presentation, 44th Annual Scientific Meeting of the Society of Academic Primary Care, 8-10 July 2015, Oxford.
6. Jeffries M, Phipps D, Howard R, Avery A, Rodgers S, Ashcroft DM. Medication safety implications of a technological intervention in primary care: a realist evaluation of eclipse live. Poster presentation, 26th Prescribing and Research in Medicines Management (PRIMM) Annual Scientific Meeting, 23 January 2015, London.
7. Rodgers S, Avery AJ. Development of prescribing safety indicators and data extraction methods in UK general practice. Poster presentation, 42nd North American Primary Care Research Group Annual Meeting, 21-25 November 2014, New York.
8. Rodgers S, Avery AJ. Development of prescribing safety indicators and data extraction methods in UK general practice. Poster presentation, NIHR School for Primary Care Research Showcase Event, 26 September 2014, Oxford.
9. Stocks, SJ, Kontopantelis E, Rodgers S, Avery AJ, Ashcroft D. Application of Prescribing Safety Indicators to the Clinical Practice Research Database: a retrospective cohort study. Oral presentation, 43rd Annual Scientific Meeting of the Society of Academic Primary Care, 9-11 July 2014, Edinburgh.
10. Rodgers S, Avery AJ. Development of prescribing safety indicators and data extraction methods in UK general practice. Poster presentation, 42nd North American Primary Care Research Group Annual Meeting, 21-25 November 2014, New York.
11. Rodgers S, Avery AJ. Development of prescribing safety indicators and data extraction methods in UK general practice. Poster presentation, NIHR School for Primary Care Research Showcase Event, 26 September 2014, Oxford.
12. Stocks, SJ, Kontopantelis E, Rodgers S, Avery AJ, Ashcroft D. Application of Prescribing Safety Indicators to the Clinical Practice Research Database: a retrospective cohort study. Oral presentation, 43rd Annual Scientific Meeting of the Society of Academic Primary Care, 9-11 July 2014, Edinburgh.
13. Rodgers S, Avery AJ, Evans D, Morrell T and the PRIMIS Training and Informatics Teams. Development of prescribing safety indicators and data extraction methods in UK general practice. Poster presentation, 42nd Annual Scientific Meeting of the Society of Academic Primary Care, 3-5 July 2013, Nottingham.

Articles submitted for publication

1. Jeffries M, Phipps DL, Howard RL, Avery A, Rodgers S, Ashcroft DM. Understanding the medication safety implications of a technological intervention in primary care: a realist evaluation of Eclipse Live. Submitted to PLOS One, October 2015.

Articles in preparation

1. Rodgers S, Vinogradova Y, Avery AJ, Sheikh A (on behalf of the PINCER Team). Interpreting findings from the PINCER trial: QRESEARCH analysis to estimate changes in rates of patients at risk of prescribing and monitoring errors in general practices in England.
2. Bell B, Avery AJ, Rodgers S et al. The Development and testing of the NIHR-SPCR Patient Safety Toolkit for general practices in England.

Public involvement

We have continued to work with Antony Chuter and Glen Swanwick, our PPI representatives, on the outputs for this study. They continue to provide lay input into grant applications, new research studies and publications. Antony supports the research by facilitating PPI focus groups and both Antony and Glen attend regular meetings.

Chris Rye (and colleagues from the PPI Senate of the East Midlands Academic Health Science Network) has had a major role in our successful bid to the Health Foundation to provide the funding for a large-scale rollout of the PINCER intervention across the East Midlands.

In addition to the above, in order to increase our engagement with patients and the public, we have:

• Presented our work to the East Midlands Academic Health Sciences Network PPI Senate;
  http://emahsn.org.uk/how-to-guidance/em-ppi-senate
• Presented at one of the Research Engaging Patients and the Public (REPP) Events and continue to work closely with Shahnaz Aziz, EMAHSN Patient and Public Lead;
• Developed strong links with members of the Research Users Group (RUG) from the GMPSTRC and have worked closely with the former RUG Chair, Ailsa Donnelly on developing grant applications;
• Attended the 43rd North American Primary Care Research Group Conference, October 2015 to deliver a workshop with colleagues from the GMPSTRC on sharing and learning from contrasting methods of patient and public involvement (PPI) in the primary care setting.

Impact

Examples of impact are listed below. In addition there has been considerable media interest in our work, including our recent BMJ publication and Tony Avery has done a number of media interviews including one with BBC Radio Nottingham:

• Patient Safety Toolkit (including PINCER prescribing safety indicators) launched on RCGP website;
  http://www.rcgp.org.uk/clinical-and-research/toolkits/patient-safety.aspx
• Inclusion of the PINCER intervention in the National Institute for Health Care Excellent ‘Medicines Optimisation Clinical Guideline’ published 04/03/15;
  http://www.nice.org.uk/guidance/ng5/evidence
• Inclusion of the PINCER prescribing safety indicators in the NHS England Medicines Optimisation Dashboard to improve prescribing safety (2014/5);
  https://www.england.nhs.uk/ourwork/pe/mo-dash/
• Inclusion of the PINCER prescribing safety indicators in First Databank’s medicines optimisation software ‘Optimise Rx’ and ECLIPSE Live software to improve patient safety at the point of care;
• New series of e-learning materials developed and made available through the Royal College of General Practitioners;
  http://www.rcgp.org.uk/learning/online-learning/ole/prescribing-in-general-practice.aspx
• Release of the PINCER Query Library in February (taken up by almost 2,000 individual general practices across 196 CCGs) to identify at-risk patients who have been prescribed drugs commonly associated with medication errors;
  https://www.nottingham.ac.uk/primis/about/news/newslisting/pincer-launched-feb13.aspx
• Piloting of the PINCER and PINCER+ Query Libraries in 20 general practices in Northern Ireland with potential to rollout to all GP practices in Northern Ireland;
• Rollout of the PINCER intervention to 70 general practices in South Derbyshire and Erewash CCGs.

In addition, this work has given us the opportunity to apply for further research funding:

• Siriwardena N, Avery AJ, Rodgers S et al. Improving prescribing safety in general practices in the East Midlands through the PINCER intervention. Health Foundation Scaling Up Improvement, December 2014. Amount awarded £500,000 (with a further £250,000 pledged by East Midlands Academic Health Science Network to cover the costs of rolling out the PINCER intervention to all general practices in the East Midlands);
• Avery AJ, Carson-Stevens A, Edwards A, Rodgers S et al. Understanding the nature and frequency of avoidable harm in primary care. Department of Health Policy Research Programme, March 2015. Amount awarded £349,793;
• Rodgers S, Avery AJ, Ashcroft D, Bell B, Salema N, Phipps D, Swanwick G, Chuter A. Feasibility pilot study to collect linked primary and secondary care electronic health records for use in a future Phase IV Trial. NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre. March 2014. Amount awarded £20,000;
• Rodgers S, Avery AJ, Silcock N, Bassi M, Bell B, Salema N, Elliott R, Ashcroft D, Sheikh A, Swanwick G, Chuter A. Preparing for a Phase IV implementation trial using PINCER methodology aimed at reducing the incidence of serious hospital admissions. NIHR Research Capability Funding to develop an NIHR Programme Grant for Applied Research, January 2014. Amount awarded £49,878;
• Phipps D, Ashcroft D, Rodgers S, Avery A. Assessing the ability of a technology-based intervention to improve medication safety in primary care. NIHR Research Capability Funding Round 1, July 2013. Amount awarded £41,883;
• Rodgers S, Avery AJ, Elliott R, Bell B, Franklin M. Modelling the cost effectiveness of prescribing safety indicators to identify those that are likely to be most cost-effective for inclusion in a rollout of the PINCER trial intervention. NIHR SPCR Round 7, June 2013. Amount awarded £29,973;
• Avery AJ, Knox, R, Gookey G,  Salema, N, Marsden K, Bell, B, Rodgers S, Ashcroft D, Phipps D. Evaluation of prescribing safety e-learning materials for GPs (RCGP e-learning materials project). Greater Manchester Primary Care Patient Safety Translational Research Centre, April 2013. Amount awarded £27,189;
• Avery AJ, Silcock N, Bassi M, Marsden K, Bell B, Salema N, Rodgers S, Knox R. GP Registrars and safety of prescribing in primary care (the “first 100 prescriptions” pilot project). NHS Nottingham City Research Capability Funding, December 2012. Amount awarded £34,630.

This project was funded by the National Institute for Health Research School for Primary Care Research (project number 94)

Department of Health Disclaimer

The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the NIHR School for Primary Care Research, NIHR, NHS or the Department of Health.