Measuring quality in community nursing: a mixed-methods study

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 12/209/02. The contractual start date was in April 2014. The final report began editorial review in January 2017 and was accepted for publication in June 2017. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Lizanne Harland reports personal fees from Bristol Clinical Commissioning Group (CCG) and Gloucestershire CCG outside the submitted work. Lesley Wye is a member of the Health Services and Delivery Research (HSDR) Prioritisation Commissioning Panel. Chris Salisbury is a board member of the National Institute for Health Research (NIHR) HSDR programme, which funded this project. He is also a board member of the NIHR School for Primary Care Research.

Copyright statement

© Queen’s Printer and Controller of HMSO 2018. This work was produced by Horrocks et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2018 Queen’s Printer and Controller of HMSO

Introduction

Public perceptions and trust in the medical and nursing profession were shaken by the Bristol Royal Infirmary inquiry in 20011 and the failings in care at the Mid Staffordshire NHS Foundation Trust,2 both of which related to care quality in hospitals. NHS England (NHSE)3 responded with The 6Cs and Compassion in Practice, a paper characterising provision of high-quality nursing under the categories of the six ‘Cs’ – ‘care’, ‘compassion’, ‘courage’, ‘communication’, ‘commitment’ and ‘competence’– that could be applied to nursing in any health-care setting, and that has since been rolled out to include all practitioners in the NHS. The importance of having well-educated, graduate practitioners providing leadership at the front line in the nursing workforce to ensure the care and safety of patients has also been confirmed. 4 Since 2016, Nursing and Midwifery Council (NMC) revalidation has been introduced for registered practitioners to ensure that adherence to the code of professional standards for nurses in care delivery is being upheld. 5 However, the setting and monitoring of care standards is somewhat easier in the acute sector than in the community, and it is notable that the majority of work in relation to assessing nursing care quality has been undertaken in a hospital setting. Given the comparative invisibility of care delivery in patients’ homes, it is crucial that care quality can be monitored to protect patients.

Nurses constitute the largest service group within community-based health provision and work in a range of roles supporting patients in their homes to prevent unnecessary hospital admission and facilitate timely discharge. They may work in specialist roles, such as palliative care or heart failure, or as part of nursing teams (e.g. district nurses and community matrons), providing a full range of nursing care to people who are housebound, therefore serving the most vulnerable and frail NHS users. 6 Care provided in the home is largely invisible to planners and managers and, therefore, it is crucial to understand how quality measures for community nursing are identified and their usefulness for assuring service quality. 7,8 Although an understanding of how to measure the quality of services is growing in the hospital and primary care sectors, comparatively little is known about how best to measure service quality in the community nursing sector,9 or how patients perceive the value of current quality indicators.

The setting and measurement of quality standards have been formally incorporated into community health service commissioning decisions since 2009 and a database of quality indicators (QIs) has been published for use in community services. 10 The use of standardised QIs across community services should, in theory, enable similar services to be compared. Despite substantial investment in QI schemes, however, little is known about how quality measures are used in practice and their usefulness in improving and monitoring quality in nursing. A recent report by The King’s Fund found that community care providers still lacked ‘robust, comparable national indicators that would enable them to benchmark their performance’ and that information technology (IT) and infrastructure were underdeveloped to support quality measurement. 11 The report covered the full range of community services; however, many community services actually take place in clinic and health centre settings and provide time-limited episodes of care for which outcome measures may be readily identified. In contrast, quality measurement in community nursing is complicated by the fact that the majority of service users are cared for in the home and have complex and long-term health problems or deteriorating conditions, making suitable outcome measures difficult to design. 12,13

Quality indicators are measures that aim to numerically describe a service to enable comparison or to assess improvement. 10,14,15 They can come from multiple sources, including Transforming Community Services,10 Community Information Data Set16 and National Institute for Health and Care Excellence (NICE) quality standards,17 in addition to local derivation. 18 The concept of quality measurement in health care incorporates a range of different dimensions,19–21 and perspectives on quality measurement can vary between commissioners, providers and service users. 18 Donabedian’s health-care quality framework19 comprises three domains, namely structure (the organisational system or environment under which care is provided, e.g. training of staff, staffing levels), process (e.g. components of service provision) and outcome measurement (the consequences for health of the care delivered). There has been debate about the relative importance of process and outcome measures in quality assurance. 21 Although outcomes represent the ultimate goals of health care and are more easily measured and understood by service users, they can be affected by extraneous factors outside the control of the health service and may not be apparent at the time of care delivery, so are more difficult to attribute. For this reason they may be mistrusted by service providers. 7,18,22 Process indicators, in contrast, are likely to be within the scope of the health-care provider and less likely to be influenced by external factors, but are less easily measured or understood by those outside the service. 22

More recently, NHS quality of care assessment has reflected an emphasis on efficient use of resources and effectiveness, the latter reflecting both clinical outcomes and user experience. 21 This approach was incorporated and further developed within the current NHS quality domains, namely patient experience and clinical effectiveness, with the addition of patient safety. 23 At the time of the current study, these quality domains were incorporated into an annual, nationally mandated quality scheme called Commissioning for Quality and Innovation (CQUIN). The CQUIN scheme, included within the NHS standard contract, offers community services providers additional income to meet quality goals (e.g. improvements in end-of-life care planning) by measuring quality outcome indicators (such as the number of people with a record of death in a preferred place). 24 The intention was to allow commissioners to negotiate with health-care providers on developing services to meet local priorities, while operating within a broader national framework based on national and international best practice. The scheme has the potential to provide information for comparison purposes and incentivise innovative practice and higher-quality aspirations locally. However, a national evaluation of the scheme (which included both acute and community providers) found that few of its interviewees consulted national indicators when developing their CQUIN schemes and that the variety of local indicators that emerged were often unclear or lacking in precision, making it difficult to follow up performance or benchmark services. 25 Several recommendations were made, including reducing the number of local CQUINs schemes and the provision of a small number of benchmarking QIs with a menu of appropriate indicators for use in acute and community settings. Since the changes following the implementation of the Health and Social Care Act 2012,26 it is unclear how, if at all, use of the CQUIN scheme has changed under Clinical Commissioning Groups (CCGs) led by general practitioners (GPs) and how useful it is in the context of measuring quality in community nursing.

The delivery of NHS community nursing services is also subject to a number of quality requirements or key performance indicators (KPIs) incorporated within the standard NHS contract for monitoring and accountability purposes. 27,28 National quality requirements applicable for community nursing include ‘duty of candour’ and ‘never events’, that is, the need for staff to speak up if they are aware of ‘near miss’ incidents of potential harm to patients in order that learning can be gained to prevent similar harms from happening in future. A medication error involving insulin (much of which is administered by community nurses in patients’ homes) would be designated a ‘never event’, as its impact on the patient could be catastrophic. 29 The duty of candour states that staff should be honest and transparent in their communications with patients and their relatives when things go wrong. 30 The contract also includes locally agreed incentive schemes and requires reporting on service quality issues, such as complaints, serious incidents, staff numbers and skill mix, all of which can provide a regular snapshot of real-time feedback on service quality. In addition, providers are required to provide feedback from patients and staff on a regular basis.

Regular approval from the Care Quality Commission (CQC)31 is another requirement of community nursing service providers. CQC inspection overlaps with several areas incorporated in the NHS standard contract, but also includes quality standards related to ‘caring’. Caring is defined as the degree of dignity, respect and compassion while receiving care, people being involved as partners in their care and people who use the services receiving support to cope emotionally with their condition, none of which is directly monitored by the NHS contract. Another key difference relates to the approach to monitoring quality, which always involves an observation of practice and assessment of leadership, as well as an inspection of relevant documentation and talking directly to front-line staff and patients. Taken together, there appears to be a battery of quality assurance requirements on which community service providers report, suggesting a large workload to set, monitor and report on indicators of service quality, without evidence of their perceived usefulness for community nursing from the perspectives of commissioners or service providers and those in receipt of nursing care.

Quality indicator schemes have the potential to improve care quality by provision of feedback to clinicians to improve performance and reduce variation between services, yet there is a body of evidence that suggests that there can be unintended consequences of their use. 32 Unintended consequences documented in the literature included data collection using IT interfering with patient interactions and demotivation of clinicians when they felt their professional judgement being over-ruled. A study exploring primary care professionals’ experiences of the GP Quality and Outcomes Framework, a pay-for-performance quality scheme, suggested that the nature of professional and patient consultations had changed since its introduction, with patient centredness and continuity negatively affected. 33,34 Rambur et al. ,35 reporting on UK studies that included nurses, found evidence of ‘measure fixation’, that is, inappropriate attention on isolated aspects of care conflicting with patient-centred care. 36,37 Although the nurses referred to above tended to work in general practice or acute settings, little is known about the impact of incentivised QIs such as CQUINs on care delivered by community nurses. Furthermore, incentivised QIs for nursing that have been applied in acute care have been rolled out into the community, taking little account of the differing care context for nurses,38 and it is not clear how useful these are for assessing quality from the perspectives of commissioners, providers, patients or front-line community nursing staff.

In this report, findings are presented from a study comprising a collaboration between four universities (University of the West of England, Bristol; University of Bristol; London School of Hygiene and Tropical Medicine; and University of Manchester) and a NHS partner (the NHS Bristol CCG). Patient and public involvement in the study was integrated firmly at the outset by the inclusion of a lay co-applicant, who instigated the initial research question of how the use of performance indicators might affect nurses’ delivery of personalised care. The study aimed to understand how QIs were being selected and applied in the community nursing context, whether or not these were considered useful by patients, commissioners and providers, and how they could best be used to confer benefit to patients being cared for by community nurses.

Background

The recent background for community nursing provision has been one of continuing organisational change and financial pressure. The NHS is expected to make £22B in efficiency savings by 2020. 39 A key factor in meeting this target is to shift care from acute service provision to the community, placing increased pressure on community services to reduce avoidable hospital admissions and enable the timely discharge of patients. The following section presents the key themes from our review of the literature, which drew on both a structured search of research relating to quality and community nursing and the knowledge of relevant literature within the research management group (see Appendix 1 for the search strategy). It outlines recent policy in the arena of commissioning and community service provision, including the drive to integrate health and social care services, workforce issues and the care context for community nursing, against which the measurement of service quality takes place.

Summary

Despite recent progress in developing a framework for commissioning community nursing, there remain a number of practical challenges in measuring community nursing service quality. The impact of workforce issues and organisational changes such as IT implementation may not be fully understood in relation to community nursing. The nature of the work community nurses do, frequently over a lengthy time scale, makes finding suitable quality measures difficult. Moreover, it is not known how suitable or valid it is to use measures developed in acute settings for the purposes of assessing service quality in the community. There is a gap in the literature about the type of quality measures in use by commissioners and their effectiveness in really monitoring and improving the quality of community nursing care. Recent drives to encourage new forms of integrated organisations are likely to involve community nursing, and hence appropriate measures of quality will need to be determined.

In this study, we aimed to investigate how quality metrics are selected and agreed between commissioners and providers of community nursing services, as well as how these metrics are subsequently used in practice with the purpose of understanding how to improve them for the benefit of patients. We also focused on the perceived usefulness of current measures in improving quality of care provision, including patient and carer perspectives, and explored the challenges facing service providers in collecting information relating to quality of care.

Chapter 2 sets out the methodology of this three-phase, mixed-methods study. For clarity, this is immediately followed in Chapter 3 by the findings of the first phase, the national CQUIN survey. The following five chapters (see Chapters 4–8) report the case study findings. Chapter 9 draws together the findings from all of the data streams, which are then discussed in the final chapter (see Chapter 10) of the report.

Aims and objectives

The study aimed to investigate the selection, application and usefulness of quality measures in use for community nursing from April 2014 to June 2016 in order to identify how they are used and the factors that influence their usefulness in achieving their intended goals of ensuring high-quality care for patients.

The research questions developed for the study were:

• Which QIs are selected locally, regionally and nationally for community nursing?

• How are they selected and applied?

• What is their usefulness to patients, commissioners and community provider staff?

The associated study objectives were to:

1. map QIs in use for community nursing

2. identify the processes for the selection of QIs for community nursing at local, regional and national level

3. clarify the processes for introducing and applying QIs into community nursing services and to explore how data are collected, analysed and quality assured

4. explore the usefulness of indicators in terms of meeting priorities, assessing the quality of services, influencing commissioning decisions and bringing about changes in service delivery from the perspectives of patients, front-line teams and commissioners (Figure 1).

FIGURE 1.

Flow diagram of study methodology. a, Interviews will also explore selection; b, interviews will also explore selection and application.

A mixed-methods design in three phases was utilised, which comprised:

• phase 1 – a national cross-sectional survey of QI schemes

• phase 2 – an in-depth qualitative case study of the selection, application and usefulness of quality measures in use for community nursing in five study sites

• phase 3 – a series of stakeholder engagement workshops to validate and check the transferability of our findings.

Mixed-methods design and conceptual framework

The study used a pragmatic mixed-methods design involving an integrative approach of ‘connecting’ quantitative and qualitative data rather than ‘merging’ or ‘embedding’. 60 Mixed methods were employed sequentially to provide both breadth (phase 1) and depth of description of QI use in community nursing (phase 2). Quantitative data were collected first to describe the range of QI and incentive schemes in use nationally in community nursing during 2014/15, and then to give a sample of providers from which to purposively select five sites for the case study. Case study research enables an in-depth exploration of contemporary events using a combination of qualitative data collection methods. Such methods include interviews, non-participant observation and documentary analysis. 61,62 One of the strengths of the case study approach is that it enables the triangulation of data, thus supporting the validity of the overall analysis. Our case study in multiple sites built on the survey findings by exploring in depth the relationship between policy directives on quality measurement and influences on local implementation. It enabled the identification and probing of the processes of selection and application of QI schemes and other quality measures currently used by community nurses, as well as an insight into the perceived effectiveness of such measures for the purpose of assessing service quality.

Individual and local factors can affect the success of national policy implementation, so an understanding of the interplay between these is fundamental in ensuring that any benefits from national initiatives, such as QI schemes, are obtained. To conceptualise our study, we used a framework originally developed by Franco et al. ,63 which had been utilised successfully in our earlier work. 32 This multilevel framework includes factors that influence individual behaviour and attitude, and incorporates organisational and wider health system contexts and relationships, helping explain the interplay between them. These contexts are all likely to have a bearing on which QIs get selected, as well as how they are applied in practice.

We ‘co-produced’ this project by working in partnership throughout the research process with patient and carer representatives, commissioners and community nurses, who were part of the larger study team. This included drawing on their expertise and knowledge when formulating the research question(s), scoping the literature, collecting and analysing data, and identifying the key messages, mediums and target audiences for dissemination. In a subsequent phase (phase 3), we held engagement events with commissioners, health-care practitioners, patients and carers in 10 locations nationally to check our analyses and interpretation of evidence to improve the dependability and transferability of our findings and also to support the development of good practice guidance (see Appendices 2–4).

Setting

The study provided a national snapshot of QI schemes applying to community nursing, followed by an investigation of the selection, application and implementation of QI schemes in five sites across England. Each case site comprised a dyad of a community nursing service provider contracted to provide NHS services and their associated CCG. At least one case study site was an independent provider. The included community nurses were registered nurses providing home-based nursing care to adults. A geographical spread of case sites and different types of community nursing provider enhanced the reach of the project and transferability of findings.

For clarity, the methodology of the national survey will be reported first, followed by the case study.

Phase 1: national survey

Phase 2: case study

A case study is a preferred method for examining, in detail, contemporary events over which it is not possible or desirable to exert control. 61,62 It constitutes an in-depth exploration of a ‘bounded system’ (p. 61),62 or ‘case’, over time, through the use of multiple data sources, rich in context and detail. The proposed case study was to explore in depth the processes in use for measuring quality in community nursing services in England. The aim was to collect data over the duration of the annual commissioning cycle, focusing on agreeing contracts and quality monitoring and evaluation in order to understand the processes involved. We also aimed to investigate the impact on front-line staff and patients and their carers of the application of quality measures in practice, and the perceptions of all participants of the usefulness of such measures for determining the quality of nursing. Following the Health and Social Care Act 201226 and the implementation of clinical commissioning organisations, and in discussion with the research management group, it was decided to explore whether or not the new organisational context might determine or help to explain any variations found across different sites. A multisite case study approach was therefore deemed appropriate and we recruited five case sites across the country.

Data analysis

The case study generated multiple sources of qualitative data from the sites, including interviews with patients, local senior managers (commissioner and providers) with a role in quality implementation, community nursing team leaders, focus groups with front-line nurses, observations of front-line nursing staff and quality meetings, and documentation relating to the selection, application or monitoring of quality. Interview data were audio-recorded and transcribed verbatim, and contemporaneous notes were made during the observation of meetings and the shadowing of front-line staff. Data were anonymised and entered into NVivo 10 software (QSR International, Warrington, UK), or, in the case of front-line observations, into a Microsoft Excel^® spreadsheet to aid comparison.

Qualitative data analysis drew on constant comparison techniques by which analysis proceeded with and informed data collection,67 with members of the research team liaising regularly to share their interpretations of findings from the case sites. Data were open coded initially to generate concepts that were then validated by comparison and discussion with members of the research team, whose composition reflected key audiences. Once the codes were agreed, these were used to construct a coding framework within NVivo and a team approach to coding proceeded, with each member of the team taking responsibility for coding a particular set of data across the sites [e.g. one member of the team coded all of the provider manager interviews (KP) and another coded all of the commissioner manager interviews (CP)]. All members of the team coded the meetings that they themselves had observed. Front-line team focus group data were analysed by one researcher (LD) and the service user interview, shadowing and observation data were analysed by the nurse researcher (SH). In this way, the research team gained a cross-site perspective. The individual coding databases were merged regularly into an NVivo master database to check progress and assess whether or not there were any differences in the way codes were being applied. The resulting codes were then further analysed to determine their attributes and dimensions, resulting in themes relating to selection, application and usefulness of QIs in the case study sites.

An adapted version of the five-level analysis theoretical framework developed by Franco et al. 63 was used to aid our interpretation of the interview and focus group data (Figure 2). This framework was developed in previous work that examined the relationship between financial incentives and behaviour in a range of primary care and community settings in health care. 32 It is applicable to the selection and implementation of all types of QI schemes, not only those that include financial incentives. The framework assisted in enabling the matching of codes and emerging themes with potential contextual facilitators and barriers to use of quality measures in terms of meeting priorities, assessing service quality, influencing commissioning and bringing about changes in service delivery. To increase the rigour of the analysis, use of multiple sources of data provided opportunities for comparison and contrast across accounts. For example, patients’ views, expectations and experiences demonstrated an alternative perspective on the values driving selection and application of QIs by commissioners and health-care providers.

FIGURE 2.

Framework used to aid analysis. 63

Documentary analysis was limited due to the relative paucity and incompleteness of documents supplied to the research team across the case sites. The standard NHS contract was analysed to assess the degree of content relating to quality of community nursing services. When it was possible to obtain documents containing information about QIs, these were analysed to identify which QIs were being used for contracting community nursing in 2015/16. Documents included quality reports, ‘dashboards’ and schedules 4 and 6 of the NHS contract from the case sites (see Chapter 4). The analysis compared how, or if, quality measures related to the professional standards set out in Vision and Strategy for Nursing, Midwives and Care Staff 3 and CQC quality criteria now being applied to community services. The documentary analysis was undertaken by a member of the research team who was actively working as a community nurse at the time of the study.

The next section describes the organisation of the stakeholder engagement events.

Phase 3: stakeholder engagement

Evidence suggests that research is most effectively disseminated using multiple vehicles, ideally with face-to-face interaction. 68 Dissemination events were an integral part of the study design and were planned to start towards the final few months of the project. We planned to hold workshops across the country, both putting on special events ourselves and contributing to other organisations’ meetings and conferences in order to check the study’s emerging findings and develop our good practice guidance with a wider audience. These events were intended to include a mixed group of commissioners, service managers, front-line staff, patients and carers. Owing to the time constraints on NHS staff, these were timetabled to last either a morning or an afternoon, when possible, and attendees were able to hear about emerging findings from the study and discuss the good practice guidance.

Researchers contacted participating case sites and used personal and professional contacts and networks in different parts of the country to identify other organisations that might be interested and willing to assist in holding a workshop locally. Once a workshop was agreed in principle, a key local lead for community nursing was identified and asked to give an introduction to the day. An experienced facilitator with a commissioning background, a member of the core research team and one other member of the wider team involved in the project facilitated the workshops. To gain stakeholder feedback about some of the study findings as well as statements from the draft good practice guidance, a key activity during the workshop was to engage small delegate groups in a deliberative dialogue exercise. 69 These small group discussions were distinct from focus groups in so much as data were not collected with which to inform the study; rather they were designed to discuss our findings in relation to delegates’ own experiences. Each workshop produced feedback on emerging findings and draft good practice guidance. Nursing participants received certificates of attendances for professional revalidation purposes. Participants completed evaluation forms and facilitators reflected on each workshop, making adjustments to timing and materials in response to feedback (see Appendix 2).

Projected outputs

• Development of a project website on which outputs based on emerging findings of the study have been published (www.QuICN.uk). As our NHS co-applicants stressed the importance of getting findings out early, we began to disseminate emerging findings via our website within 6 months of starting the project, with the analysis of the national QI database and publication of discussion papers (www.quicn.uk).

• Publications, including the full report, evidence summaries for participants and other stakeholders, peer-review journals, local NHS newsletters and A5 laminates, are in progress.

• Using the findings of our project we have developed good practice guidance (see Appendix 4) on how best commissioners and managers can approach the selection and application of appropriate quality measures for community nursing and their perceived usefulness from the patient and carer perspectives.

Patient and public involvement

There has been substantial input from service users (patients, carers and/or members of the public) throughout the study. In earlier work to develop local QIs for community services in Bristol, a service user expressed concern that the administration and collection of data for QIs would distract community nurses from delivering patient-centred care. This individual was a co-applicant on the successful proposal and has been the co-lead on the user aspect of this study. As a former performance manager and member of the local community health forum, he contributed to designing the study and recruiting other lay people to become involved by sitting on the research management group and/or being part of a service user reference group (SURG) for the study. The SURG has contributed to the design of data collection tools for patients and carers, including an aide-memoire to assist researchers in covering topics thought by members to be important (see Appendix 6), the interpretation of findings and the content of the final report. SURG members have also been involved in the development and delivery of the stakeholder engagement workshops. In addition, a member of SURG was involved in interviewing and selecting research staff.

Patients and carers have attended workshops where emerging findings have been discussed and contributed to the interpretations of evidence and development of good practice guidance.

Ethics and NHS permissions

We considered our project to be low risk (as it is not an intervention study and we were not recruiting patients unable to give informed consent for interviews). Therefore, on advice received from NHS ethics, we initially submitted via the proportionate review process. On their closer scrutiny, the project was deemed to involve risk as it involved the shadowing of front-line staff during patient visits, and the submission was, therefore, forwarded to the next full NHS ethics meeting. At this meeting, the proposal to include the shadowing of front-line staff was considered to be intrusive to patients. Moreover, the frailty of community nurse patients, some with fluctuating mental capacity to give informed consent for an observer to be present, led to the project then being referred to a specialist NHS Ethics Committee. Approval for the study was finally obtained in July 2014 (National Research Ethics Service Committee Yorkshire & The Humber – Leeds West 14/YH/1059). The process for shadowing front-line staff to observe documentation of quality data in the field (i.e. patients’ homes) required that nurses visiting patients a few days before the day of the observation visit took a study information sheet to patients in order to allow them to consider the nurses’ involvement in this aspect of the study. On the day of shadowing, the visiting nurse was consented to the study and went into the home alone to see the patient to ask if they would agree to a nurse researcher being present to observe their nurse for that visit. If they answered positively, the researcher was admitted. The researcher then obtained verbal permission from the patient to observe the nurse. Formal written consent to take part in the study was not required from patients themselves, as they were not study participants and no identifiable data were collected from or about them. We subsequently sought and received additional ethics permission to recruit patients to a focus group when we found that we were unable to recruit enough patients via front-line observation visits. These participants were recruited through the SURG.

We were unable to start NHS R&D permissions until we had identified potential case sites from the survey findings. Recruitment entailed initial contacts to both a community services provider and a commissioner from the local CCG, and approval to participate from both partners was required before an application for NHS permissions could be sought (this was sometimes a time-consuming process). Commissioning organisations were comparatively new and there were difficulties identifying the appropriate person within the organisation to give approval. Moreover, community nursing services provider organisations have a range of types (e.g. NHS community trust, combined acute and community organisations, independent or private), meaning that it was sometimes difficult to find the person who could take responsibility for signing up to the project. The NHS permissions process was lengthy and involved duplication and slight variation of requirements across the sites, where each application was considered separately. Owing to the multiplicity of data collection methods and range of participants in a mixed-methods study, there was a large amount of related paperwork for R&D departments to review. This created a lot of extra work for both the R&D departments and the research team. Overall, the administration involved in the permissions process was time-consuming for each organisation, and, on one occasion, having just achieved successful sign-off from R&D in one site, permission was suddenly withdrawn by the provider without explanation. One other case site took several months to obtain internal agreements, having initially expressed interest to the researcher, so we eventually withdrew and selected another site from the list in a similar area. We were eventually able to recruit and obtain permission for all 10 organisations constituting the five case sites, although we were late starting data collection in two sites.

Deviation from the original protocol

Given the degree of organisational change and workforce pressures present during the course of this study, we were fortunate to have established good links with our case sites and to be allowed access to undertake data collection. However, the following small changes to the protocol had to be accepted.

The number of national and regional NHSE quality leads we interviewed was comparatively small. The interviews we conducted suggested that their strategic role with CCGs and current priorities tended towards the acute rather than the community sector and, although they were knowledgeable, their actual role in relation to quality process implementation in community settings appeared to be more limited. These interviews were, however, very useful for informing subsequent data collection in the case sites.

As explained earlier in this chapter, we took a pragmatic decision to alter our means of data collection for the national survey in response to the incompleteness of the CCG data available online at the time of the survey. We had planned to undertake an electronic survey, but the requisite e-mail addresses were not provided online owing to data protection, and the telephone numbers supplied usually went to a local administrative assistant who did not know to whom our enquiry should be directed. We were frequently asked to go through FOI channels to access the information we required, as these departments were set up to provide advice on a range of topics, so we used this as our main method of data collection. We also decided to focus our analysis on CQUIN data only for this part of the study, as only these data were provided in a form that was sufficiently comparable across organisations.

One case site did not allow us to shadow their front-line staff. As it had been hoped to recruit patients for interview from the shadowing exercise, this led to the provider organisation helping us with the identification of patients by offering us access to their local patient group, with limited success. There was considerable similarity in the findings from the four sites where front-line shadowing did take place, and evidence from the focus group of front-line staff that was held there confirmed that the same contextual pressures and views on quality measurement found in other sites pertained.

The supporting documentary analysis we had planned was constrained by the variation in the documents supplied to us by the case sites. We were able to compare CQUINs across the sites again, but, owing to the changes in the CQUIN schemes over time and current policy-drivers, the variation in what sites were doing nationally and locally had reduced. There may also have been some concerns in case sites at the potential sensitivity of documents being used.

Descriptive statistical analysis of data

Analysis of local community nursing CQUINs

As the focus of the wider study is on process, including an exploration of the way in which CCGs and providers collaborate to select and implement CQUINs, findings are presented for the total number of local community nurse CQUINs (n = 484). The majority of local community nurse CQUINs (n = 309) were uniquely worded, suggesting that they were tailored to local circumstances. First, the local community nurse CQUINs were analysed for their fit with the main quality domains used by CQUIN schemes.

Distribution of local community nurse CQUINs across care themes

The intended aim or goal of each local community nurse CQUIN as stated by the organisation was coded into 13 care themes (Figure 3). A specific clinical focus accounted for 202 (41.7%) of the total number of community nurse CQUINs, namely long-term conditions and care planning (58, 11.8%), end-of-life care (36, 7.4%), dementia (34, 7.0%), PUs (34, 7.0%), falls (24, 5.2%), addiction (9, 1.9%) and care of older people (7, 1.4%). A further 191 CQUINs (39.5%) related to organisation of care (83, 17.1%), organisational issues (80, 16.5%) or admission and discharge (28, 5.8%). Fifty-three (10.9%) CQUINs addressed patient safety (28, 5.8%) or patient experience (25, 5.2%). In addition, 38 (7.9%) had a non-specific focus on clinical effectiveness. The preponderance of CQUINs addressing organisational aspects of care and/or service provision suggests that commissioners and providers alike were focused on adapting the service to the demands of the changing health-care landscape in England.

FIGURE 3.

Distribution of community nursing CQUINs across care themes (n = 484).

Content analysis of key care themes

Given NHSE policy priorities in relation to the reduction of avoidable hospital admissions and timely patient discharge,24 it was surprising to find that this was not apparently an explicit key goal in relation to choice of CQUINs for community nursing. Although the categorisation of care themes in the survey gives a broad indication of CCG priorities in relation to community nursing, we investigated four areas of particular importance in community nursing day-to-day work to see if a link with admissions avoidance or timely discharge was evident therein: organisation of care and organisational issues, long-term conditions, care planning and end-of-life care.

Summary

The survey analysis provided a snapshot of the range and variation of CQUIN schemes applying to community nursing nationally and the types of organisations involved in commissioning and providing nursing services since implementation of the Health and Social Care Act 2012. 26 Although CCGs tend to commission services from a single provider, it is evident that nearly half of the provider organisations deliver services for more than one CCG, suggesting a considerable workload in relation to CQUIN negotiation for providers. The number of CQUINs varied in each organisation but averaged < 10 (as suggested by the guidance for CQUIN 14/15), and this did not appear to be associated with the numbers of organisations involved.

Scrutinising the survey data in relation to aspects of the two quality dimensions, namely patient safety, patient experience and effectiveness of care, and structure, process and outcome, revealed the complexity of community nursing CQUINs in use. It was clear that a sizeable proportion of CQUINs addressed more than one of these aspects. It was also notable that, although the aspect of quality dimension 1 most frequently addressed was effectiveness of care, only a relatively small percentage of CQUINs focused on outcome, either alone or in combination with at least one other aspect of quality dimension 2. Process was the aspect of quality dimension 2 most often addressed, despite the presumption that CQUINs are designed to address safety, experience and care effectiveness, all of which should arguably be construed as outcomes. These findings suggest an assumption that focusing on process implies effectiveness of care. As nearly 60% of the CQUINs examined explicitly addressed organisation concerns, with many of the remainder implicitly focusing on the structure or processes within organisations, these findings together suggest that all of the CCGs recruited to the survey, along with their community nursing providers, were focused almost exclusively on adapting to the demands of the changing landscape of health-care provision in England. This perception was reinforced by the content analysis of the four care themes, namely (1) organisation of care and organisational issues, (2) long-term conditions, (3) care planning and (4) end-of-life care, where an implicit drive towards reducing patient admission to, and time spent in, hospital was evident.

The findings from the survey highlighted particular issues in relation to the case study phase of our research, namely the way in which the quality of outcomes was being assessed and the focus on developing structures and processes to meet the rising demand for community-based care. In the next chapter we present a description of our case studies and our findings concerning the local context for community nursing quality measurement within them.

Case study sites

Each site comprised a CCG and a service provider. Community nursing services were provided by four NHS organisations (community trusts, combined acute and community trusts or community foundation trusts) and one social enterprise. In all of the case sites, community nurses were located in geographically based teams serving a number of general practices. Teams were accessed by a single point of entry system, whereby there was one centralised telephone number for referrers and other patient enquiries. For tables summarising the characteristics of the case sites and the numbers and types of participants, see Chapter 2.

The following text provides more details of the commissioning and provider organisations. Pseudonyms have been used and any similarity with existing place names is unintentional.

Participants in the case study sites

A total of 111 individuals (commissioners, provider managers, community nursing team leaders, front-line community nurses, patients and carers) took part in 62 interviews and nine focus groups across the five sites. A further focus group of eight patients and carers took place locally. Members of the research team observed 27 meetings attended by commissioners, provider managers and/or community nursing team leaders.

Four of the five sites consented to a nurse researcher shadowing community nurses during their day-to-day work to observe processes in place for recording QI data both in patients’ homes and in the nurses’ community bases. Twelve community nurses were shadowed in 13 sessions and visited 66 patients in their homes. Three patients refused for their nurse to be observed on the day of visit. On average, home visits lasted just under 30 minutes, but the range was from 10 to 95 minutes. In addition to visits, front-line nursing staff were observed completing their records on computers at their base and other administrative tasks such as referrals and ordering equipment. Handover meetings were observed at each participating case site.

CQUINs in use in 2014/15

Documentary data collected for phase 1 of the study identified that all of the case sites implemented two national CQUINs in 2014/15, namely the FFT and the NST, involving the measurement of catheter-acquired urinary tract infections (CAUTIs), new PUs and falls. The fourth domain of the NST was not reported on in the community. Some sites also included local CQUINs for community nursing that complemented/added to the national requirements concerning PU and falls measurement and reporting. In addition, some implemented local CQUINs focusing on dementia. Phase 1 analysis revealed that number of local CQUINs implemented for community nursing across the case sites in 2014/15 ranged from 1 to 12, with Dogwoodheath (the social enterprise) adopting the largest number. There was some commonality in the areas of care addressed, namely dementia care, discharge planning/review, multidisciplinary working, patient education and patient experience (Table 9). However, there was also variation across the sites in areas of care prioritised through the adoption of CQUIN ‘stretch’ targets, as shown in Table 9.

Table 9 shows only those CQUINs that were definitely applicable to community nursing in 2014/15 in each site. There were other CQUINs included in the provider organisation contracts that may have applied to community nursing. Areas of care addressed by these CQUINs included many of those listed in Table 9, as well as training for non-registered nursing staff and learning from safeguarding.

Quality measures used in the case sites in 2015/16

In response to our request for documentary data, the case study sites submitted a mixture of details from schedule 4 and/or schedule 6 of the NHS standard contract, as well as dashboards or scorecards through which quality requirements were measured and reported. There was no set format for these; some looked at specific aspects of monitoring, for example a dashboard concerning only workforce compared with others incorporating all the national and local contractual quality requirements. One case study site submitted the local quality requirements (schedule 4), an example of the dashboard and a scorecard with considerable overlap in what was being reported on in each. No case site submitted a complete data set in this regard (Table 10).

Alderton West commissioners were particularly focused on partnership working, patient consultation and education. It was also their intention to work towards achieving NHSE outcomes such as improved life expectancy, improved patient experience and moving care outside hospitals and closer to people’s homes where possible. Documentary analysis revealed that the service’s quality dashboard for 2015/16 contained 44 local quality requirements applicable to community nursing. Indicators included measures for access to services, delayed transfers of care and completed assessments, response times for patient complaints, the number of serious incidents, never events, staff appraisals and training, vacancy rates and reporting on the NST as reported below.

Beechbury Community Health Trust’s quality reporting for care is aligned with the five CQC quality domains and the fundamental standards of care. The trust was obliged to meet > 40 QIs for community nursing services, although some of these applied only to caring for patients in residential settings. Particular attention appeared to be focused on patient safety, the responsiveness of the organisation and on staff experience and capacity.

The Cedarham commissioners agreed with the trust not to have any CQUINs for the contractual year 2015/16. Nevertheless, a district nursing quality dashboard for that period indicated that they were collecting data on a number of local quality requirements: sickness and vacancy rates; governance issues around the numbers of incidents and complaints; caseload size; the percentage of face-to-face contact time and the number of grade 3–4 PUs and falls. Also captured were the number of contacts for palliative care and with general practices. Other highlighted areas were medicines management, serious incident reporting, training of staff for safeguarding adults, patient experience and patient safety. Copies of the dashboards were placed on staff noticeboards in the bases (front-line observation notes).

Dogwoodheath Respond was expected to meet approximately 20 local quality requirements regarding community-based care in Dogwoodheath in 2015/16, including some relating to waiting times and end-of-life care. There was a particular focus on community nursing issues rather than broader governance compared with other case site studies, and on capturing patient experience through a range of surveys, which formed many of the requirements. Local reporting requirements included information about referrals and case management.

In Elmhampton, the EFT was required to meet 10 QIs for community nursing services. Over half of these related to patient safety, the remainder being involved with the responsiveness of the organisation. There were additional indicators relating to transfers of care that could also involve this service. Again, in at least one of the main bases there was a noticeboard with quality data posted so that staff could see how the well the organisation was doing (front-line observation notes).

All of the participating sites had CQC inspections at one point during data collection for this study.

The context of care

National policy directs and influences the way that health service quality is measured and applied in the context of community nursing. Throughout the study, we found that front-line staff, patients and carers were keen to offer their thoughts about local issues affecting the quality of care provided and its measurement. In the following section, we present findings related to the context of quality measurement in community nursing in relation to the national picture and across the five study sites locally. In discussion within the research team it was evident from the within-case analyses that there was a great deal of similarity in findings with respect to all our emergent themes. We have therefore presented findings from a cross-case analysis in this chapter and in the following chapters using data drawn from all of the sources across the five case study sites, as well as the data from the NHSE interviewees.

Identifiers for quoted excerpts from interviews and focus groups in this chapter are shown in Table 12. Excerpts from field notes recorded during the observation of meetings and of front-line staff at work are labelled accordingly.

In each case, an individual participant is identified by a unique number preceded by the name of the case site (e.g. Beechbury CNFG1), except for the local patient and carer focus group FGP1/FGC1.

Cross-cutting themes emerging from the data in relation to the current context of community nursing quality measurement included national–local interface, reorganisation of care, IT implementation and community nursing caseloads.

Summary

The five case sites comprised four urban localities and one rural locality; all of the former were areas of high deprivation compared with the national average. Population sizes were comparable across the sites. Organisation of the community nursing services differed little between the sites, being mainly geographically based. One site (Alderton West) had a mixed provision, with some nurses located in teams attached to particular general practices and other larger teams being geographically based. Community nurses operated in integrated or multidisciplinary community teams in three of the sites. Community nursing services were provided by two NHS foundation community trusts, one NHS foundation trust offering both acute and community services, one NHS community trust and one social enterprise.

There was a notable range in the number of local CQUINs implemented across the case study sites in both 2014/15 and 2015/16. Nevertheless, most of the four sites using CQUINS in 2015/16 had reduced the number of those adopted compared with the previous year, although Dogwoodheath continued to have the greatest number. Variations in local priorities were evident in the focus of local CQUINs adopted in the different sites in both years. Other quality requirements for community nursing in 2015/16 reflected a common concern across the five case study sites for issues relating to patient experience and safety, as well as for staff development and well-being.

The national and regional NHSE representatives interviewed made it clear that they cannot undertake the monitoring of organisation performance in terms of quality comprehensively, due to a lack of resources and of robust quality data concerning community nursing services. Moreover, there is an explicit agenda to promote CCG flexibility and autonomy. The combination of these factors has implications for the consistency of quality standards achievable across different CCGs and providers. However, it may be impractical to assume such consistency is achievable in any circumstances, given the impact that local conditions can have on a service. The variety in the number and focus of indicators in use across the five case sites during the period of data collection evidence the differences in organisational priorities; these priorities could not be accommodated if CCGs were not able to decide how to respond to local conditions.

It appeared that reorganisation in response to the new care landscape in the UK involved a degree of restructuring in all the case sites. Although many aspects of these changes were welcomed by staff, a sizeable minority were reported to be struggling with altered shift patterns, new geographical configurations of teams and updated or problematic IT systems. Moreover, recruitment, retention and sickness rates among community nurses were an issue in all the case study sites.

All these factors appeared to have an impact on the ability of staff to deliver a high-quality service, which was further compounded by the increasing complexity of caseload management. Although increasing the numbers of staff through increasing skill mix was a favoured solution for this problem, the data showed that it did not necessarily alleviate the situation, as numbers of nurses with the requisite high-level skills were not always available.

The overall picture of the community nursing services in all the case sites was, therefore, of a service under pressure, struggling to deliver care to the required standard. This situation calls into question the feasibility of improving the quality of care for patients without the investment of additional resources into the service.

Processes used to select indicators

Typically, providers and commissioners enter into negotiation, with both sides clarifying their priorities. Commissioners reported that, every contractual year, they present to their providers an initial list with QIs and suggestions for local CQUINs. They then spend a period of a few months discussing the feasibility of these indicators with the providers. The process of selecting the initial list of indicators usually takes into consideration national guidance, their local priorities and commissioning intentions, and any gaps in quality identified locally throughout the year. Sources for the selection of indicators, according to commissioners, derived from:

• compliance with national policy guidance and national indicators, for example time scales for conducting root cause analyses for serious incidents, Five Year Forward View,28 NHSE commissioning priorities

• national quality requirements that may need to be supported by additional local requirements

• local commissioning priorities resulting from CCG membership consultation

• the KPIs included in the service specifications for particular clinical conditions

• CQUINs (national and local); previous CQUINs tend to become KPIs

• any gaps in service provision identified via the monitoring of services throughout the year (including quality visits by commissioners to the providers)

• suggestions from the providers.

At the time the project was conducted the annual process of commissioning for selection for quality measures had been starting between September and November each year and ending with the signing of the contract in April. Although the agreement of the local QIs and CQUINs is meant to be a collaborative exercise, it was acknowledged by both commissioners and provider managers that the selection process is usually driven by the former:

The CQUINs were really very much designed by the commissioners and then fairly late on they had a conversation with our contracting team. I think it was really almost towards the very end when they were almost signed and sealed that it became a ‘do you want to comment?’ . . . I think we found out a couple of weeks before the end of the contracting time.

Beechbury PM1

Provider managers certainly felt that they should have the opportunity to work jointly with commissioners to develop indicators. For some of them, their reason for this was previous experience of commissioners not always being aware of relevant issues concerning systems in place:

I think one of the biggest issues is commissioners setting up indicators that they don’t actually understand and they have read somewhere or pulled out somewhere, and actually defining what it is that you’re counting in the first place.

Dogwoodheath BM1

It appeared that a lot of discussion can take place between commissioners and providers about the level of detail and the amount or type of information that commissioners need. This impression was reinforced at meetings where details of prospective indicators were on the agenda:

The CCG finance and performance manager said that they could start by tracking down what they need to reduce, e.g. length of stay, etc., and then have a measurable target . . . There has to be some learning. The provider head of performance said that they first have to come up with a sensible suggestion which they could then discuss with the finance director and the trust board. He said the question is whether the trust can agree to any reduction given how little understanding there is around the issue and how many parties are involved.

Alderton West meeting (field notes)

Given the complicated nature of discussions and the complexity of service activities, it is unsurprising that participants reported that this process can be fairly lengthy and time-consuming before agreement is reached between the parties:

It took us 25 iterations before we got the first spec signed off. Twenty-five drafts.

Dogwoodheath PM1

Some commissioners felt that this can be because the process involves consultation of staff at all stages of the process. It was also noted that GPs can influence both the operationalisation and the adoption of indicators and a few provider managers stated some difficulties in this respect. To avoid such problems, in Elmhampton a community nursing best practice group discussed potential challenges with GPs to help define and/or refine the indicators under consideration:

All the GPs are advocating using the [personalised care planning tool] plan. It’s 45 pages long . . . There are elements of it that are really good and what we’re going back to the CCG and saying is, there are some bits that we absolutely want to use but it’s too short a time frame to get everybody using the same tool . . . we’re all using different paperwork . . . It’s at times like that that I will take that to the senior district nursing group and go, I don’t know, what do I do? How can we manage this? They normally come up with some really good and positive suggestions and so it works.

Elmhampton PM2

The processes of selection appeared often to be based in what had happened in the past or in what data are available. One business manager thought that this could result in the selection of indicators that are not really fit for purpose:

That’s one of the key concepts we’re really trying to push – it’s to start with ‘What is it we want to know?’ . . . Often my experience is we’ve started [with] ‘What data have we got?’ That’s what we use and therefore it doesn’t generate that improvement that perhaps all the effort put into collecting and measuring that data should result in . . .

Beechbury BM2

It was generally agreed that indicators should be meaningful, specific, patient related, achievable and SMART (specific, measurable, appropriate, realistic and time bound). 74 Consistent terminology should be used and data collection processes should be simple and systematic and make use of mechanisms already in place. It was also thought essential that resources are provided for collection of quality data to avoid any detrimental effect on front-line clinical services and that the stability of core services must be ensured with staff overload avoided. Participants noted the importance of, and the challenges involved in, ‘keeping the day-to-day work safe’, although there could be differences of opinion about what this actually entailed:

The head of quality [CCG] said that it had been agreed that the CQUIN would be implemented in two neighbourhood teams [NTs]. He felt that it is too restricted to implement only in specialist teams, as this is too small a beginning to the CQUIN implementation. He is happy for it to be rolled out to two NTs, in line with the requirements of the national CQUIN. The director of nursing [provider] responded that the trust wanted to work with the spirit of the CQUIN, but that it is very late in the year to do so, and front-line teams are experiencing winter pressures on their workload. She said that, given the time frame, she felt that it was reasonable to restrict the CQUIN implementation to specialist teams, as it would be far too onerous to roll out to NTs. The CCG contract manager said that a contract variation has been put forward to change the CQUIN, and a decision is awaited from the management board.

Beechbury meeting (field notes)

Indicator appropriateness

During the discussions between commissioners and providers, one key factor determining which QIs to include appeared to be deciding how feasible or achievable indicators are. Commissioners indicated that, because of the financial element, the negotiation of CQUINs focuses particularly on the issue of indicator achievability, with the caveat that they should not be too easy to achieve, in order to provide value for money. It was felt that a lot of debate and frustration during the application of CQUINs can be avoided if the indicators are clearly stated in the contract and are also measurable:

[Providers] want something that they can deliver basically. So if it’s having something that’s either too difficult to measure or can’t be measured or too woolly . . . we know that we’re not going to get anywhere with that.

Cedarham CM1

However, despite reported attempts to develop feasible and achievable indicators, some participants more closely involved with the practical delivery of care stated that some indicators in use were not fit for purpose. Problems stated included indicator requirements conflicting with those of good nursing practice and adoption of indicators designed for inpatient settings without adjustment for use in the community:

The commissioners decide what we’re going to do and then that’s passed on . . . a couple of times we’ve had CQUINs that have just been so poorly thought out that we’ve been set up to fail.

Alderton West NTL1

Nurses expressed unease when they felt that care delivery was being driven by a QI protocol as opposed to professional judgement. There was concern expressed at a team leaders’ meeting during discussion of an indicator for which data collection processes were not aligned with recommended clinical practice:

One of the nurse team leaders asked about the need to continue doing urinalysis on patients. She said that this is not good practice, but as it is included in a quality indicator, nurses are having to do it more often than clinically indicated.

Alderton West meeting (field notes)

A similar opinion was expressed by a front-line nurse in Beechbury:

Purpose T [a pressure ulcer risk assessment tool] carried out on every patient – although the nurse commented that in her view, clinical judgement should take precedence over whether this is an essential standard to meet.

Beechbury (front-line observation notes)

Some indicators were reported to have caused offence, such as requirements to ask all patients about alcohol consumption or continence.

Another issue arose through national requirements for community providers to achieve nationally mandated CQUINs originally designed for application in acute settings:

What you immediately run into actually, is that they are the set ones nationally. Everybody has got to do them but it doesn’t quite match what community services provide.

R3

Provider organisations stand to lose income in relation to non-achievement of QI targets for PU prevention (an accepted marker for good-quality nursing care in the acute care sector), and new approaches to risk manage responsibility for PUs had been instigated in all the case sites, including the introduction of purpose-designed patient information leaflets about PU prevention.

This indicator (based on best practice) was criticised by community nurses as sometimes conflicting with patient-centred care:

. . . a 15-year-old who’d been in a road accident and had two toes amputated.

Beechbury CNFG4

At the other end of the spectrum, patients in the terminal phase of their illness may have to be disturbed to be repositioned in bed in order to comply with the PU prevention protocol. If PUs did occur in these circumstances, a serious incident investigation would be triggered:

If they’re comfortable, do they [managers] really want us moving them around just to check their bottom . . . when we know that they’ve got 7 days or less to live? Patient centred, you could argue, is whether it’s appropriate to do these things to the patient at the time. So to roll the patient who actually is nice and settled . . .

It’s distressing actually for a lot of them. They really don’t want it. A lot just want to be left.

We spend all this time trying to keep them in a state where they’re not agitated and then agitate them.

And I just think how can they [managers] justify that [serious incident investigation] because the body’s breaking down. The oxygen levels have dropped.

Another factor impacting on the lack of suitability of the PU prevention indicator in community settings was that, unlike in the acute sector, this indicator was not entirely within nurses’ sphere of influence:

I mean, even if you went into a patient every day, I mean it’s only, what?

Maximum of an hour a day. So you’ve got 23 hours of not [being able to oversee them], legs going back down again, cushion going . . .

Yes. You know? It is – more the patient’s responsibility – isn’t it? – to take on our advice that we’ve given and, you know, help us in helping them, whereas in the hospital . . . you’re there more so you can encourage them more. Oh, you know, come on, you know, can you lay on your side for a bit? You know?

Inclusion of all relevant stakeholders

A common opinion among participants was that indicators should incorporate both clinician and patient perspectives. Commissioners reported valuing indicators agreed after discussion and input from the providers. They felt that ownership from the provider was essential for the success of a CQUIN and that community nursing staff will be more committed and willing to implement indicators that they have helped shape. One provider business manager thought that, to promote engagement and/or ownership from the service, it could be useful to move towards standardising questions rather than answers, so that staff would be able to self-select measures appropriate for their practice.

Provider managers agreed that having informed professional input enhances the quality of any measures developed and helps to establish ownership among nursing staff. The data revealed that, in a number of the case sites, initiatives were being put in place to ensure that staff views could contribute to indicator development, and most provider managers said that staff had some input into indicator identification and selection. Nevertheless, community nurses across all five case sites refuted this view, describing the process as ‘a top-down’ approach; the general feeling expressed by the nurses was that indicators were imposed when and if there were signs of possible problems. Many nurses were unsure who was involved in the selection process and were suspicious of QIs, feeling that they could be used punitively with staff:

I think it’s the people that put the numbers in the computers and think, ooh, this looks a bit dodgy there, and we’ll send it down I honestly do.

Yes.

They haven’t got a clue what’s going on.

It’s probably risk assessed, isn’t it?

I think, you know, if you’ve put lots of Datix [serious incident forms; Datix Ltd, London, UK] in for pressure sores, then they’re obviously going to audit that.

In one site, a nurse commented on the language of ‘harm-free’ care and the implication that they were otherwise treating patients in a way that was detrimental to their health:

So it’s meant to be a useful tool to improve care [NST], now it seems to be to find out why we’re getting . . .

Measuring to see how bad we are, yes.

I think that’s why it’s worded wrong, because it’s all about harm. So you feel it’s negative from the start, because they’re trying to find out how much harm you’ve created.

A provider quality analyst from Beechbury who worked closely with front-line staff on quality data reiterated the importance of the vocabulary used when talking about quality, preferring the phrase ‘see how you are doing’ to more punitive and management language about missed targets and performance.

Community nurses said their role was poorly understood by commissioners and many examples showed how their work differs from that of nurses working in hospitals. Differences include the need to factor in time spent travelling to patients’ homes, tabling visits to fit around medication timings, connectivity problems with mobile technology, unrecorded work necessitated by other ‘missing’ services, isolation of patients, lone working and the practicalities of working in people’s homes. These differences underpin nurses’ views that QIs for community nursing could be made more useful and better reflect the quality of care given if the nurses themselves played an active role in their design and analysis, rather than simply collecting the data. Nurses indicated that they would appreciate being involved in the whole process of quality assessment. When asked how quality measures could be made more meaningful for them, nurses in one site responded:

Maybe involving us in, you know designing the indicators, you know . . .

And that you saw some good come out of doing it . . .

And being involved in action planning to improve it when you get the results.

Patients are not involved in identifying or selecting indicators generally. There was only one instance of patient involvement in the development of an indicator in the data, concerning wound care:

We surveyed about 21 patients to ask them what they want from wound healing. What’s the thing that upsets them most about their wounds? And it talked about things like pain and healing rates, so that was a quite useful exercise.

Cedarham PM2

Some participants showed an awareness that perceptions of what makes care ‘good’ may differ between patients and professionals. Nevertheless, there was evidence of intention to tackle the issues of involving patients appropriately in indicator selection processes.

A number of commissioners thought that the future direction of travel for measuring quality will be towards outcomes-based measuring, dependent on patient input:

It might be more on the patient outcomes and I think that’s the direction that we’re going in . . . waiting times are important because you can monitor the access but we are working more and more on patient outcomes. So when we’re redesigning or designing from scratch a service, part of the public engagement would be what outcome would you expect from this service and then we use them as our core qualitative measures.

Dogwoodheath CM1

Summary

The data showed that, despite stated aspirations to involve other stakeholders, particularly front-line nurses and service users, current processes of selection largely involve a ‘top-down’ process involving commissioners and senior provider managers. The fact that the process is already very time-consuming must mitigate wider stakeholder involvement without radical restructuring and reordering of QI selection.

Nevertheless, the disadvantage of excluding community nurses from the selection process appeared to have sometimes given rise to measures which could clash with provision of patient-centred care. This situation inevitably placed nurses in an invidious position, as they were duty bound to apply the indicators while simultaneously bearing a professional obligation to deliver care in line with best clinical practice.

Despite participants’ stated concerns to select feasible indicators, the decision to extend quality measures suitable for acute settings into the community without appropriate modification appeared to render some quality measures as problematic from the nurses’ viewpoint. It was apparent that, where nationally mandated indicators are concerned, this matter was out of commissioners’ hands.

The combination of all these factors implied that nurses were being placed in unnecessarily stressful situations with respect to care quality measurement. This appeared to exacerbate further the current difficulties experienced with the delivery of community nursing services, as detailed in Chapter 4.

Quality indicator monitoring

Quality measurement in a service is a component of quality monitoring. Routine monitoring of QIs happened in all of the case sites by means of ‘dashboards’ and scorecards, usually reviewed at regular quality meetings, which were either limited to individual organisations or attended by representatives of both the CCGs and the providers. All of the quality review meetings observed included these items on the agenda:

They discussed the following indicators from the dashboard: a. Mandatory training: the CCG quality manager observed that a number of indicators relating to mandatory training had not met the required contractual threshold. The provider head of performance explained the process that the trust have in place in order to monitor non-compliance and improve staff completing the relevant training requirements.

Alderton West meeting (field notes)

One participant described these joint meetings as an important part of monitoring because they allowed commissioners to be both reactive and proactive when it comes to quality issues:

[The meeting membership] can set its own agenda and it can hold the trust to account . . . it can react to quality issues that it sees in community services.

Cedarham CM2

In relation to the monitoring of indicators, it appeared that commissioners relied mainly on the written evidence presented to them by the providers. This was often categorised according to ‘risk ratings’ – green, amber and red – reported through the performance dashboards:

The chair asked what the risk rating for quality in relation to staff numbers was. The trust said it is red for recruitment and amber for quality.

Cedarham meeting (field notes)

However, one commissioner implied that an element of trust was essential when considering the credibility of reports. A number of commissioners relied only on provider self-reported data, but others supplemented the data from providers with visits to various practice settings, both announced and unannounced. One commissioner stressed that it was important that such visits were led by members of staff with a detailed understanding of the service under inspection.

Another commissioner explained that the purpose of the scheduled visits (which included shadowing of district nurses at work) was not so much a formal inspection of the provider’s performance as an effort to understand the challenges the service might be facing. A commissioner working with a rurally based provider emphasised the value of visits by commissioners in understanding the work district nurses are doing. The following quotation indicates that the quality of a lot of care that district nurses are offering is not easy to capture:

I went to this one very isolated farm in the middle of nowhere – this little old lady, 85, on her own; and it was the only contact she had and actually, the district nurses do have to do a bit of the social thing, and just checking they’re all right. So it’s all through the conversation and you can see there are key points coming out . . . actually seeing it in reality . . . it really brought it down to that patient level.

Elmhampton CM2

In some sites, instead of monitoring on the basis of referrals and activity, commissioners and providers were moving into monitoring community nursing services in terms of caseload and patient pathways through integrated teams. This development was not without its problems in terms of measuring quality, mainly due to the reported initial uncertainty and lack of knowledge about the numbers of patients being supported by the teams, and about the length of time a patient may spend on the team caseload. Gaining this information was seen as a priority for the quality of the service to be monitored effectively.

Some commissioners complained that the guidance for the national CQUINs is not always clear. For example, commissioners were confused about how to apply the CQUIN on reducing avoidable emergency admissions (2015/16 contracting year):

It’s a new CQUIN for this year on reducing the proportion of avoidable emergency admissions. It looks a bit too complicated and they’ve not put any metrics in and they’ve not put any targets.

Elmhampton CM1

Some indicators are reported to be more difficult to apply than others and require a developmental approach. In these cases participants agreed that 2 or 3 years might be needed for criteria to be met, with commissioners and providers agreeing the stages and milestones for each year.

In addition to the use of ‘dashboards’ and scorecards, provider managers stated a range of different strategies in use in their organisations to monitor the quality of the service, which were used on an ongoing basis. These included assigning practice teachers to inspect a sample of community nurses’ case notes on a monthly basis, to ensure that care delivered was appropriate for each patient:

We have community practice teachers who go into every single [community nursing] team once a month. They select five case files and they go through those, using a standard template to check things off, then they submit that data. Everybody is aware of that, the teams are aware, the team manager’s aware, everybody knows [when] they have to be done by . . . that’s very much embedded.

Alderton West PM4

Other initiatives involved undertaking regular audits in different areas (e.g. wound care and hand hygiene), and appointing leaders and champions for particular indicators to liaise with and encourage front-line staff to meet required criteria. However, in some case sites a lack of time was reported as a significant barrier to monitoring quality:

I think the difficulties for us are around manpower to do that in time . . . my ideal would be that somebody works with every nurse at least once a month to take a view of their practice, conduct with patients, what they’re writing down. We don’t have the staff to do that, it just goes by the by.

Cedarham PM2

It was noted that improving quality requires close collaboration between commissioners and providers, and many participants stated the importance of maintaining good relationships in this respect. This perspective was noted at an internal CCG meeting held prior to a joint quality review meeting with the provider:

Points were raised from the minutes of the previous meeting, some discussion about how to take things forward with respect to a CQUIN that had not been achieved. The focus appeared to be on gaining clarity about issues and challenging the provider where necessary, while still maintaining cordial relations.

Dogwoodheath meeting (field notes)

Communication about indicators within the provider organisation

The data revealed that most information about indicators is relayed to front-line staff and reported back through existing management structures and team meetings, as well as through internal bulletins or newsletters. Some managers make a point of visiting community bases regularly, whereas others rely on e-mails and other electronic communication:

The head of planning and performance explained that a link to the Friends and Family Test is circulated on e-mail now and is advertised in the staff newsletter.

Elmhampton meeting (field notes)

It was noted that effective face-to-face communication can be hampered by geography or the size of community teams:

So getting the message to them, when geographically they’re spread over Cedarham, that’s always really difficult. They’re always out and about, so you’re trying to meet up and get the message to them on a regular basis.

Cedarham PM1

A theme which came through strongly across the case sites from community nurses was the lack of communication with them prior to the implementation of a new QI. Nurses could be told of an indicator only days before it was due to be used, potentially creating problems from the outset. Several examples were given, including the following from a focus group which included some senior nursing staff:

You may even want to ask my colleagues now, did they know there was a [named] CQUIN coming in the next couple of months?

No.

Right, so . . . our managers know about it, but they haven’t told anyone else. And we’re looking at the training for it . . . and so on. But all that’s happening behind closed doors.

 . . So it’s something else they’ll be doing they know nothing about . . .

How good would that be just to let us know so we can start preparing . . . our own learning from learning and development, a course you’ve got to attend.

Although there was acknowledgement that community nurses generally provide very good care and work extremely hard, a common theme found in the managers’ data was that they often do not fully appreciate the importance of collecting indicator data, or do not understand why they are being asked to do so. It was noted that it was often challenging to engage staff in this respect, particularly as most nurses, if short of time, always prioritise direct patient care above QI data collection. One manager felt that there was a vicious circle in operation, in that a lack of attention to data collection inevitably led to poor-quality data; she felt that this only reinforced nurses’ views that data are not worth collecting in the first place. There were also problems acknowledged with the clarity of communication between different levels in the organisation, so that by the time relevant information reached front-line staff, instructions regarding data collection could be quite confusing and lack sufficient detail.

Managers were generally aware of the need to help staff understand the importance of collecting the required quality data, although some interviewees also acknowledged that the organisations have more to do to in this regard. It was also recognised that staff are more likely to embrace an indicator if they can see how it brings clear benefits either to patients or themselves. A national participant noted that a problem with achieving CQUINs is that often the teams that have worked for them do not know what resulting benefits may have accrued, because the funding gained goes to the organisation rather than directly to the teams that implemented them:

They do it to get the money and then . . . either it doesn’t make that much difference or they get the money, but that particular team doesn’t get the money, it goes into the organisation.

N1

In one organisation, a business manager visited the front-line nursing teams to explain the CQUINs in detail and to highlight their importance and how, if good-quality data were collected, the service could be improved:

I gave a presentation to a district nursing team event recently . . . I ran them through all the CQUINs and say this is why we do the CQUINs, this is what the aim of the CQUINs was . . . the feedback was that it was helpful because they could actually see what it was all about . . . I showed them the activity graphs and how activity had increased and I said ‘because you collect your activity data, because you actually log it on RiO [an electronic care record system for health and social care organisations] we are able to show them that it’s gone up by 7.2%, we can show them how run off your feet you are’.

Elmhampton BM1

He reported that the quality and consistency of data collected improved significantly after this visit. However, front-line staff are not always informed about how the data they collect are used. Community nurses commonly reported that they received only limited feedback on their QI outcomes and some of the provider managers interviewed noted this to be a failing of the system and an issue that needed to be addressed:

That’s really important, about giving feedback back into the nurses . . . it’s all around morale and how you look after your staff.

Cedarham PM2

In one case site, however, regular feedback from management was identified:

They sort of bring everything together and collate the figures. And in most cases will get back to us . . . And they do a graph for each [area] for each month. And our managers will normally come back. If we are doing well, we’ve been commended. If we are not, she feels disappointed.

Cedarham CNFG2

In a different focus group from this same case site, a senior nurse spoke of their collaborative approach to feedback and analysis:

What we tend to do with our team when we achieve something better, we might buy just a cake and celebrate . . . And we look at it and say what went wrong and again like the patients’ survey. When somebody brings, I said . . . Please open and let me know what your patients say. So that way we look at what the key themes within the area, whatever they say, oh, nurses are normally late. They don’t call me or whatever. Then we work around that . . . Because when we fall, we fall together. When we rise, we rise together.

Cedarham CNFG9

In a different site, during an internal provider meeting to discuss CQUIN achievement, the senior manager chairing the meeting made a point of asking that front-line staff be congratulated on their work and asked for input about how to use the money accruing:

The chair reiterated that the community staff have done ‘a fantastic job’ and asked that they should be told this. Money that has been earned through meeting the CQUIN targets is now available, and she asked that staff should be consulted about possible ways of spending it.

Dogwoodheath Meeting (field notes)

Data collection processes

Some participants stated that combining a number of apparently simple processes can result in a very complex system and that frequent ongoing monitoring is required to get an accurate picture of quality in the service, which is not always possible due to capacity issues. Data collection was generally acknowledged to be time-consuming, particularly where duplication of data entry is unavoidable. All data need to be recorded electronically, but some are still collected initially by hand on paper. Although there was a focus in most sites of reducing this activity as soon as possible, it was also acknowledged that it would probably not be possible to remove paper notes completely:

We have to have notes in the patient’s home . . . it’s not just us that are involved with those patients; we’ve got adult social care; we’ve got GPs. We’ve got a lot of family that would like to be more involved in the care . . . so our home notes are integral to good, personalised care. But at the same time we need to put things [on computer].

Elmhampton PM4

The increasing emphasis on documenting information for quality purposes was perceived as having been accompanied by a decrease in the time available to spend with patients, so that nurses felt that their workload was being driven by indicator reporting requirements:

It’s totally taken over. Patients don’t come anywhere near priority now, they’re right down at the bottom. It’s all these targets, audits, data this that and the other.

But the hardware isn’t up to speed yet, that’s the problem.

And I think over the years, face to face contact with patients has gone down, because you’re spending more of the afternoons on a computer.

From a patient point of view, although recognising the time taken to complete notes, the nursing records held in the home were a valuable resource and found to facilitate involvement in their own care, enabling them to monitor progress and also accurately to inform other visiting professionals of current nurse input:

Yes. I mean, I think the nurses have a heck of a lot of paperwork to do. I mean, B is writing for ages when she’s here but every single thing is focused on; everything is jotted down. The notes are left here so she’s not writing down anything that we can’t know about. So we know what the blood pressure was that day. We know the blood test results, and it’s wonderful . . . If the doctor rang and wanted to know what his blood pressure was, I’d know instantly because it’s there in the notes.

Alderton West C1

One unintended consequence of increasing electronic data recording, which has the potential to reduce nurse time spent documenting quality information, is that patients could be potentially excluded from the benefits of having access to paper care records in the home. However, it appeared that the amount of documentation required can impact heavily on nurse workload, in particular the need for data to be recorded in a variety of formats:

It’s all in the patient’s notes, documentation I think, some people have the [electronic] notebooks . . . And online as well, put it on the system.

Yes . . . We still have paper notes and [IT package 1] . . . and some practices have [IT package 2] as well so we have to fax or e-mail consultations for them to get on to their systems. So it’s all, it’s quite difficult at times because some of them are triple entry.

There is a national drive to encourage patients to take some responsibility for their own care and prevention of further health problems. This has been interpreted in at least one site as requiring a signature from the patient to say they understand their responsibilities to avoid PUs occurring. However, in the context of financial penalties for not achieving quality outcomes it could suggest an element of defensive record keeping as such a record could constitute evidence that the nurses had done all in their power to prevent the PU and would be helpful, perhaps, in reporting back to the commissioners. Additionally, for safeguarding reasons, the nurses recorded in the notes the fact that all options regarding prevention and equipment had been discussed with the patient or carer. During a shadowing session, a staff nurse at a handover meeting reported a problem with a patient who had been discharged from hospital that day, provoking an immediate response from the team leader:

Another nurse brought up a newly discharged patient from hospital who has come onto caseload with a black heel (PU) – ‘to be Datixed immediately’. PUs are a quality standard – ‘need to demonstrate it did not start in our service’, ‘to be done ASAP’.

Cedarham (front-line observation notes)

Datix (Datix Ltd, London, UK) is the electronic serious incident reporting proforma used in provider organisations. If a patient on the caseload was identified as having a grade 3 or 4 PU (classed as moderate/severe), a serious incident investigation would be triggered. These investigations include interviews with the staff involved and scrutiny of care records leading up to the date the PU was identified. Such investigations are badged as learning opportunities for staff but can be construed as punitive and take a long time to complete:

I think there’s a fear as well, isn’t there, that the finger’s going to be pointed. People do feel scapegoated. I did. I felt terrible. I wasn’t even here when the woman developed it, I was off.

Beechbury CNFG3

During a shadowing session in Elmhampton, one clinical leader said that she thought she spent about 30% of her time on investigations prompted by the Datix system.

Collecting QI data appeared both to have an impact on, and be affected by, difficulties for the workforce arising from issues associated with the complexity of the caseload, adapting to new IT systems, high levels of sickness and staff shortages, as outlined in Chapter 4. Many of the participants mentioned the degree of support community nurses have required to enter quality data onto the computer systems:

A lot of this stuff depends on the inputters. And many of the inputters are the clinicians and the people at the front line. So they have to have access to good IT systems to input the data as well. And sometimes, you know, the systems are slow and clunky.

Alderton West PM2

Recording ‘outcomes of care’ is a key community nursing performance QI. Nurses are required to record the outcomes of their visits on a daily basis. As outlined previously, electronic care records often cannot be updated when nurses are out on visits due to connectivity problems, so ‘outcoming’ (the updating of visit outcomes in order to enable ongoing planning for allocation of workload) must therefore be undertaken when nurses access the live records back at base. Several nurses spoke of working out of hours to meet the requirement:

You stay over your time. On your day off, you come in. Because you don’t want your name to be called as someone who has not done their outcome.

Cedarham CNFG4

It was also noted to be difficult to collect data from some patients on the community caseload, as many of them are frail or lack capacity. In some instances there may also be language problems adding to the challenges. Patient satisfaction surveys are either posted or taken into patients’ homes and left there for completion. Nursing team leaders did not think it ethical for nurses to sit with patients while they complete a survey.

A suggestion was made that it might be better if such forms were administered by an independent body. This would have an additional advantage of circumventing another factor which may contribute to the apparently low completion rates – that the nurses do not like to give the forms out – expressed by one nurse in Elmhampton as feeling ‘like touting for compliments’. At one nurse team leader meeting, it was clear that some nurses did not think that they should be responsible for collecting these data from patients at all:

Manager 2 said that an envelope should be given out with the forms so that staff can collect them. Team leader 7 said that it should not be the nurses’ responsibility to collect responses.

Alderton West meeting (field notes)

Data quality

The quality of data collected by the front-line teams was sometimes considered to be poor or incomplete and it was reported that in some cases clinicians complete instruments incorrectly or do not record activity (e.g. providing a patient with an educational trigger tool, or emotional support and reassurance provided). This obviously has an effect on the robustness of information available about relevant quality issues in the organisation. A data analyst manager in Dogwoodheath noted that the community nursing service was ‘probably not one of our top performers in terms of data quality’. A nurse in the same site suggested that one of the reasons for the hasty or incorrect completion of indicator data was the result of staffing shortages causing staff to feel under pressure:

We are so short-staffed . . . time’s being taken away from patients, filling these audits and quality measures in, we’re rushing them, so they’re probably not being filled in correctly. And then all our results probably aren’t accurate, which is then making us look worse than we are.

Dogwoodheath CNFG5

Another reason given for incomplete data being recorded during front-line observations was that computer systems required a time allocation for each task entered; a lengthy process. Nurses could either estimate the time spent or omit recording some care tasks they had carried out on the computer. This suggests that the full extent of nursing care given is not routinely electronically recorded. Managers, therefore, may not be aware of all the components of care given, so this is not taken into account with regard to quality assessment or measurement.

The possibility of erroneous data collection was compounded by the ambiguous wording of some indicators. An example that caused confusion for some nurses was the categorisation of PUs as ‘new’ or ‘old’ with a timing specified ‘within the last 72 hours’:

So are you supposed to record if someone has had a PU that is older than 72 hours or is it anything that just occurred in 72 hours or do you put that as an old one or is old technically if it’s happened in a different trust and it’s always a new one if it’s happened in our trust?

Beechbury CNFG1

Targeted efforts to improve data quality include offering relevant training. However, it appeared that staff do not always have time to attend training offered prior to implementing a new QI, due to competing workload pressures. It can therefore be impossible to release staff from front-line care for this purpose:

You plan for everyone to go but then circumstances arrive in each and every team that means you have to keep staff at base, doing the work, rather than sending them on training.

Dogwoodheath CNFG4

In a number of provider organisations, this situation had led to contractual breaches. In one case, the provider failed to meet a CQUIN target for training due to staff shortages and sickness and was subsequently financially penalised.

External factors, such as the patient environment, can have an impact on the successful implementation and achievement of QIs in the community. It was noted that, even where staff can attend relevant training, it usually occurs in the context of a training environment rather than in the patient’s home. The different conditions operating in these environments can make it difficult to apply apparently straightforward indicators during an actual episode of care:

You will get training on them [indicators] but it’s very different in a training environment, they will show you, well you do it like this, this and this, but when you actually get into the patient’s home . . . it’s never straightforward, and it’s really hard to determine actually, what I am writing where and what I am supposed to be writing, and how to classify and define things they are asking for.

Beechbury CNFG1

In all the sites, data were collected on a set day of the month for the audit of nationally mandated indicators, including those recording CAUTIs, falls and PUs. Observations conducted in Elmhampton on ‘NST day’ bore out evidence from focus group data across the sites suggesting this was a paper exercise where visits were conducted as usual and then, once back at the office, the paperwork was completed and passed to the administrative team to upload onto the national database. Only patients actually seen on the day of the audit were included in data collection, so that patients with ulcers or urinary tract infections seen on other days of the week were excluded, regardless of their clinical condition:

A patient had called to say his leg catheter bag had bloody urine in it so we visited to assess – the nurse wanted to prescribe antibiotics as he had previously nearly died with sepsis. GP said to wait, as no sign and symptoms of systemic infection. Therefore the safety thermometer returns did not show any patients with catheters with infection either. The nurse said virtually every person with a catheter would have a UTI [urinary tract infection].

Elmhampton (front-line observation notes)

If the audit was conducted on the same day of the month each time, owing to length of stay on the caseload, the same people might be included a number of times.

In relation to the FFT, another national mandatory QI, staff reported problems in obtaining feedback from patients. In the acute sector, once care is completed, patients are asked to complete the survey when they are discharged. The low level of discharges from the community nurse caseload where a patient is well enough to complete a survey means that, in order to meet a desired response rate, nursing staff might have to ask the same person more than once:

Yes, but we have to get a certain number done each month. So say like 30 per team. So we’ve actually exhausted all of the patients now because we have 113 patients in my team and we’ve, you know, questioned them, questioned them, questioned them with the same survey.

Cedarham CNFG4

During shadowing sessions in one case site, it became apparent that collecting data for this indicator was not a priority for the nurses:

FFT – done on paper questionnaire [in patients’ notes], meant to be done regularly but done ‘when remembered’.

Elmhampton (front-line observation notes)

Various methods were observed to be in use to ensure response rates. Front-line staff across the sites reported that their managers had tried introducing incentives or prizes to teams to increase distribution rates. Another strategy noted was to include the relevant forms in the patient records to be used at discharge. However, overall, an ad hoc approach seemed to be used, where QI feedback to managers filtered down to staff and a ‘drive’ for improved completion rates would then ensue.

Summary

In all the case sites it was clear that significant resources are invested in collecting and analysing data for quality purposes and monitoring providers’ performance against quality and KPIs. However, it was also clear that the current system can produce only a partial, and sometimes inaccurate, picture of how well the community nursing service is operating in terms of delivering high-quality care. Although participants reported occasional initiatives to assess quality in the service through methods complementing the use of the ubiquitous scorecards and dashboards, both CCGs and provider organisations lacked resources to undertake regular, comprehensive monitoring by other means.

Organisational systems and work pressures appeared to frequently present obstacles to clear communication with community nurses about the use of QIs, both prior to and after their application. Nevertheless, there were some examples of constructive action that could be taken to improve the situation. An irony of the current system was that, where contractual conditions for staff training were breached due to work pressures, the resulting financial penalties resulted in the organisation incurring further curtailing of resources, potentially exacerbating the situation.

The time required for quality data collection was another area which appeared sometimes to cause conflict between nurses’ responsibilities to the organisation and to their patients, in which case nurses seemed invariably and understandably to prioritise the latter. It was clear that the duplication of data, particularly through the use of both paper and electronic systems of record keeping, was an undesirable use of the nurses’ time, when considered from an organisational perspective. Nevertheless, it was also clear that maintaining paper notes to be kept in patients’ homes is an essential component of good clinical care and was valued by patients and carers.

Data from all the case sites portrayed a picture of a system in which significant effort and resources are spent to monitor service quality with results that, at best, arguably lack effectiveness and, at worst, exacerbate the pressures under which community nurses currently operate.

Contractual issues and benchmarking

A regional representative of NHSE recounted how the introduction of CQUINs for community services had not been well received:

When the first sort of guidance came out, community services were quite angry because it was seen as almost a penalty, that you were pulling money back and then you were reissuing it if they improved quality. So you know it wasn’t received terribly well.

R3

However, commissioners reported seeing the value of CQUINs as providing a focus on specific service improvements. Nevertheless, there was a perception that the application of CQUINs needs to change. Although the premise behind a CQUIN is that targets can be set and measured along a trajectory, it was acknowledged that it is not often that ‘you can ever distil anything down to be that simple’. It was reported that many CQUINs have been adapted to be made more qualitative and with broader scope than originally intended.

There were differing views among commissioners about how a provider’s perception of a CQUIN’s achievability affected their attitude towards it. One commissioner said that CQUINs can act as an incentive if the provider perceives them as achievable; in contrast, another commissioner said that if the CQUINs were too easy to achieve, the provider would tend to direct the CQUIN money to other services in the organisation.

Some commissioners tried to make sure that CQUINs achieved intended outcomes, even after they had been embedded in an organisation. In some cases, providers were asked to conduct a full analysis of associated outcome benefits for patients and the service. Providers complained about the fact that past CQUINs are usually included in the contract without extra funding. This results in the service having to meet an increasing number of targets while being unable to refuse to take on new patients, no matter how the community nursing caseload grows or how complex it becomes. This situation is compounded by the ongoing block nature of the community nursing contract, which is based on volume rather than the nature of the service being provided.

One commissioner felt that it was not at all clear if or how achievement of the national CQUINs was monitored by NHSE or their regional area teams, and cited instances of provider organisations interpreting national CQUINs idiosyncratically or adapting them to suit their own circumstances. In contrast to acute care services, contracts relating to community care services at the time of data collection did not contain national targets which could result in financial sanctions if not met by providers. However, similar to acute care services, in community care services commissioners could withhold payment to providers if certain conditions in the contract, usually related to timely submission of data, were not met. One commissioner doubted the effectiveness of financial penalties in improving quality of care:

You do need to be able to hold the trust to account, but penalties are not necessarily the best way to do that. But, nevertheless, they are formally in the contract and are potentially a contract lever.

Cedarham CM2

In one case site, the non-achievement of CQUINs led to negotiation concerning the resulting financial implications:

Three CQUINs have not been achieved, including attendance at information sharing meetings, as there was no evidence to support this. The provider business manager stated that attendance had happened, but had unfortunately not been minuted. The GP [chair] replied that the CCG would ‘take it on trust’ . . . the provider business manager also raised the issue of dementia training for staff, saying that the target had been set with all good intentions of supporting it. Due to recruitment patterns, targets in the early part of the year had not been met, but she assured the meeting that all staff had been booked on to training for the second half of year. So the provider appeared to have done well in this respect, but according to the letter of the contract, did not achieve the set target. GP suggested over-riding the initial decision to consider the provider in breach of contract in this respect, but the CCG director of quality governance said that she was unwilling to ignore her staff’s recommendation in such an arbitrary fashion. She agreed to review the situation.

Dogwoodheath meeting (field notes)

A number of participants bemoaned the lack of formal benchmarking systems for QIs. It was notable that many commissioners interviewed were not aware of the Pick List (the national database of CQUIN indicators that commissioners can use), or of the QIs database available at NICE. One commissioner specifically stated that it would be helpful to have such a website, so that commissioners could have access to it and adapt relevant CQUINs to their local circumstances:

Wouldn’t it be great if there was a database online, you could go in, you could say I want a CQUIN around pressure ulcers? . . . It’s not about copying them, because I realise that locally there will be different health economies and different situations.

Alderton West CM1

Provider managers also wanted to be able to benchmark their quality and performance against other providers of community nursing services. In order to undertake such comparisons, two case sites belonged to the in-house NHS Benchmarking Network (www.nhsbenchmarking.nhs.uk/). For all the sites, however, informal benchmarking was the norm, with individual managers contacting nursing organisations, for example the Queen’s Nursing Institute (QNI) and the Directors of Nursing networks, or engaging in ad hoc conversations with counterparts in other provider organisations. Although much of the information gleaned in this way was acknowledged to be anecdotal, the advantages cited of this interorganisational contact included the opportunity to learn from other providers:

We’ve borrowed several of the models and the ideas from [provider] that they’ve tried and that have worked. And we’ve talked to them about what the pitfalls were, et cetera, so from the perspective of sharing good practice that has been quite useful.

Elmhampton BM1

Impact of collecting data about quality

Some evidence of direct positive impact on care resulting from the application of QIs was apparent across the case sites. Areas of care reported to have been positively affected by the implementation of QIs included PU prevention, discharge and dementia screening. Positive results of indicator implementation were usually couched by managers in terms of QIs having helped to raise staff awareness of, and improve attitudes towards, relevant issues and changes in practice.

Some participants said that having indicators in place and monitoring them regularly were useful in terms of increasing transparency of services, provider accountability and confidence among both commissioners and provider staff in the quality of service being delivered. One commissioner felt that frequent reporting requirements (usually monthly) provided a mechanism for driving the quality agenda, and had proved to do so. A business manager noted the importance of QIs as being an inherent part of organisational activity, as he felt that without them, it would not be possible to have a clear picture of the service being delivered.

Despite the apparent lack of feedback about service quality in most of the case sites, some nurses were able to identify examples in which they had benefited directly from QIs, particularly in relation to the technology and documentation allied with integration. Teams spoke of being able to access information about their patients which they would previously have only known by word of mouth, a tool that can be seen to be particularly useful for night staff:

It’s [IT package 1] opened up now with [service X] and, you know, speech and language we can see, and a few other bits . . .

Yes, that’s just come in. And also for alerts, you know, for patients that obviously are, could be potentially abusive, or they’ve got a rabid dog, or you know, you’ve got a little alert.

Other benefits reported by the nurses from achievement of QIs were increased funding for district nurse training, new equipment for community nursing teams and extra capacity to assist with administration involved with Datix investigations.

The only time that we’ve really known was the, when the DN [district nurse] students last year got funded through CQUIN money.

Didn’t we get them [independent vascular services] nurses from some kind of target?

Dementia target, yes . . .

We had some equipment bought for our service.

We did yes. And wasn’t the mobile working supposed to have come from that, extra money somewhere . . .

We’re supposed to be getting some of them Doppler machines as well aren’t we?

A nurse in another case site was able to identify a successful outcome of Datixing:

We ended up getting another full-time member of staff because we actually put a Datix in to do with staffing. And our community matron phoned up and said ‘what’s this about staffing? I didn’t know you were short’ . . . You know, because nothing was being written down . . . We started putting in all our extra hours. And then we, luckily, managed to get our staff.

Elmhampton CNFG6

However, QIs were not always associated with obvious benefit and could sometimes cause unintended and undesirable consequences:

We suddenly then had to start visiting patients that were at risk of developing pressure ulcers far more frequently than we ever had done and it actually incurred another 1400 visits per year for the community nursing team, without any additional capacity to do that . . . other visits had to give way, so things that weren’t life threatening perhaps, like a continence reassessment . . . the work to achieve the CQUIN visit would take priority over that sort of continence care.

Dogwoodheath PM1

Reservations noted above about the introduction of new IT systems notwithstanding, it was thought that this had helped to demonstrate the pressures affecting services and had also enabled some community nursing teams to compare their own performance with those of others in the same organisation.

Perhaps owing to the reservations about validity and reliability of QI data and associated workload pressures, front-line staff were guarded in their responses to the usefulness of QIs being used in practice. Patients, on the other hand, were largely positive about the idea of nurses reporting back to managers on quality and the goals of national and local QIs currently in use, but were not always in agreement that they should be used as measures of community nursing service quality.

Nevertheless, nurses, patients and carers all spoke (albeit with some reservations) about benefits arising from QI application with respect to some clinical conditions, namely CAUTIs, PUs and falls. Several patient participants had indirect experience of people who had become acutely ill with a urinary tract infection and were aware of how people can become confused and their health can deteriorate, so considered it an important indicator:

Oh I think it’s a good thing because a kidney infection is a severe complaint. I’ve had them in the past. I have and I tell you something they are no joke.

Cedarham P2

However, some reservations were raised with regard to whether or not these could be prevented by good nursing care. One patient, who was a wheelchair user, commented:

I am not sure that’s nursing care or not to be honest with you. I mean for spinal patients it’s probably not, because a lot of them are in manual chairs which propel and their hands are touching a wheel and then you touch this and then you are bringing your own infection to yourself.

Elmhampton P1

Although agreeing that CAUTIs are an important negative outcome for people with urinary catheters, one carer focused on nursing activities, doubting whether or not the indicator in use provided a sufficient link with important nursing processes, that is, correct and timely catheter care and nursing assessment and monitoring to prevent infection:

I am not sure that highlights the quality of the care of the catheter as it does not indicate the bags have been changed regularly, it doesn’t indicate a record being kept of when the catheter was inserted and whether it was changed every 6 weeks or 12 weeks.

Beechbury C1

There was consensus among patients that preventing PUs is important for their comfort and safety:

Oh obviously yes, it is important for the nurses to know, of course it is, without a doubt.

Alderton West P1

However, some participants called into question the validity of the measure in relation to district nursing quality, due to difficulties in distinguishing some pressure lesions from moisture lesions and the lack of 24-hour nursing supervision of patients:

You can’t really blame the nurse for that, I wouldn’t have thought. They can’t be there 24/7, can they? And, as you say, if the patient is not feeling like moving, this can happen . . . But it’s – I suppose if somebody lives on their own and they’re not getting any help from somebody, other than the nurse coming in, I suppose pressure sores can develop. But, you know, I listen very carefully to the information I’m given and [husband] is asked to move every hour without fail and I try to encourage him to do that.

Alderton West C1

At an operational meeting for front-line nurses in Cedarham, a graphic illustration of this problem was discussed during a review of patients with PUs on the caseload:

The next incident referred to a patient who was chair bound but somehow went to an airport and sat there for hours, after which he developed an ulcer. The nurses felt that the patient needs to be co-operative – he has been given all the relevant information but the problem is how to change his personality and attitude. There is no particular learning from this case, but just to highlight the fact that nurses cannot do a lot if the patient is not co-operative.

Cedarham meeting (field notes)

There remains some debate around correct identification of PUs, with some wounds being attributed to the effects of moisture and, therefore, exempt from being counted as a PU.

Despite misgivings about interpretation of wording, the focus group evidence from nurses suggested recognition that collecting quality data, with respect to falls, can lead on to a full falls assessment for the patient. This was considered to be important, as falls can be a sign of deteriorating health and nurses would not necessarily routinely ask patients if they had fallen in the absence of the indicator. Patients agreed that an indicator relating to falls is important and that nurses would not otherwise be aware if they did not ask the patient:

She slipped out of the chair. They think she slipped out of the chair, we don’t know whether she tried to stand up or what she did but I think she just slipped out of her chair because her blood sugars weren’t very good yesterday. We got her back into the chair but I don’t think the district nurses would be aware that she slipped out of the chair and I haven’t told them . . .

Beechbury C1

To patients, the important part of the falls QI is what is done with the information:

Yes I think it is. It hasn’t happened to me but yes I think that people who are elderly, stumbling, falling – yes it should be noted, recorded and perhaps passed onto the doctor, information.

Elmhampton P1

And I think they should be making a phone call to check. It’s not going to take a lot. I know phone calls do, but you know, it’s caring isn’t it? And it’s ideal for, you think, oh that was nice of them . . . Well it’s following it up, isn’t it? It’s following up. And they, it needs to be followed up.

FGC1

Limitations of current indicators

A common criticism of indicators in use was that they focus on counting activity, rather than recording quality processes and, as such, do not provide an accurate reflection of the quality of community nursing on their own:

What we don’t have is an opportunity to actually record [information] . . . you might be going in to do medication but, actually, a lot of your time might be spent trying to talk the [alcoholic] patient round into having their medication and maybe not having a drink today, and you can’t record that.

Alderton West NTL2

A lot of the indicators in use focused on measuring quality by counting things that could be counted. This is unsurprising, as some participants said that process measures (e.g. number of referrals or number of visits) are easier to measure than measuring quality of care. One provider manager used the example of community nurses being required to have regular meetings with GPs; however, data were only collected about the number of meetings attended, rather than on any outcomes for patients arising from them.

The danger of the ‘counting’ approach was articulated by a national lead who was aware of criticism that some CQUINs seemed to be a box-ticking exercise without any concrete benefits resulting from them:

Some people I go up to talk about, oh, it’s great because it gave them money to do X, Y, or Z, and other teams are quite cynical, that they’ll do all this work and then they don’t see any benefit from it, so I think it’s mixed.

N1

There was a perception that patient experience can be assessed through patient surveys, and managers in most sites reported that the collection of patient experience data was good and showed general satisfaction with care. However, it was also noted how difficult it can be to formulate appropriate wording and questions. Furthermore, when a patient is unable to complete a survey independently, it was also recognised that results can be affected by whoever helps the patient with this task. A further issue relating to this process was the staff perception that patients would be afraid to complain:

A patient can be thinking, ‘OK, do I really be really honest?’ Because they’re still on the caseload. So ’if I was to write something really bad, is that going to impact on my care that I’m going to receive afterwards?’

Elmhampton CNFG7

This observation was borne out of evidence from patient interviews and calls into question the validity of the survey with community patients:

Probably one of the best ways is by questionnaires and questioning the people who are receiving the service and getting anonymous feedback where people can feel that they can write down exactly what is going on without thinking ‘I don’t want to’, as people are always frightened of complaining in case it reflects on the care that they get later on aren’t they?

Beechbury C1

Misunderstandings as to the purpose of such forms can also arise. One patient said that being surveyed to ask if she would recommend the nursing service to friends and family sounded as though the service was marketing itself, when she perceived it not to have sufficient resources to expand:

I can remember having one leaflet, whether it was for the district nurses or not I wouldn’t like to say but I am a bit sceptical on that, because why are they sort of trying to advertise that when they are having a job to keep up with, what they are doing? [Laughs]

Elmhampton P2

The limited use of QIs in some clinical situations was demonstrated during a shadowing session with front-line staff in Alderton West:

Brief discussion about quality indicator for the service – admissions avoidance – difficulty showing effectiveness with this indicator since admissions were not being reduced overall – but thinks this is because profile of caseload increasingly old and ill with complex comorbidities.

Alderton West (front-line observation notes)

How to measure quality in community nursing

It was generally acknowledged that quality in community nursing care is difficult to measure, partly owing to the nature of the service:

It’s difficult to monitor it [community nursing] because it is out and about and delivered in patients’ homes . . . by its nature it is quite hidden and it is difficult to assess how good that’s been without relying on your patients to tell you really.

Elmhampton PM3

This situation appears even more complex when viewed in the context of other national policy requirements. One commissioner explained:

It’s certainly made monitoring [even] things like workforce trajectories, which should be very, very simple to monitor, quite challenging because you’re looking at core workforce plus BCF workforce plus whatever an individual CCG might be commissioning for an area. So if you’ve got a pool of nurses for one bit and a pool of nurses for that bit, do they still count for that bit, and how do you measure things?

Beechbury CM2

Both commissioners and providers thought that moving towards outcomes-based commissioning would enable a more sophisticated assessment of quality. It was generally thought that there is a need to use a mixture of narrative and hard metrics and to focus on collecting more in-depth data about smaller numbers of patients/incidents in order to gain a better assessment of quality in community nursing. Interviewees gave examples of some initiatives implemented to improve quality by focusing in more depth on relevant issues [e.g. the inspection of case files by community practice teachers in Alderton West (see Chapter 6)]. A similar focus on using case files as evidence of the quality of community nursing care was found in Beechbury:

In order to demonstrate effective multidisciplinary working the organisation has to present a case study to the commissioners – the community matron has prepared a case study of a patient with whom she worked who, previous to her involvement, was a frequent 999 caller and GP call out patient . . . Since her input reviewing medication and referrals involving the district nurses and other members of the team to monitor concordance with medication, has had no 999 calls or GP call outs . . . Evidence of monthly MDT meetings attendance is also required for this CQUIN.

Beechbury (front-line observation notes)

Examples cited in other case sites included multidisciplinary mortality reviews in the case of sudden unexpected deaths and learning networks open to all staff (including those in clerical and administrative posts), where key information from the provider’s quality and safety group is presented and discussed.

Some participants suggested that differential targets may be required for measuring quality in areas of socioeconomic deprivation or with differing clinical conditions. Different types of service delivery could also require different outcome measures, for example a rapid response service to avoid acute admission compared with a service supporting patients living with long-term conditions. One commissioner stated that a balance needs to be struck between standardised quality measures and allowing providers to have the flexibility to improve services as long as they can show that overall outcomes are met.

It was felt that community nursing QIs should have a holistic focus (looking at all factors that may affect an individual’s recovery), measuring real health outcomes instead of counting interventions that may not necessarily result in healing. A national lead emphasised the need to make the most of every nurse–patient encounter:

But if you are a community nurse for example, just going in and saying oh yes, here is a leg ulcer, that’s what it measures, that’s the dressing that we put on it. But actually in that space of time with the individual who might be sat there with a glass of beer on the side, with an ashtray full of cigarettes, is clearly overweight and is not doing any exercise; the quote for me is, well what next? So within that space and time that the nurses are providing that clinical intervention for that particular problem, actually where is the intervention in terms of okay, what other things do we need to address in order to help you heal?

N2

Many ideas about increasing the usefulness of QIs were offered by nurses and patients. QI documentation could be useful as an aide-memoire for good-quality patient care and those that incorporated training requirements could enable nurses to access staff development. Retention of QI information on IT databases that would not previously have been stored, such as venepuncture data, was considered to be helpful for other purposes such as providing evidence to build a business case for more staff or efficient streamlining of Datix investigations:

I think if they can pull off what they’re pulling off, it’s great isn’t it? Because that’s a form we’re not filling in.

Elmhampton CNFG2

When indicators were able to indicate effectiveness, these were viewed as especially useful:

When there was the CQUIN . . . You could see they had like safety crosses, didn’t they? You would see when you got a PU in there, and at the end of the month . . . You would see that they’re coming down. So that was a good way of saying that yes, there was one, but we’ve put this in place and they’re now healing. And you could record healing or, in hospital, or static.

Alderton West CNFG4

Quality indicators could also be used with patients to demonstrate effectiveness of care. In the patient focus group, one participant with complex comorbidities talked about setting milestones or goals in relation to measuring progress with care and how difficult it was to say whether or not she was improving owing to the long-standing nature of her illness. The use of small goals such as those used by physiotherapists was thought to be a good use of indicators that could be applied in nursing too:

I think that’s, I think it’s good to have and we can prove it because we’ve had goals and we’ve got there, we’ve done it and you get a good sense.

FGP1

Suggestions made by nurses regarding how indicators might better measure the quality of their work involved the inclusion of ‘softer’ indicators alongside the numeric outcomes that are largely used. Nurses do see the value of quantitative measures assessing effectiveness, but suggest a balance needs to be struck. When asked how they measure the quality of their work themselves, they spoke both of direct feedback from their patients and of healing:

What I find anyway is patient feedback, you know, they will say oh that’s really comfortable if you have done a leg dressing or just an injection and oh I didn’t even feel that go in or you know, something like that, that is where I get the majority of my feedback from on a day-to-day basis anyway . . .

I think the fact that you know, if it is a wound or a leg ulcer, the fact that it’s healing . . .

In addition to their own input into QIs, nurses felt that if others involved in the decision-making were to observe them occasionally in their work, this would inform the selection of appropriate indicators. Instances were given of this happening to an extent and were welcomed:

I think it is difficult, isn’t it, when you’re sat in an office to visualise something, isn’t it, and that’s not your background. We’re really pleased that they’re coming out really . . . Because they’re commissioning the services, aren’t they, and it’s nice for them to see what they’re commissioning. The practicalities.

Elmhampton CNFG2

Patients also suggested that managers should go out on visits with staff to assess service quality and that this would be a valid means of both increasing their understanding of the context for care and reporting back on service quality:

I really can’t think how it could be done, other than spot checks.

Alderton West C1

They would have to go around and check them, because, if the head one went around and checked they would see what happened to the people.

Cedarham P1

From the perspective of both patient and front-line staff, it was clear that having sufficient time to spend with patients was important to both nurses and patients, and thought to be a fundamental characteristic of a high-quality service. The length of visits observed during the shadowing sessions ranged from 5 to 95 minutes. There was a notable difference between the average duration of patient visits by senior nurses and specialist community nurses (35 minutes) compared with band 5 district nurses (23 minutes). It was apparent during shadowing sessions in Beechbury and Alderton West that the former enabled a more holistic approach to care, which patients and carers appeared to value. A patient with a young family with a potentially life-limiting condition commented:

She sits down, she has a chat with us and she is straight with us, she is just so nice. It is hard to explain, it is just nice to see her . . . As I say she does check me, I mean she always checks me for everything, you know.

Alderton West P1

Many people receiving community nursing services live alone or may spend most of the day on their own. District nurses aspired to be less task focused during home visits and patients clearly valued an opportunity to have a conversation during a visit:

So you know, if you can’t speak, spend time to speak to a 95-year-old you shouldn’t be in community nursing is my kind of motto.

Cedarham CNFG3

One patient with partial sight and mobility problems expressed succinctly the loneliness and isolation of many people confined to their homes:

Some people have people that come in, some people don’t have, . . . and they wait for the nurse to come, so they could [have a] little talk or a little something, because they’re not just there to give the injection of the medication, sometimes a little bit of talking helps them to go along, help them along the way.

Cedarham P1

Patients frequently valued the ‘willingness’ and kindness of staff delivering the service, making the difference to their perceptions of care quality. It was the staff who were able to have a friendly conversation and ‘know’ them as a person during the course of care delivery who were most appreciated:

I want somebody who is intelligent and who knows the job so that they have the professional capacity, but they are human and caring and you can talk to them on an everyday level about anything, you know. Because we don’t talk about medical things we talk about the garden or anything and that just makes you feel you have a connection with the nurse.

Elmhampton P1

These quotations highlight the importance not only of nurses having sufficient time and the willingness to develop relationships with their patients, but also of including this aspect of care in quality measurement.

Summary

The study data showed that, in the wider context, an unavoidable consequence of the intention to promote flexibility and autonomy for CCGs in terms of QI selection and application was the lack of benchmarking opportunities. Commissioners and provider managers did not feel that there were any easy ways to compare the quality of the care provided through their services with that of other care organisations. These data painted a picture of individuals responsible for quality in the case study organisations having to operate ‘blind’ to a certain extent. There was also some scepticism expressed about the value of the CQUIN system per se.

It was acknowledged that, owing to the nature of the service, it is difficult to measure quality in community nursing. However, participants agreed that current indicators in use were helpful to promote the transparency of the service for commissioners and provider managers, and also brought some benefits to front-line staff in terms of streamlining work systems, particularly with regard to accessing information about their patients. It was also agreed that areas of care addressed by current indicators are important to monitor (e.g. PUs). Nevertheless, it was also reported that QIs were not always associated with obvious benefits and could sometimes cause unintended and undesirable consequences for both the front-line staff and patients.

A significant issue in this regard appeared to be the fact that many indicators focus on counting activity, rather than on linking outcomes to quality processes, and as such do not provide an accurate reflection of the quality of community nursing on their own. Patients, carers and front-line staff shared similar priorities in relation to quality, emphasising the importance of there being sufficient time to offer person-centred or individualised care. A common view among participants was that effective assessment of care quality should entail the use of both hard metrics and narrative; it was noticeable that in most of the sites, there were ongoing attempts to establish processes through which to collect quality data about the ‘softer’, less easy to measure aspects of the service.

In addition, a drive towards using outcomes-based QIs could be more closely aligned with patient-centred care to demonstrate the effectiveness of the service. A few participants suggested that the use of small goals, tailored to the individual (such as those used by physiotherapists), could be appropriate indicators of care quality in the community nursing service.

The study data suggest that the current system of measuring quality in community nursing, although of some value to all stakeholders, has only limited effectiveness, both in relation to representing service quality to planners and managers and in representing the nursing values incorporated in the 6Cs,3 which are important both to users and those delivering front-line care.

Fragmentation of care

Given the current lack of integrated or ‘joined-up’ care, being a patient in the community has become increasingly complex with the proliferation of different care providers. There was a perception among both commissioners and provider managers that service restructuring has led to a task-focused service where there is very little continuity of care for patients:

The other thing is that the service has become very task focused, when historically you would have expected it to have undertaken a holistic assessment of everybody’s needs . . . The CCG quality manager said that she thought that having smaller teams with case load responsibility and which are well led is crucial.

Beechbury meeting (field notes)

In contrast to this ideal, models of care in the case sites entailed community services being provided by a number of different agencies: some specialist and time limited and some not; some funded and some having to be paid for by patients; equipment can come from different sources or be provided privately. Patients and carers often spoke of the responsibilities for caring for people with complex, long-term, often deteriorating conditions over many years and the problems navigating health and social care boundaries and, for some, managing and worrying about the financial implications of care choices. In the example below, a patient who used a wheelchair related her problems in this regard:

. . . We have got savings, so we have to pay until the money dwindles, which is hard, because it is expensive . . . I am paying about £800 a month in care and the State are paying me about £280 a month. So that’s about £500 a month extra I have to find . . . to have more care or more help around the house . . . I cut it down to basics because we can’t afford to do everything.

Elmhampton P1

In the course of patient interviews it was apparent that several key aspects of health service care important to, and having an impact on, community patients and carers were not picked up in relation to nursing quality measurement, for example transitions between care, continuity and the quality of collaboration or shared working between nurses and agency carers or district nursing and staff in primary care. The patient quoted above also related her problems to getting someone to syringe her ears as requested by her audiologist. She was unable to access the GP surgery easily owing to her immobility and the practice had refused to send their practice nurse, as district nurses were visiting her. A district nurse had syringed her ears without good effect so the patient had to pay someone privately. Soon afterwards her ears had blocked up again:

I am hoping that one of the nurses will syringe my ears. I have been to the audiologist and he says that they are fully blocked and he can’t adjust the hearing aid until I get the wax removed. We had all this back in May and one of the girls did syringe my ears but it didn’t clear and so I had to go to have it done privately . . . That was only in July, so I don’t really want to have it done every 2 months at the price he costs but there is only one nurse on duty who can do it.

Elmhampton P1

In another example, a patient with a PU on discharge from hospital received care from a number of teams visiting her home, but there was disagreement as to who should be providing what care. A specialist enablement team came for a time-limited period, and then district nurses worked alongside agency care staff to deliver personal care:

It was more like a mini row about who was going to do it [check pressure areas] and I thought ‘if I was sat there and in [patient]’s shoes listening to this I wouldn’t be very happy. I would be thinking I am a burden and arguing over who is going to do something for me’; no I wasn’t very happy but it was sorted out; it was finally agreed that the carers would put it [Intrasite Gel, Smith and Nephew, London, UK] on as long as the district nurses double checked the area at least twice a week. But it does seem to be quite, what’s the word, it’s not set in concrete who’s doing what when.

Beechbury C1

One effect of fragmented care provision is that there is little chance for patients to build a trusting relationship with community nurses; this important aspect of nursing care is not considered in relation to quality outcomes:

You can’t communicate with people that you don’t see often. I could communicate far more . . . the [private] carers understand me; I understand them. They know my standards . . . and there’s a real relationship there and caring. Caring. The care from those two girls is awesome. So, obviously, we have a relationship. With somebody who comes in just once a fortnight [like the community nurse], well, you don’t develop a relationship on those terms.

Elmhampton C1

Negotiation

There was broad agreement that integrated services would be beneficial for patients. In this context, communication and integration of the community nursing service with GPs and other related services (e.g. care homes, out-of-hours services, social care) emerged as a main theme. In one case site a unified contract had been developed for cardiac care, irrespective of where the patient was being treated.

In Beechbury, negotiations were in process to develop joint indicators. However, these were proving difficult to conceptualise and required a lot of time for planning and preparation. Key issues identified included a lack of streamlined processes and conflicting priorities:

The chair [CCG] said that he was still struggling with the concept of the shared CQUIN, and asked if the CQUIN could be to actually develop a CQUIN with shared risk. The CCG head of quality said that the providers would be expected to develop a framework within quarter 1. The trust manager said that providers would need a whole year to develop an effective plan for working together. The CCG director of nursing said that that was not acceptable, they cannot have a year without a product, as they cannot justify this spend of public money.

Beechbury meeting (field notes)

It was also noted that negotiating joint monitoring of quality could be hampered by advantages accruing to different provider groups from current financial systems (e.g. incentives for GPs to develop care plans for patients). Other difficulties concerned variation in different organisations’ ways of measuring quality, which mean that it is not always possible to compare quality measurements directly for the same area of care; an example given was the different systems used to measure PUs in a community nursing service and in the local acute provider:

We don’t always know if we’re comparing like for like, and people measure things differently. We saw this around the pressure ulcers . . . one of our local acutes had an extra category, so we weren’t comparing the same things they’d introduced themselves.

Alderton West PM4

There was widespread recognition of the difficulty in aligning quality monitoring systems across health and social care, given the differences in the way quality is measured in the two services. These differences will need to be smoothed out if service integration is to succeed, a task recognised as requiring a considerable amount of time, effort and thought.

Accountability

Correct attribution of a quality measure is problematic in a situation where more than one service provider or carer is involved. Some patients and carers reported living with the uncertainty, contingent on lack of clarity concerning the allocation of responsibility for care:

I’ve had to keep an eye on the carers here and say to them you need to change the bag, weekly and the night bag weekly and I don’t, the district nurses have told the carers that they must do it and it’s down to the carers not the district nurses. The district nurses can only get involved in catheter care when the catheter becomes blocked or there is a problem with it.

Beechbury C1

Commissioners and provider managers were concerned about the inevitable risk involved when quality measures are dependent to any extent on other organisations’ performance or processes and accepted that monitoring QIs in integrated or collaborative services would be difficult. This was evident at a meeting in Beechbury where the development of their proposed joint indicator was being discussed:

CCG manager 1 said that there is support in principle among all parties [Community Services Provider, City Council, Mental Health Trust] for a shared CQUIN. However, there appears to be some anxiety about who will carry ultimate responsibility for delivery of such a CQUIN.

Beechbury meeting (field notes)

Individual commissioners were also mindful of provider concerns in this regard:

The measures that you put in place have to be measures that have the ownership of the provider that is within their control to measure. So you can’t ask them to do something that relies on somebody else measuring it for them.

Dogwoodheath CM2

In a number of case sites, nursing team leaders (not involved in ongoing negotiations about joint indicators), also raised the problem of responsibility and accountability for meeting data collection targets if they have no control over many aspects of the process:

Many of those at the meeting felt that this CQUIN is quite problematic, as there is a new system of care co-ordination which cuts across organisations and which also involves local authority functions – this means that provider staff do not have control over many aspects of the process.

Dogwoodheath meeting (field notes)

Good communication between all organisations involved was thought to be crucial.

Despite the drive towards joint working, persistent practical problems reported included issues of confidentiality and incompatible computer systems. Currently it is only possible to share very limited data across organisations: patients have to approve sharing of their personal and clinical data in many cases. As noted in Chapter 4, general practices, community nursing services and social care services all have different computer systems, making widespread easy electronic communication impossible.

Impact

The data showed that community nurse interactions with other health-care professionals such as GPs, hospital and care staff can have an impact on implementation and achievement of quality standards. An example was given of patients being transferred to district nursing caseloads from hospitals without the required continence assessments:

Most of the INCO [incontinence] assessments are from the hospital . . . but they expect us to do it . . . It is time-consuming because you have to send in your 3 day fluids chart first . . .

And the wards know they are meant to be doing that . . . because the patient has been in a bed, and they’ve been in there for over 3 days. It is their responsibility as part of their assessment to provide a level of quality care . . . knowing why is the person incontinent.

A similar issue was discussed at a quality monitoring meeting in another trust:

Other issues discussed included the necessity to do targeted work with nurses and physiotherapists involved in patient discharge from acute services, as hospital staff appear to lack understanding about community services/facilities. The GP asked about the role of care co-ordinators. A general discussion ensued, with the provider director of quality highlighting the complexity of the service, and the CCG assistant director of quality saying that the issue was on the table for multidisciplinary action.

Dogwoodheath meeting (field notes)

Nevertheless, the data revealed that nurses were mindful of promoting good relationships with other professionals:

GP request to ensure patient attends outpatient appointment following admission for accident and surgery . . . Nurse discussed importance of responding to GP requests in this sort of case – building and cementing good relationships – GPs appreciate them and are always available when required. The nurses are based in the same building as one of the three GP practices they cover and this facilitates good relationships.

Alderton West (front-line observation notes)

In Beechbury, similar good relationships in some areas of cross-organisational practice had been noted, although other aspects of the organisation of care still needed to be addressed:

QM [quality manager] CCG reported that she had undertaken some scheduled visits to three neighbourhood teams in one area. She had identified really good partnership working, including GPs, psychologists and school nurses. In particular, staff feedback indicated improved relationships with GPs. However, the intermediate care team [ICT] was not well integrated and felt separate from other organisations and teams. Estates need to be reviewed, as ICT members feel separate and isolated.

Beechbury meeting (field notes)

Collaboration across health and social care appeared still to be largely underdeveloped. Joint assessment of patients that depended on arranging meetings with social care staff and completion of requisite documentation to meet reporting targets created particular difficulties for community nurses:

That was a few years ago now, but the, you know the target was 21 days you had to do the assessment, get all the information and all that. But you often couldn’t get, you know it’s got to be a joint piece of work and you couldn’t actually get the social worker and all that, so it would go on and on.

Elmhampton CNFG2

However, joint working between different practitioners was not the only factor noted in this regard. Issues affecting one organisation involved in providing care could also impact on how staff in another organisation function, as evidenced at a meeting in Elmhampton:

The community manager reported that the local authority [LA] are in significant financial difficulty, and are putting in place strict criteria with regard to funding, which will impact on the services in the community. There is ongoing communication between the trust and the LA, and this issue will need to be flagged up to the CCG.

Elmhampton meeting (field notes)

In Cedarham, it was noted that, when short of practice nurses, some GPs assumed that the care for which they were responsible could be provided by district nurses:

Relationships with GPs – also trying not to lose money through not achieving targets – see DN [district nurse] service as a way to ensure this does not happen.

Cedarham (front-line observation notes)

Despite ongoing problems in establishing integrated care systems, participants accepted that QIs for community nursing have to take account of collaborative care provision, recognising that quality in community-based care depends on a number of interrelated services. When asked about the effectiveness of existing QIs for community nursing, national and regional participants from NHSE suggested that the focus on quality in community nursing should move to an integrated care approach:

So I think when we talk about quality indicators for community nursing, I think what we need to think about are quality indicators for people who are cared for in the community and look at it from an individual’s point of view, rather than siloing the nursing service, because . . . some individuals have 20, 30 people coming into contact with them, which is not good, so it’s about the experience for that individual, and I think that’s where we need to focus the quality.

N1

A regional participant from NHSE made a similar point:

I think CQUINs, if we really want to improve patient care, I think it’s very rare you’re going to get CQUINs that are just going to involve one organisation. So I think for CQUINs to really work, but it’s really difficult to do, because that’s actually how do you incentivise that part of the patient journey? But actually I think for something to work, to collectively come together. So, say the ambulance service, community services, and acute, let’s come together to develop some really good, tangible CQUINs, would really improve quality.

R2

The development of joint metrics across services involved in integrated care is still in the very early stages and anticipated outcomes are often system outcomes. However, in two case sites it was reported that a joint indicator had helped drive collaboration between services in different sectors, although with variable results. In one site, an indicator requiring the community nursing service to work with adult social care, primary care and members of the voluntary sector where patients were at risk of hospital admission had resulted in improved joint working across the different organisations. However, in another site, where an indicator focused on sharing care plans with primary care, the community nurses felt that they were being challenged to try and find more patients who fit the criteria than were actually on their caseload, in order to meet the targets set.

Blanket implementation of QIs could also cause unintentional conflict with wider MDT partners if there had not been sufficient communication. One team member spoke of a blood sugar monitoring CQUIN that they had had to implement:

So random capillary glucose monitoring. And if that was abnormal you’d do an HbA_1c [glycated haemoglobin], or you’d be liaising and faxing the GP with all kinds of results and the GP would then have to act upon it, and most we found that 80% of the GPs knew nothing about it . . . And when we were faxing GPs with the results, their fax machines were burning out and breaking down because about four million faxes going through every day, because the amount of visits we do and the amount of abnormal blood sugar levels you find, the system would crash . . .

Alderton West CNFG5

Summary

All the case sites were addressing the issue of providing community nursing care in collaboration with other organisations, with widespread recognition and acceptance among the study participants of the need for effective quality measures of integrated care. The problems arising for patients due to the current fragmentation of care services served to emphasise this need.

However, it was also widely recognised that measuring the quality of integrated care brings its own problems. Key issues identified include lack of streamlined processes, including IT, conflicting priorities and agreeing appropriate allocation of responsibility for meeting indicator criteria. Unsurprisingly, joint indicators across organisations seemed difficult to conceptualise and to require a lot of time for planning and preparation.

Of particular interest in this context was that, when compared with the rest of the sample, the one vanguard site in the study appeared to be struggling with very similar problems in this regard. A major factor affecting the development of joint working appeared to be the ongoing resource and staffing difficulties experienced in all the sites, which made transforming existing practice conditions and organisational systems, while continuing to provide effective patient care, challenging.

Analysis framework

Consideration of the study findings in relation to Franco et al. ’s framework63 has allowed exploration of how different factors influencing quality in community nursing relate to one another. We discuss these issues firstly in relation to factors within the wider health systems, categorised at the outer level of the framework (see Figure 2), moving sequentially inwards to end with consideration of conditions affecting community nurses as individuals.

Wider health systems factors

The main findings from the nationwide survey of CCGs revealed that in 2014/15 there were still few providers of community nursing services outside the NHS fold. Social enterprises and private providers constituted < 20% of community nursing service provision working with CCGs in our sample. Nearly half of community provider organisations were serving more than one CCG, which can arguably be assumed to contribute to a challenging workload for these organisations.

The focus on incentivised QIs in this survey suggested that the main objectives of commissioners in 2014/15 were to use the CQUIN scheme to develop community nursing organisations in order to meet the aims of the BCF, that is to provide care closer to home and outside hospital wherever possible, within a more integrated service provision. No appreciable differences appeared in relation to the numbers of CQUINs or overall goals of these in respect of provider organisational type, though within these categories there were small differences in terms of local priorities. There appeared to be a preponderance of QIs focused on ‘process’, that is, the way services are delivered, as opposed to outcomes, suggesting commissioners and providers accepted that a link to improving the way in which community nursing services are delivered will lead to improved health outcomes.

It is clear from the findings of both the survey and the case study that wider health systems factors affecting community nursing services and those delivering or receiving care arise from reorganisation of structures at both the national and local level following the passage of the Health and Social Care Act 2012. 26 The creation of NHSE, and its role in mandating national QIs for adoption by CCGs, appears to have had some unforeseen consequences (e.g. the rolling out of indicators from hospital to community settings). There seems to have been little consideration of possible variations in methods and context of care delivery in relation to different settings and possible effects on the usefulness or appropriateness of relevant indicators; our study findings show that the value of some of the indicators in use in the community are questionable in this respect. The lack of comprehensive benchmarking opportunities for both CCGs and providers compounds this situation, as there is no national forum for relevant discussion.

The diversity of health service providers resulting from the Health and Social Care Act 2012,26 namely NHS organisations, not-for-profit bodies and private companies, has resulted in increased competition between community providers. Although this was a stated aim of the Act, our findings have shown potential difficulties for NHS community providers. Private providers or larger providers can take advantage of scales of economy and existing infrastructure, which many fairly new entities, such as social enterprises, do not have. Following implementation of the Act in 2013, the financial reins are now held by CCGs, who are responsible for allocation of funds to providers. There is no system for matching funds available to service demand in community nursing; in none of our case sites had the providers been successful in negotiating a different basis for payment for community nursing services, other than the standard block contract. This factor, combined with the drive to offer care in the community to patients with increasingly complex conditions and the obligation of the community nursing service to accept all referrals, has resulted in a situation where the service appears to be struggling to provide care which truly meets patient need and, at times, borders on being unsafe. 75 The situation is further compounded by national problems concerning recruitment and retention of community nurses. 75 In this context, the requirement for providers to meet criteria for QIs appears at best to have only a limited effect and, at worst, may actually impact adversely on the quality of care that community nurses are able to deliver.

Although all parties involved appeared to welcome the move towards establishing integrated care structures for community-based patients, our findings show that difficulties concerning logistics, communication and culture change between and within organisations persist; none of the managers interviewed offered any concrete suggestions about how to effect necessary changes. These problems are not restricted to our case sites. Recent literature highlights similar issues in many health and social care organisations, with the lack of resources and time for the creation of suitable interorganisational infrastructure appearing to be a key issue. 76–78

The assignment of financial control to CCGs has also resulted in their having considerable power to determine priorities for care at a local level. The range of local indicators in use across our case sites illustrates both the commonalities and diversity of need operating in different areas. However, our data show that, despite stated intentions to the contrary, providers are often bound by CCG perspectives, rather than the realities of service demand and logistics, of which commissioners appear to be sometimes unaware. Nevertheless, in our study there appears to be little doubt that providers and commissioners are focused on collaborating constructively for the benefit of patients, despite system shortcomings. Methods in use for monitoring and reporting quality, particularly dashboards providing detailed records of performance against targets, can be useful for creating confidence at a managerial level in both commissioning and provider organisations. However, their effectiveness as catalysts for care improvement seems debatable.

Summary

Provider organisations’ explicit goals have had to change and align with current national and local agendas, as interpreted by commissioners. Although providers have always aimed to provide good care for patients and their families, the service now appears to be significantly focused on meeting targets related to relieving pressures in the NHS acute sector and which are of dubious validity in relation to provision of high-quality nursing care in the community. As such, the values implicit in a culture of setting and meeting targets appear at times to clash with those of front-line nursing staff, who consequently feel little engagement or ownership for QIs currently in use.

As professionals, the front-line community nurses expressed concern about the effect on care of some QI targets. The difficulty of measuring ‘softer’ aspects of care appears to prevent recognition of the true quality of the service provided. Many patients and carers consider these aspects of care to be as essential as nurses’ clinical and technical competence. Their obligations as employees to collect data for QIs can at times compromise nurses’ professional autonomy.

The current climate of staff shortages and ongoing logistical problems, which appears at times to be exacerbated by the need to collect data for quality measurement, has contributed to high levels of sickness among community nursing staff. This situation does not bode well for the capacity of the service to provide consistently high-quality care to patients on the community nursing caseload.

Although it is clear that all of the organisations in our study site are striving to improve and measure aspects of care quality, there is some debate about the degree to which they are succeeding. The Framework for Commissioning Community Nursing53 stipulates that quality entails ‘high-quality care’ delivered by a workforce composed of ‘the right people with the right skills in the right place at the right time’ (p. 16),53 and the development of ‘effective relationships’. Given the climate in which all community nursing service providers are currently operating, it is unclear how these aspects of quality are to be achieved.

To validate our analysis and findings we took some of our main findings to a series of 10 stakeholder workshops across the country, where they were discussed in the light of people’s experiences locally (see Appendix 2 for a full report). These discussions consolidated the development of good practice guidance (see Appendix 4).

Discussion

In Chapter 9 we collated the findings from the various data streams collected over the course of this mixed-methods study and considered our data in relation to a framework that identified how wider contextual factors interplay with the processes around quality measurement in community nursing. 63 This chapter picks up some of the significant findings from our study and discusses their implications for the use of such measures in community nursing service quality assessment. It concludes by suggesting how quality measurement may be improved for the benefit of patients and carers.

Study strengths and limitations

The choice of a mixed-methods design incorporating a survey and case study has enabled the team to investigate contemporaneously and in depth, the processes for selecting and applying QIs in community nursing quality assessment. 95 Despite the initial problems of obtaining data from the new CCGs, our survey succeeded in obtaining a 69% response rate, providing good strength of evidence of the national picture in relation to variation in organisations providing community nursing and the number and range of local QIs in use for community nursing in 2014/15. Inferences made about suspected workload involved in respect of selecting and monitoring quality measures based on the survey findings were able to be explored in the case study. The use of five health settings has facilitated comparison of findings across different types of community services provider organisations. Our within-case analysis suggested very little variation in contextual characteristics impacting on how community nursing service quality is measured within the different organisations. This is possibly due to the strong policy-drivers from NHSE for delivery of care closer to home creating some turbulence by pushing forward organisational change and increasing integration in all the sites, following on from the implementation of CCGs in 2013. Although the generalisability of findings from case studies could be argued to be limited, this study has incorporated triangulation into its data collection methods by conducting front-line and meeting observations, which can both inform and validate our interviews conducted with participants in the field. A dissemination phase enabled us to test and validate our findings with stakeholders from around the country, thus reinforcing confidence in our findings.

We purposively selected case site organisations from scrutiny of the findings of the survey, and consequently there may have been some differences between those sites that agreed and those that did not, which might have influenced the case study findings. Two sites we approached initially seemed to agree to participate, but in one case took so long to provide requisite information for the NHS permissions process it had to be abandoned. In another potential site that included a private provider, despite a great deal of enthusiasm from its nursing lead, we were unable to recruit as, at the final stage, a high-level decision meant that the site suddenly withdrew. It is possible that market sensitivity and concerns about confidentiality were behind the decision to withdraw. This experience highlights a number of issues that need to be considered with the wider involvement of private providers in this area of provision. First, claims of the need for confidentiality of commercially sensitive information may make it increasingly difficult to scrutinise the activities of companies in order to understand lessons from which other organisations can learn. Second, an unwillingness to take part in research (which is essential for NHS improvement) will be problematic unless a requirement to support NHS-funded research is made explicit. NHS organisations are themselves subject to a clear expectation that they will support research as a core activity.

It is recognised that people who volunteered to be interviewed or shadowed in the case sites may have been different from those that did not allow access. However, our study design incorporated a dissemination element that enabled our findings to be shared with a range of stakeholders across the country to test out the extent to which our analysis and interpretations resonated with staff and patients in different locations around England. The research team considered the effort and resources expended for this valuable exercise justified, in that these events validated our findings and produced valuable insights into how they could be translated into actionable good practice guidance, all of which significantly enhanced the strength of our study and its outputs. The use of all the above methods of triangulation and validation considerably strengthens the trustworthiness of the findings.

New CCGs were not quite embedded in 2014 when the national survey was undertaken, and this was the reason for a pragmatic decision to use FOI processes as a data collection method. The use of FOI ensured that the organisation found the correct source of information and advice, as telephone operators rarely knew to whom they should address our enquiry, a problem that has been reported elsewhere. 95 Furthermore, utilising the knowledge of our commissioner co-applicant, and in order to increase our response rate, rather than ask people to complete a long questionnaire, we asked them to send us their documentation relating to quality of community nursing in 2014/15. Requesting documentation designed for work purposes resulted in receipt of a variety of documents at different stages of completeness. The most consistently complete data reported the CQUIN scheme and, as our design incorporated an ensuing case study, affording the chance for in-depth analysis of other quality measures, we focused on CQUINs for the survey. This possibly narrowed our focus too closely on this aspect of service quality measurement; however, negotiating and monitoring the scheme proved to be a key area of work for commissioners and enabled insight into the competing quality priorities between commissioners and service providers. The focus on CQUINs also highlighted the difference in emphasis given to front-line concerns and the national agenda. As the scheme was incentivised it also offered the opportunity to add to the meagre literature on the unintended consequences of incentivised care in relation to community nursing. We were able to scrutinise a range of supporting documentation on other measures from commissioners and providers, including front-line staff, although this was comparatively limited owing to variation and incompleteness in the documentation obtained. Commercial sensitivities following the introduction of competitive commissioning may also have influenced what documents were made available to the team.

Owing to the prevailing wider circumstances relating to staff vacancies and a high degree of change impacting on community nursing services, we were sensitive to ensuring that front-line staff shadowing was timed for the convenience of organisations, missing the winter pressures and only commencing when the research team had built a relationship of trust with nursing team leaders. Nevertheless, one of the sites, the social enterprise, did refuse to allow shadowing of staff without giving any explanation, although front-line staff were able to participate in a focus group. As the findings of that focus group were in accord with focus groups in different sites, it is arguable whether or not there would have been different issues emerging in relation to front-line observations but, again, commercial sensitivities may have been behind the decision to deny access. In the other four sites, community nurse team leaders allocated staff for shadowing and in several cases this was a more senior member of the team. Although this might have influenced the findings of observations to an extent, it did enable a more in-depth discussion of care context and quality than might otherwise have been the case.

Lack of parity in documentation received meant that it was difficult to identify any meaningful differences in our cross-case analysis of this aspect of the data collection. The main difference identified was that one case site (a combined acute and community trust) had chosen not to participate in the 2015/16 CQUIN scheme. It was unclear whether this was due to a pragmatic decision to minimise financial risk or because it did not prioritise community services. Additional documentation, such as retrospective quality accounts, was complete and accessible online. In theory these reports enable patients and carers and other stakeholders to access information about service quality. In reality, although quality accounts do provide evidence of provider services’ achievements against quality targets, accompanied by comments from the local Healthwatch organisation, the reports have the appearance of a public-focused marketing brochure (their structure and appearance differed across the case sites, again making comparison unfeasible). Furthermore, the ‘marketing‘ of quality accounts highlights the problem of linking QIs to financial awards and penalties, as it is bound to lead to organisations seeking to market themselves and present the most positive face possible, rather than openly highlight and learn from problems in order to improve them.

Implications for practice

Commissioners and provider managers alike reported difficulties in finding appropriate health outcome QIs for community nursing and applying meaningful targets and goals. In our study, even where patients and carers agreed on the importance of health outcomes being reported nationally, they did not think that such measures could be used to assess nursing quality, having first-hand knowledge of other factors that could influence these outcomes. There was consensus that QIs would be most useful if front-line staff and patients or carers could be involved more closely in the process of selection. The practicalities of achieving this input are yet to be decided. However, clinical QIs that could be used as an aide-memoire by front-line nurses were found to be useful by staff (e.g. assessment of falls). This measure was also considered important by patients and carers and is known to be an indicator of deterioration in health and injuries that could potentially lead to hospital admission. 96 Although agreeing with the importance of a falls assessment, however, patients cautioned that the fact that an assessment had been undertaken was not, of itself, an indicator of high-quality care, placing the quality emphasis instead on provision of follow-up and ongoing monitoring.

Meaningful care quality outcomes for both patients and nurses could be achieved by adopting an approach that incorporates patient-centred care with outcomes of care measurement. Patients with long-term conditions thought that defining appropriate goals with their community nurse and setting small milestones to measure their improvement over the course of an episode of care would be motivating, both enabling them to take more control of their health themselves and also be amenable to measuring a health outcome of importance to them. A small unpublished pilot study suggested this approach fitted in with personalised care planning and was possible in a community nursing context. The use of such measures can have the effect of uncovering related, albeit previously unknown, health concerns of the patient and open a patient-led discussion (S Horrocks and L Wye, University of Bristol, 2014). Small goals or milestones that can be influenced by nursing can arguably give a better reflection of patient-centred care than a standard satisfaction measure. However, such an approach would require education for staff and perhaps entail more time in assessing and monitoring patient progress by front-line nurses. A discussion paper on outcome-based commissioning in the social care sector has emphasised that although there are some benefits to patient-determined goals, it is resource intensive and requires effective systems to accurately identify client needs and overall service objectives and then to manage the processes of monitoring and audit, a finding that may apply likewise in the health-care sector. 97

Helping staff identify how they themselves could use quality data to support their work might encourage greater interest and investment from the front line. Some staff in our study recognised that accessible quality data could potentially support business cases more efficiently than before. In addition, there was evidence of a team having achieved some success with applications for administrative support using such data. At a more senior management level, team managers found the quality dashboard useful for tracking indicators relating to staff activity, sickness and vacancies on a monthly basis. Where feedback of these figures was made available to front-line staff, they were found to be useful in helping teams compare their own outcomes month on month and with other teams in the area (and act as an incentive to improve their own outcomes). 32

Further research

Our findings suggest that further research is required into the costs of administration of QI pay-for-performance schemes in community health settings and how these schemes can be modified to be of greater benefit to patients in supporting quality improvement and innovation in the new integrated care environment.

Research is needed to identify an agreed way of assessing if and how complex interactions between different aspects of care can be weighted and measured in new integrated care organisations. A related and pressing issue is how to minimise transaction costs in relation to time and workload associated with quality monitoring, without compromising patient safety.

More research is also required to identify the most effective way to provide front-line nurses and patients with a voice when quality measures are selected to ensure validity and to ensure that patient priorities are met.

The effectiveness of new configurations of community nursing service provision and its impact on patients and other stakeholders requires assessment in relation to continuity of care for patients and ease of access to the MDT.

Retrospective analysis of the sustainability of CQUIN-funded schemes should be undertaken to identify their longevity once the CQUIN funding has ceased.

A trial of patient-directed outcome measures could be undertaken to identify their usefulness in the community nursing or integrated care context. A pilot study in this area using a slightly adapted Measure Yourself Medical Outcome Profile questionnaire (freely available), suggested that it might provide a measurable clinical outcome for patient-centred care as a QI for community nursing.

On the basis of research suggesting a link between positive nurse assessment of service quality and positive health outcomes for patients in acute settings, the feasibility of using community nurse assessments of service quality as a proxy for their patients could be trialled.

The validity and reliability of closer alignment with CQC approaches for quality measurement in community nursing, including front-line inspection, should be investigated.

Evaluation of the costs and benefits of different methods of stakeholder and public engagement should be undertaken to review the long-term benefits of these to validating research findings and implications for practice.

Conclusion

This study aimed to investigate the ways in which quality in community nursing is measured, specifically the selection, application and usefulness of QIs being used in commissioning community nursing services. The context for the overall question was an awareness that public confidence in nursing quality has been shaken by hospital and care home scandals in recent years. Yet little was known about the quality of services being provided in the home, or how their quality is measured. Our study findings suggest that the quality measures currently in use appear to fulfil the accountability requirements of commissioners and health service provider managers in relation to investment in organisational change, but do not sufficiently capture the values espoused by front-line staff or service users in relation to the experience of care.

Furthermore, the requirement for service providers to meet existing criteria for nationally mandated and local incentivised QIs appears to be risky at a time of financial constraint, have a limited positive effect and could actually impact adversely on the quality of care that nurses are able to deliver because front-line nurse time needs to be utilised in data collection and follow-up. Poorly conceived indicators may also result in unintended negative consequences for patients and other members of the health-care team.

The validity and reliability of data currently collected by front-line community nursing staff is questionable for quality assessment purposes, and the resource that appears to be required to agree, implement and monitor quality measures in practice suggests pay-for-performance approaches to quality measurement may not be cost-effective. In contrast, CQC inspections incorporate quality domains, such as the nature of caring and responsiveness, that encompass the values of patients and nurses more effectively than current health service quality measures, and front-line inspections conducted by objective observers can more fully contextualise service quality. There is an argument to suggest reducing the number of different indicators on which providers must report and adoption of the five CQC domains as a basis for quality measurement.

The need for managers to monitor service quality will, however, continue, and increasing evidence suggests that for this to be meaningful both to community nurses and patients and carers, measures should specifically reflect their concerns about appropriate staffing and resources, such as appropriate time allocation for visits. There is potential for improving quality measures by involving all stakeholders, including patients and front-line staff more effectively, as required for effective outcome-based commissioning. This can be achieved by using more innovative ways of assessing service quality, such as incorporating real-life patient case studies and by consulting staff at each stage of the commissioning cycle. Extended contract time scales currently being discussed by NHSE should facilitate improved engagement with front-line staff, patients and carers in quality assessment. Personalised care planning may present an opportunity to further develop patient-derived goals that could be used as health-care outcomes for community nursing service quality measurement. Although problems with connectivity continue to impede the use of IT for the purposes of data collection in community nursing, in time, new technologies are likely to promote ease of data collection alongside clinical record keeping and could also be used to bring about collaboration across providers in the community.

Research facilitation and participation

We are extremely grateful to all the individuals in our case study sites who facilitated our access to the participating organisations and to all our participants for the time they generously gave us and their willingness to share their views.

Stakeholder engagement events

The opportunity to include in our final report stakeholder feedback about our findings and draft good practice guidance considerably enhanced our study. We would therefore like to thank all those delegates who took the time and trouble to attend the stakeholder engagement events in support of the project.

Special thanks to Martin Howard, who brought to bear his invaluable expertise and experience in advising and assisting with the facilitation of the workshops.

Service user reference group

We also wish to thank the members of SURG – Geoff Loydon, Joan Bayliss, Jill Boreham, Christine Donald and Enid Smith – who provided us with pertinent advice throughout the project, which helped to ensure that our study processes and discussion of findings incorporated and reflected service user perspectives.

Colleagues

Our thanks also go to Katalin Bagi and Claire Chappell for their sterling support with the financial management of the project throughout its duration, and to Jo Hartland and Paul Roy from the Avon Primary Care Research Collaborative management who were always available for consultation and advice when we needed it.

Study Steering Committee

Finally, we are indebted to our Study Steering Committee members whose considered and judicious advice helped us to maintain and refine focus at key stages of the project.

Contributions of authors

Sue Horrocks (chief investigator) participated in the conception and initial design of the project, contributed to data collection and analysis of both the quantitative and qualitative data and was lead author for the final report.

Katherine Pollard (project manager) contributed to data collection and analysis of both the quantitative and qualitative data, co-led the facilitation of the stakeholder engagement events and contributed to writing the final report.

Lorna Duncan contributed to data collection and analysis of both the quantitative and qualitative data, helped with the facilitation of the stakeholder engagement events and contributed to writing the final report.

Christina Petsoulas contributed to data collection and analysis of the qualitative data, helped with the facilitation of the stakeholder engagement events and contributed to writing the final report.

Emma Gibbard contributed to the design of the project, analysis of the qualitative data, co-led the facilitation of the stakeholder engagement events and contributed to writing the final report.

Jane Cook contributed to the design of the project, analysis of both the quantitative and qualitative data, helped with the facilitation of the stakeholder engagement events and contributed to writing the final report.

Ruth McDonald participated in the conception and initial design of the project, provided material for the final report and helped to revise it for intellectual content.

Lesley Wye contributed to the design of the project and to analysis of the quantitative data, as well as helping to revise the report for intellectual content.

Pauline Allen participated in the conception and initial design of the project, contributed to higher level data analysis, provided material for the final report and helped to revise it for intellectual content.

Pete Husband (service user) participated in the conception and initial design of the project, chaired the project SURG, provided material for the final report and helped to revise it for intellectual content.

Lizanne Harland participated in the conception and initial design of the project, contributed to the analysis of the quantitative data, helped facilitate the stakeholder engagement events, provided material for the final report and helped to revise it for intellectual content.

Ailsa Cameron contributed to the design of the project, provided input to the conduct of the study and helped to revise the report for intellectual content.

Chris Salisbury contributed to the design of the project and the grant proposal, provided input to the conduct of the study and helped to revise the final report for intellectual content.

Publications

Cook J, Horrocks S, Gibbard E, Harland L, Wye L. How commissioning affects community nursing. Nurs Times 2015;111:18–21.

Cook J, Horrocks S. Community nursing quality indicators for end-of-life care in England: identification, preparation, and coordination. Br J Community Nurs 2016;21:119–23.

Petsoulas C, Allen P, Horrocks S, Pollard P, Duncan L, Gibbard E, et al. Using contractual incentives in district nursing in the English NHS: results from a qualitative study. Public Money Manage 2018;38:223–32.

Data sharing statement

All available data can be obtained from the corresponding author.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care.

Introduction

In this appendix we present the key discussion points from 10 stakeholder engagement events. Eight workshops and two facilitated conference sessions were held over the space of 4 months, from June to September 2016. The workshops were held in different cities across England: Birmingham, Bridgwater (Somerset), Bristol, Leeds, London, Nottingham, Southampton and York. The two conference sessions were in the programme for the conference ‘Community Nursing: Innovation and Transformation’ held in Leeds in July 2016 and for the QNI conference held in London in September 2016. Three or four different members of the research management group with a range of professional backgrounds, including a member of SURG, together with an independent consultant with expertise in group facilitation and commissioning, facilitated each event. After the first workshop, at least one facilitator at each event had been involved in a previous event.

The purpose of the stakeholder engagement events was to:

• test the emerging findings from the case study sites to check our analyses and interpretation

• review and improve draft good practice statements and identify further areas of good practice (see Appendices 3 and 4)

• engage key stakeholders in our research and its outputs to enhance impact.

The workshops were designed to engage a mixed audience of commissioners, service managers, front-line staff, patients and carers, and involve them in a deliberative dialogue exercise. 69 This was achieved in the majority of the events, although, owing to practical reasons, the composition varied and not all stakeholders were represented at every workshop. The eight workshops were attended by a total of 120 delegates; attendance ranged from 10 to 27 individuals at a single event. All stakeholders were represented at three events; it proved impossible to recruit any patients or carers to three events and commissioners were not present at two other events. However, all of the workshops involved at least two of the three stakeholder groups targeted.

Each workshop lasted for 3 hours and followed the same format. Each started with short presentations to set the scene and context for the research. This typically included a keynote presentation focused on quality and community nursing delivered by a local quality or service lead, followed by a short presentation by a member of the research team on the what, why and how of the research. We emphasised here that the research focus was on the ‘how’, that is ‘how is quality measured?’, rather than the ‘what’, that is, ‘what does a good QI for community nursing look like?’ The next stage involved the use of deliberative dialogue,69 where participants worked in small groups to share, discuss and capture their thoughts, ideas and experiences in relation to four findings and associated good practice statements presented (see Boxes 1–4). Key points from these discussions were then fed back to the wider group for further consideration.

Each workshop was evaluated to seek feedback on format, quality of presentations and facilitation, as well as asking participants to identify an action they could take as a result of attending the workshop. The feedback from the first workshop was used to review and refine the workshop content and format for future events.

The two conferences had a greater focus on reaching community nurses. Twenty-four nurses attended the session at the conference held in Leeds and 122 took part in the session at the QNI conference. The conference sessions used the same presentation as the workshops and lasted for approximately 45 minutes. Owing to these time constraints, the discussions focused on one finding only (see Finding 1: indicators are not always fit for purpose). Owing to the scale of these conferences and the timings of the conference programmes, we were unable to evaluate the individual sessions.

The findings presented and discussed at the stakeholder engagement events were those that emerged from data collected through interviews and focus groups with commissioners, providers, front-line staff, patients and carers, across all five case studies. Owing to time constraints it was obviously not possible to offer the opportunity to delegates to discuss more than a few of the many findings which emerged from the study. The findings presented at the stakeholder engagement events were selected because they were consistent across all five sites and suitable for discussion by all stakeholder groups:

1. Not all indicators selected are fit for purpose as the selection process does not always involve all the right people.

2. Indicators rolled out from hospital settings often do not work well in community nursing settings.

3. Staff do not always receive adequate information about indicators either before or after they collect relevant data.

4. Quality in community nursing is hard to measure and a focus on collecting numbers does not give a true reflection of the service being delivered.

It was interesting to note that these findings were all challenges inherent in current processes. The draft good practice statements developed were based on the evidence collected from the case study sites.

To facilitate discussion in their small groups, participants were asked to consider the following questions:

• Is this finding a key issue for you?

• Do the good practice statements address this issue?

• How could the good practice statements be put into practice?

• How else could this issue be addressed?

Stakeholder discussions

In the following section, key points brought up during the stakeholder discussions at both the workshops and the conference sessions are presented for each of the four findings and their associated good practice statements (see Boxes 1–4). There were common messages and themes that emerged from these discussions which highlighted a degree of interdependency between the four findings.

Cross-cutting themes

A number of cross-cutting themes emerged from discussions that reflected many of the challenges currently affecting community nursing. 75 These included issues around recruitment and retention, changing caseloads and increasing workload; the impact of indicator data collection on staff workload and the time available to care for patients; the interdependency of nursing services on other services and the move towards integration of care; the challenges with time scales, IT and documentation; and the lack of communication, engagement and training around QIs with front-line staff. All of these themes reflect our wider study data.

Interestingly, discussion also highlighted the relationship between the four findings presented, with interdependencies noted between all of them. There was a consistent message that the appropriate involvement of the right people, good communication throughout the process and clarity of purpose would address many of the challenges presented within the findings. Another common message was the need to change the focus of indicators to concentrate more closely on patients and their outcomes, together with a focus on improvement rather than judgement.

Workshop evaluation

Eighty-eight evaluation forms were completed across the eight workshops, on 84 (95.45%) of which the workshop was rated overall as either excellent or good. Sixty participants identified an action they could take forward following the workshop. These included:

Evolve CQUIN differently from now on – with those who will be collecting/involved in the indicator collection.

As a commissioner, ensure that development of outcome measures is carried out collaboratively.

Think about greater use of PROMs [patient-reported outcome measures].

Review the research when available, discuss with colleagues.

The research team felt that holding the workshops was a very valuable exercise, as it enabled testing of some of our study findings with wider stakeholder groups, as well as enhancing the quality of the good practice guidance developed following data analysis. However, it was also noted that this series of stakeholder engagement events was extremely resource intensive. In particular, administrative input into the process was substantial. It is recommended that funding for these valuable engagement activities needs to be properly estimated in grant applications submitted to funders.

Summary

Through these events we have reached and engaged with > 260 people, with 120 participants attending the eight workshops together with 146 people in attendance at the conference presentations.

The discussions at the stakeholder engagement events were very useful. First, they not only revealed that the four study findings presented reflected attendees’ views of their local experiences, but also confirmed some of the wider study findings not presented at these events. This suggests that our analysis and interpretation of our study data are accurate and that our findings transfer to other areas of the country. It was notable that none of the attendees found our findings surprising. In turn, the facilitators of the events reflected that the additional information emerging from the discussions did not reveal anything unexpected.

Second, workshop attendees’ feedback confirmed that the good practice statements were reflective of the findings and addressed the key issues. Useful suggestions for improvements were made and these have been incorporated into the final version of the good practice guidance (see Appendix 4). Focusing discussion on how the good practice statements could be implemented helped to identify some of the challenges and possible solutions, enabling delegates to share good practice within their own organisations, which further informed the good practice statements.

The workshops were very positively evaluated and undoubtedly enhanced the robustness of the study findings as well as the good practice guidance developed as a study output. However, they were extremely resource intensive, particularly in terms of administrative input; the cost of such initiatives needs to be properly estimated in grant applications to funders.

Selection

• Start the process early to enable time to identify, engage and involve stakeholders in the identification and selection of QIs. Consider starting with the co-design and co-production of service specifications as this will inform the development of person-centred QIs.

• Actively engage and involve representatives from key stakeholder groups. This may include commissioners, providers, community nurses and other clinicians across all grades, patients, formal and informal carers and their representative organisations, researchers and academics, as well as experts in measurement such as health economists and data analysts.

• Involve front-line staff in the selection and design of QIs as this will improve ownership and understanding, and ensure that it is a valid indicator of service quality and can be used to improve it.

• Use information from existing data and learning from previous QIs and link back to service specifications to help inform indicator identification and selection.

• Be explicit about the purpose and intended goal of a QI. Ensure that the focus is on improving the quality of services and consider using person-centred outcomes.

• When selecting indicators consider both the intended and possible unintended consequences.

Application

• Ensure that the total number of QIs is appropriate for the setting within which they are to be implemented and that they link back to the service specification.

• Avoid introducing too many simple processes which, when all combined, result in an overcomplicated process for staff on the ground.

• Ensure that the indicators selected are appropriate for the context within which they are to be applied. Take account of individual patients’ needs, the care environment, the current infrastructure (including IT), resources and existing data, as well as the knowledge and skills available.

• Consider using indicators that are SMART.

• Develop clear communication and engagement plans to support the identification and delivery of QIs.

• Make sure all those involved in collecting data, including front-line community nursing staff, are fully engaged in the process and informed well beforehand about what data they have to collect, why they are doing so and what will happen to the information they gather.

• Make sure that all stakeholders involved in the process, particularly front-line staff, receive timely, regular and accessible feedback about data collected and resulting findings so that these can inform further quality improvement.

• Invest in streamlining communication systems between different organisations and sectors.

Usefulness

• Ensure that the right information (data) is collected to help measure progress towards achievement of an indicator.

• Capture the true quality of the service by varying how it is measured – consider using a mixture of both numbers (quantitative) and narrative (qualitative) data, including case studies and patient stories that involve patients and carers directly.

• Ensure that the indicator is fit for purpose (i.e. that the intended benefit is being produced).

• Ensure that the right measures are selected to reflect the goal and consider use of proxy measures for those with long-term outcomes.

Ensure that the data collected reaches an agreed standard and has been validated and triangulated where possible. Consider providing education and training to staff to support data entry, data collection, the understanding of quality and QIs, quality improvement, clinical audit and evaluation.

Interview schedule for commissioners

The interviewer will typically follow each question with additional supplementary questions in order to obtain a clear, in-depth understanding of the interviewee’s answer. Each interview will start by asking the participant for the title of their current post and the length of time for which the post has been held.

Question 1: you kindly provided details of your local QI scheme for our survey. Can I just check we have the correct details please? (Show scheme.)

Question 2a: please could you walk me through the process you go through to identify and select QIs for community nursing?

Question 2b: what were the drivers influencing this process?

Question 3: how useful have QI schemes been in driving quality improvement in community nursing services?

Question 4: what factors have you identified that makes an effective quality scheme for community nursing?

Question 5: specifically looking at financial incentive and penalty schemes such as CQUIN, could you explain how these work locally for community nursing?

Question 6: is there anything I have not asked you but that you think is important for this study to be aware of?

Interview schedule for provider managers

The interviewer will typically follow each question with additional supplementary questions in order to obtain a clear, in-depth understanding of the interviewee’s answer. Each interview will start by asking the participant for the title of their current post and the length of time for which the post has been held.

Question1: what is your involvement with the selection of QIs for community nursing?

Question 2: how useful have QI schemes been in driving quality improvement in community nursing services?

Question 3: to what extent are staff in control of factors that could influence achievement of particular QIs – any examples of these?

Question 4: how could indicators be improved for the benefit of patients?

Question 5: what is the system of data recording for QI monitoring?

Question 6: what are the local barriers/challenges and facilitators for implementation of quality schemes on the ground with front-line staff?

Question 7: how confident are you that QIs accurately reflect service quality?

Question 8: in what ways (if at all) does the need for data collection impact on nurses’ relationships with patients?

Question 9: how can the idea of capturing quality of service be translated into routine practice?

Question 10: is there anything I have not asked you but that you think is important for this study to be aware of?

Interview schedule for community nursing team leaders

The interviewer will typically follow each question with additional supplementary questions in order to obtain a clear, in-depth understanding of the interviewee’s answer. Each interview will start by asking the participant for the title of their current post, the length of time for which the post has been held and the length of time since professional qualification.

Question 1: can we start by looking at the QIs in use for your services and talking about how front-line staff feed into this process in your area?

Question 2: to what extent are staff in control of factors that could influence achievement of particular QIs – any examples of these?

Question 3: what is the system of data recording for QI monitoring?

Question 4: how confident are you that QIs accurately reflect service quality?

Question 5: how useful have QI schemes been in driving quality improvement in community nursing services?

Question 6: what are the local barriers/challenges and facilitators for implementation of quality schemes on the ground with front-line staff?

Question 7: how can the process of capturing quality of service be translated into routine practice without impacting detrimentally on the quality of care?

Question 8: in what ways (if at all) does the need for data collection impact on nurses’ relationships with patients?

Question 9: how could indicators be improved for the benefit of patients?

Question 10: is there anything I have not asked you but that you think is important for this study to be aware of?

Focus group schedule for front-line nurses

The facilitator will typically follow each question with additional supplementary questions in order to obtain a clear, in-depth understanding of a participant’s answer.

Question 1: just to start, how do community nurses know if they are providing a high-quality service or not?

Question 2: can you tell me anything about the KPIs or QIs being collected for community nursing?

Question 3: to what extent have you been able to influence or participate in selection of indicators? (If ‘yes’) can you give any examples of how this has happened?

Question 4: what are your thoughts about these QIs for measuring the quality of community nursing?

Question 5: looking at these, to what extent do you feel front-line staff are in control of achieving these or not? (Facilitator shares some indicators in use locally.)

Question 6: can you explain how the data for quality monitoring purposes (such as for the NST) is collected in your area?

Question 7: what are the pros and cons of these QIs?

Question 8: what are the challenges in collecting QI data for you as front-line clinicians?

Question 9: how, if at all, does data collection for QIs in the course of your normal working day affect your patients?

Question 10: what happens to the data that you collect?

Question 11: what implications are there for the organisations if QIs are/are not met?

Question 12: what feedback, if any, do you get on community nursing service quality from patients and/or from managers?

Question 13: how could the processes for data collection for QIs be improved for front-line staff?

Question 14: are there any other comments you would like to make in relation to measuring quality in community nursing?

Observation schedule for front-line shadowing sessions

Observation schedule for organisational meetings

Interview schedule for patients and carers

The interviewer will adjust the wording appropriately, depending on whether the interviewee is a patient or a carer.

Question 1: please tell me about the care you receive from the nurse(s) who come to visit you at home.

Check that the patient/carer is talking about care they receive from the community nurse(s), rather than from another community service.

Try and find out what the patient/carer thinks are good things about the care they receive.

Try and find out what the patient/carer thinks are bad things about the care they receive.

Question 2: so you are saying that good nursing care involves . . . mention the issues they have talked about in their answer to the first question.

Reverse the bad things they have talked about and ask about them as well as the good things.

Talk to the patient/carer about the measures that apply to community nursing locally, making sure that you explain what they mean, and what issues are involved. For each measure ask:

Question 3: does this issue matter to you personally? Why (not)?

Question 4: do you think that nursing managers need to know about this issue in order to make sure that people get good care from community nurses? Why (not)?

Question 5: you said that good nursing care involves . . . (mention everything they said in answer to the first question). I just want to check if you think that there is anything else that nursing managers should know about, so that they can make sure that community nurses are delivering good care to patients and carers.

Focus group schedule for patients and carers

The interviewer will give each participant a paper with the examples of QIs below to each participant.

The wording of questions will be adjusted as necessary to include all participants.

Question 1: thinking about your experience of community nursing services, which aspects of the service you received were important to you? (Check participants are talking about care they receive from the community nurse(s), rather than from another community service.)

Question 2: how did you judge whether or not a high-quality service was provided?

Question 3: Talk to the patient/carer about the NST, FFT (and a few other indicators that apply to community nursing). Explain what they mean and why they are collected. For each measure ask:

Does this matter to you personally? Why (not)?

Do you think that nursing managers need to know about this in order to make sure that people get good-quality care from community nurses? Why (not)?

Question 4: what do you think about community nursing providers using patient satisfaction surveys?

Question 5: how best can patients and carers express their personal priorities for care? (Probe: explain patient identified goals.) What do you think would be the advantages and disadvantages of using these?

Question 6: what information would you like to know about the community nursing services you use and how would it best be given?

Question 7: I just want to check if you think that there is anything else that I have not asked about, that would be important for us to know in relation to how best to measure the quality of community nurse care?