Implementing the Creating Learning Environments for Compassionate Care (CLECC) programme in acute hospital settings: a pilot RCT and feasibility study

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 13/07/48. The contractual start date was in December 2014. The final report began editorial review in March 2017 and was accepted for publication in November 2017. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

James Raftery is a member of the National Institute for Health Research (NIHR) Journals Library Editorial Group. He was previously Director of the Wessex Institute and Head of the NIHR, Evaluation, Trials and Studies Coordinating Centre (NETSCC).

Copyright statement

© Queen’s Printer and Controller of HMSO 2018. This work was produced by Bridges et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2018 Queen’s Printer and Controller of HMSO

The NHS context

The need to strengthen the delivery of compassionate care in UK health and social care services, in particular to older patients, has been consistently identified as a high priority by policy-makers in recent years. 4 In addition to a series of investigations into high-profile failures, substantial and significant variations in the quality of hospital care for older people have been highlighted. 1,5 Variation exists between hospitals, but also between wards within hospitals and between staff within wards. Training, staffing levels, leadership, motivation and organisational culture are all implicated in failures of care. Although these issues are widely reported in the UK, there is evidence to suggest that they are relevant internationally. 6,7

Care failures at the Mid Staffordshire NHS Foundation Trust in the late 2000s, and the inquiries that followed, were a watershed moment for the NHS. Over a period of some years, patient care in many wards and departments at the trust had been of very low quality, with, for instance, patients left in soiled bed clothes for lengthy periods, assistance not provided for patients who could not eat without help, and indifferent and unkind treatment by staff of patients, often older patients, and their families. Two inquiries led by Sir Robert Francis QC examined the causes of the lack of care and high mortality rates. The first inquiry focused on patient care at the trust and offered recommendations for improving practice at the trust. 2 The second inquiry focused on the systems of governance underpinning the care failures and offered recommendations for the NHS as a whole. 1 In the recommendations from the second inquiry, Francis called for a fundamental change in culture across the NHS towards a culture that puts patients first. Several of these recommendations focused on promoting compassionate nursing care. These recommendations focused on how to identify and promote desirable attributes (knowledge, skills, attitudes) in individual nurses. Many other recommendations focus on the systems needed to promote high-quality care and the responsibilities that should be held by key groups and organisations such as trust boards, NHS regulators, professional bodies and educational institutions. Although there is little detail about desirable systems and processes at a ward-team level, recommendations from both inquiries provide an outline of such measures to counteract the potential for the care failures encountered at the Mid Staffordshire NHS Foundation Trust. These include providing mechanisms through which health-care professionals can raise concerns about patient care with colleagues and with senior managers; the ongoing provision of training, support and supervision to nursing staff; and investing in ward leader roles that work alongside team members, providing role modelling and mentorship.

There have been significant changes to UK health-care provision since the establishment of the NHS in 1948. The improvement in medical treatments during this time has contributed to people living longer with more complex health conditions. Acute hospital inpatient beds are now predominantly populated by older people with multiple health conditions. However, recent years in particular have seen the adult social care system and pressures on primary care services affecting secondary care. Acute hospitals have struggled to meet their performance and financial targets. During the time that this study was taking place, health and social care in England was in the midst of unprecedented demand and financial challenges, with NHS providers having overspent by £2.45B by the end of 2015/16. 8

It has been acknowledged that an increase in staffing numbers is required and a safe staffing guideline for nurses working on wards in acute hospitals has been published. 9 Although this was supported by the creation of 24,000 new nursing posts between 2012 and 2015, an increase of 8.1%, demand has outstripped supply with a deficit of 8.5% of the funded establishment recorded in April 2015. 10 This deficit is worse in adult acute nursing, with reported vacancies amounting to 9.7% of the establishment. It is common practice for wards to run with staff vacancies and for the staff complement to be made up of staff from nursing agencies. Recruitment drives targeted at overseas nurses have regained popularity, but this too is a temporary fix, with European Union nurses choosing to exercise their free movement rights. 11

Through the development and tightening of systems for financial control and performance management, the NHS has seen an intensification of health-care work through higher patient numbers and time-based targets. 12–14 Use of staff without professional qualifications is increasing and nursing staff job satisfaction is low. As we developed this study, anecdotal evidence from a number of NHS acute hospitals indicated that the leadership and team practices, such as role modelling, mutual support, reflective learning and dialogue, required to support nursing staff in their caring role15 were unlikely to be in place in most care settings.

Approach and definition of key terms

The literature is both confused and confusing in the way that compassion is used as a term. There are four key components of the narrative of compassion in nursing, and we have found these helpful to guide our thinking in this study about what compassion is. 16 The first is a set of ideas about the moral attributes of a ‘compassionate’ nurse, including wisdom, humanity, love and empathy. 17–19 These moral attributes are expressed through a kind of situational awareness in which vulnerability and suffering are perceived and acknowledged. 19,20 These perceptions underpin participation of the nurse in responsive action that is aimed at relieving suffering and ensuring dignity, and which involves the nurse in a participatory relationship in which the nurse exercises relational capacity 19,21–23 through which empathy is experienced and a caring pastoral relationship is constructed. 15,24,25

Our systematic review of research reporting older patients’ experiences of hospital care highlights the importance of this caring relationship to shaping experiences. 6 Older people want nurses, and others, to use social interactions to see the person behind the patient (‘see who I am’), to establish a warm and human connection (‘connect with me’) and to establish understanding and involvement (‘involve me’). 6 A later review focusing on nurses’ experiences indicates that registered nurses strive to achieve the caring relationship that is valued by patients, indicating that a perceived lack of compassion in nursing may not be attributable to a lack of the necessary moral attributes or situational awareness on the part of individual nurses. The findings reflected that nurses’ relational capacity and capacity for responsive action can depend on ward-level conditions, and that there is a greater tendency for nurses with low relational capacity to avoid relationships with patients and to burnout, in spite of aspirations to a higher standard of care. 15 This study builds on these findings through the development and evaluation of an intervention targeted at improving the capacity of nurses to respond to patient vulnerability and suffering, specifically their relational capacity and capacity for responsive action.

The links between positive patient experiences, leadership, work team climate and the well-being of individual staff are becoming evident through research and so interventions that focus on developing these elements (leadership, work team climate, staff well-being) would appear to be worthwhile to support the development and exercise of relational capacity. A NIHR study on culture change and quality of acute hospital care for older people found that more positive patient and carer assessments of care were correlated with higher staff ratings of a supportive team climate and a shared philosophy of care. 26 In addition, ward leadership was a strong indicator of team members sharing a philosophy of care and feeling highly supported, a finding that, together with the qualitative data, highlighted the vital role of the ward manager in shaping a positive team climate for care. 26 These findings were mirrored in a second NIHR study that highlighted the key role of the ward leader in shaping the local ward climate of care, the importance of staff well-being, and, in particular, staff experiences of good local work-group climate, co-worker support, job satisfaction, positive organisational climate and support, and supervisor support as antecedents of positive patient experiences. 27

Creating Learning Environments for Compassionate Care

Parts of this section are reproduced from Bridges J, Fuller A. Creating learning environments for compassionate care: a programme to promote compassionate care by health and social care teams. Int J Older People Nurs 2015;10:48–58,28 with permission from John Wiley & Sons. © 2014 John Wiley & Sons Ltd. CLECC is a team-based implementation programme focused on developing leadership and team practices that enhance team capacity to provide compassionate care. Its objectives are to:

1. create an expansive workplace learning environment that supports work-based opportunities for the development of relational practices across the work team

2. develop and embed sustainable manager and team relational practices, such as dialogue, reflective learning and mutual support

3. optimise and sustain leader and team capacity to develop and support the relational capacity of individual team members

4. embed compassionate approaches in staff–service user interaction and practice, and continue to improve compassionate care following the end of programmed activities.

The CLECC intervention was designed for use by ward nursing teams in inpatient settings for older people, but is potentially transferable for use in other settings. The implementation programme takes place over a 4-month period but it is designed to lead to a longer-term period of service improvement. By envisaging the workplace as a learning environment and the work team as a community of practice, CLECC brings a distinctive approach to promoting compassionate care. It uses insights from workplace learning research to develop practices that enhance the capacity of the manager and work team to provide compassionate care within a complex and dynamic organisational context.

Fuller and Unwin’s research on workplace learning and workforce development in a range of public and private sector industries demonstrates the importance of identifying and analysing both the organisational and pedagogical features that characterise diverse workplaces as learning environments. 29 They argue that this approach allows workplaces (e.g. hospital wards) to be located on the ‘expansive–restrictive’ continuum. Those at the expansive end are characterised by a range of features including the following: the knowledge and skills of the whole workforce (not just the most highly qualified or senior staff) are valued; managers facilitate workforce and individual development; teamwork is valued; innovation is important; the team has shared goals focusing on the continual improvement of services (or products); there is recognition of and support for learning from ‘each other’; learning new knowledge and skills is highly valued; and the importance of planned time for off-the-job reflective learning is recognised. It follows that an expansive approach to workforce development is more likely to facilitate the integration of personal and organisational development. This has important implications for the design of learning interventions as it requires workplace learning to be perceived as something that both shapes and is shaped by the work organisation itself rather than as a separately existing activity. Such an understanding highlights the importance of interventions that situate and integrate individual and team learning in the everyday life of the workplace (in this case the clinical unit/ward/team setting) as well as providing opportunities for off-the-job provision to foster reflection, consolidate learning and deepen understanding, thereby enhancing ownership and sustainability of new practices.

Our synthesis of qualitative research highlights the importance of the relational aspects of care to shaping older people’s hospital experiences. 6 Being compassionate requires ‘relational capacity’ in practitioners, that is, capacity to experience empathy and to engage in a caring relationship. 24 Our research also shows that nurses’ relational capacity can depend on ward-level conditions, and that there is a greater tendency for nurses with low relational capacity to avoid relationships with patients and to burnout, in spite of aspirations to a higher standard of care. 15 CLECC uses workplace learning principles to develop practices that enhance the capacity of the manager and work team to support the ongoing relational capacity of its individual members. This leadership and team capacity are key characteristics of the ward-level conditions needed to support nurses’ relational work15 and thus improve patient experiences, and are an important foundation for team activities, such as using service user feedback constructively. 30 By envisaging the workplace as a learning environment and the work team as a community of practice, CLECC brings a distinctive approach to promoting compassionate care, which enhances the capacity of the manager and work team to provide compassionate care within a complex and dynamic organisational context. 29,31 An overview of this programme theory for CLECC is shown in Figure 1 .

FIGURE 1.

Overview of CLECC programme theory. Reproduced from Bridges J, Fuller A. Creating learning environments for compassionate care: a programme to promote compassionate care by health and social care teams. Int J Older People Nurs 2015;10:48–58,28 with permission from John Wiley & Sons. © 2014 John Wiley & Sons Ltd.

Reproduced from Bridges J, Fuller A. Creating learning environments for compassionate care: a programme to promote compassionate care by health and social care teams. Int J Older People Nurs 2015;10:48–58, 28 with permission from John Wiley & Sons. © 2014 John Wiley & Sons Ltd.

During the 4-month implementation programme, CLECC learning activities are led by a senior (UK band 7) practice development nurse (PDN) or practitioner with strong influencing and interpersonal skills. 28 The PDN delivers the study days, facilitates cluster and reflective discussions, facilitates the action-learning sets and co-ordinates the peer observations of practice (see the following subsections for more detail on each of these activities). This individual is not part of the hierarchy of the ward team and this enables a distinction between CLECC activities and performance management. The activities themselves are characteristic of a practice development approach. 32 CLECC operates at two key levels: team and team manager. A focus on the team aims to develop team capacity to support team members to provide compassionate care. An equivalent focus on the leadership capacity of the team manager (in ward settings this is the ward manager) aims to develop their role in leading the team, role modelling good practice and enhancing and embedding the desired team practices.

Although the programme draws on elements that have been piloted in other programmes, it is novel in combining these elements with an explicit focus on establishing reforms to routine practice and organisational resources that establish the basis for sustained changes in compassionate care. Although the implementation process is a key element, the essence of the programme is the ongoing processes of peer observation, daily cluster discussions, weekly reflective discussion and the use of evidence-based guidelines. Table 1 sets out a typical schedule for the implementation programme.

Introduction to the study

Findings from our systematic review, reported in Chapter 2 , highlight the lack of definitive evaluation research on compassionate nursing care. Responding to a general absence of strong evidence for the effectiveness of service improvements related to compassion, and building on compelling evidence indicating that a strategy targeted at improving leadership and local ward team climate could improve patient experiences, the study reported here is a foundational step in addressing the need for well-designed and rigorous evaluation to understand what works best in improving care and patient experiences.

This study, conducted in two English hospitals during 2015–16 and reported in the chapters that follow, aimed to assess the feasibility of implementing CLECC and of conducting a CRT with associated process and economic evaluations to measure and explain its effectiveness. Conducting the study provided an important opportunity to assess the feasibility of a programme with unique characteristics designed to address the issues identified in other studies, and to design an evaluation that includes an assessment of its effectiveness. The process and economic evaluations aimed to provide important information about CLECC’s workability, its integration into practice and to lay the foundations for establishing its value for money. The findings reported below (see Chapters 7–10 ) provide the basis for planning a larger, multicentre evaluation aimed at producing evidence that can be generalised more widely to other NHS acute care providers and, together with a refined intervention package, will be a valuable resource for change and improvement for NHS managers, practitioners and educators.

Qualitative research

Recent years have seen the use of qualitative research methods to underpin the development and evaluation of a number of interventions focused on improving compassionate care, or dignity in care, at the hospital ward level. 30,34,36,47–49 Interventions developed and evaluated in this way have typically been faciliated by a senior nurse, using reflective learning, action research and/or appreciative inquiry to work with ward-based nursing staff (often using patient stories and/or observations of practice) to strengthen support for existing good practice and to make changes when needed. These interventions are often shaped by a ‘relationship-centred’ philosophy in which achieving the well-being of all groups (patients, staff, family carers) is seen as fundamental to high-quality care. 41 They have used democratic and participatory processes involving patients, staff and sometimes family carers to articulate the patient’s needs and to shape the practice changes made.

The accompanying qualitative evaluations have provided important information about the processes of change and the factors enabling and inhibiting sustainable change. Some of these evaluations have reported concrete practice changes resulting from the intervention,36,47–50 while others report more variable success. 30,34 For instance, Dewar and Nolan49 and Dewar50 used appreciative inquiry and action research to involve older people, staff and relatives in developing compassionate relationship-centred care on an acute hospital ward. Methods used included participant observation, interviews, storytelling and group discussions. The findings indicated the value of appreciative caring conversations between staff, patients and relatives enabling all parties to discover ‘who people are and what matters to them’49,50 and ‘how people feel about their experience’49,50, with this knowledge enabling them to ‘work together to shape the way things are done’. 49 In the resulting model, Dewar and Nolan49 detail how older people, staff and relatives can work together to implement compassionate relationship-centred care. 49 In specifying ‘how people can work together to shape the way that things are done here’, Dewar50 identified a number of important conditions for staff to feel able to express emotions, share experiences and ideas with each other, consider others’ perspectives, take risks, use ‘curious questioning’ to examine situations and challenge existing practice, all identified as important actions to support the delivery of compassionate care. These conditions included transformational leadership, the level of support received from colleagues and senior staff, a shared set of principles for caring, open dialogue within the team and opportunities in which people had permission and space to reflect. These conditions echo the findings from other research as conditions at the team level that can support high-quality care. 26,27,51 Dewar50 reports how these conditions developed and how compassionate caring practices became embedded in the work of the team over the course of the year-long project, providing valuable evidence that change of this kind is possible.

However, Dewar’s project took place over the course of 1 year on an already high-performing ward with a strong leader. 50 The findings informed development work across a wider Leadership in Compassionate Care project implemented in a number of settings, but evaluation of the impact of these strategies elsewhere does not report the influence of the ward climate or programme length on outcomes, so evidence is lacking that such strategies can be universally effective regardless of work team context. 52 In a contrasting study to Dewar’s that explored the use of discovery interviews with older hospital patients as a way of improving dignity in care, Bridges and Tziggili30 found that ward teams required strong and consistent leadership and intense preparation before they were able to hear and respond to patient stories about care. Both organisations involved in this dignity project experienced significant delays in the progress of the project and limitations in its impact because of a lack of leadership at the ward level and a lack of preparedness of the ward teams to engage in responding positively to patient feedback. One ward team with a strong leader was able to successfully engage with the patient stories, but only after some months of team preparation. These findings indicate that, while some wards may be ready to engage in programmes, such as Dewar’s, others could benefit from a period of groundwork in which leadership and mutually supportive team practices are established.

The evaluative focus of these studies is the mechanisms for change used, particularly the democratic and participatory processes that involve patients, staff and sometimes family carers in articulating the patients’ needs and shaping the practice changes made. These qualitative accounts often provide a fuller picture of the interventions deployed than the studies reviewed below, and often include an analysis of the enablers and barriers to change. However, they do not examine in depth the process of implementation itself and so fail to systematically identify the contexts in which successful implementation is more likely or, where contexts are not receptive, how resources, relationships and norms in the wider system may need purposeful restructuring to support implementation and sustain longer-term change. 53,54 In addition, as would be expected with a qualitative approach, there is only weak objective evidence of effectiveness of the interventions deployed in these studies in relation to impact on patient outcomes.

Review methods

The remainder of this chapter reports a systematic review of studies that evaluate the effectiveness of interventions for compassionate nursing care, using the four key components of the compassion narrative identified in Chapter 1 to provide an operational definition. The objectives of the review were:

1. to systematically identify, analyse and describe studies that evaluate interventions for compassionate nursing care

2. to assess the descriptions of the interventions for compassionate care used, including design and delivery of the intervention and theoretical framework

3. to evaluate the nature and strength of evidence for the impact of interventions.

The review was conducted, guided by the Cochrane Collaboration methods55 to assure comprehensive search methods and systematic approaches to analysis of the review materials. Sections of this review report are reproduced or adapted with permission from Blomberg et al. 16 and with permission from Elsevier. © 2016 The Authors. This is an open access article under the CC BY-NC-ND 4.0 license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Selection

An adapted PICO (population, intervention, comparison and outcome) framework was used to guide study selection. 56 We included primary research studies comparing the outcomes of an intervention designed to enhance compassionate nursing care (in any setting to any client group) with those of a control condition. Eligible designs were randomised controlled trials (including CRTs) or other quasi-random studies, interrupted time series and before-and-after studies (controlled or uncontrolled). Studies were excluded if they were focused exclusively on students, or if interventions were not directed at changing nursing staff behaviour.

The lack of conceptual clarity about compassion in the literature necessitated an inclusive approach to studies that were not necessarily labelled as addressing ‘compassion’. We developed selection criteria based on the four elements of the compassion narrative described above (moral attributes of a ‘compassionate’ nurse including empathy, nurses’ situational awareness of vulnerability and suffering, nurses’ responsive action aimed at relieving suffering and ensuring dignity, and nurses’ relational capacity) so that studies were included if they met one or both of the following criteria:

• explicit goal of the intervention was stated as improving compassionate nursing care (or a closely related construct, i.e. dignity, relational care, emotional care) (through addressing nurses’ moral attributes, situational awareness, responsive action and/or relational capacity) and/or

• primary outcomes that assessed or evaluated either nurses’ self-reports of compassion and/or ability to deliver compassionate care (moral attributes, relational capacity), and/or observed quality of interactions or other measure of compassion (situational awareness, responsive action), including patient reports of experienced compassion or a closely related construct.

The titles and abstracts from the search were screened against the inclusion criteria independently by individual review team members (JB, KB, PG and YW; see systematic review team members listed in Acknowledgements ). During the screening process, frequent meetings were held by the research team in order to compare independent selections, resolve disagreements and make decisions. On independent rating (i.e. before discussion) reviewer pairs achieved between 80% and 90% agreement. In most cases of disagreement papers were excluded after discussion. Full-text papers were retrieved for all papers that screened positively in the first stage or about which a clear decision could not be taken (due to lack of information). Each full-text paper was reviewed independently by two team members followed by a decision to include or exclude in the final review. These reviews were followed by further team discussion to finalise inclusion into the data set. The reference lists of full-text papers included were scanned for further items. The search and selection process is summarised in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow chart ( Figure 2 ). The number of duplicates removed was not recorded.

FIGURE 2.

Flow diagram for systematic review searches. Adapted from Blomberg et al. 16 with permission from Elsevier. © 2016 The Authors. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Adapted from Blomberg et al. 16 with permission from Elsevier. © 2016 The Authors. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

Review findings

The review findings are presented here to address each of the review objectives in turn. First, we describe study characteristics to give an overview of studies that evaluate interventions for compassionate care. Second, we present an assessment of the quality of reporting of the interventions in the included studies, including their theoretical foundations. Third, we present evidence of effectiveness of the interventions in the included studies and analysis of the quality of that evidence.

Discussion

This chapter aimed to provide an overview of the evidence base on the evaluation of interventions for compassionate nursing care. It began by reporting on the qualitative work in this field, work that has focused on relationship-centred approaches to improving care, interventions that we would classify as a combination of care model and staff support intervention types. These studies often provide detailed information about the intervention and its inherent mechanisms for change, and often include analyses of the factors enabling and inhibiting sustainable change. However, they do not examine in depth the process of implementation itself and commonly pay only superficial attention to the influence of actors, networks and resources on the impact and sustainability of these interventions. In addition, as would be expected with a qualitative approach, there is only weak objective evidence of effectiveness of the interventions deployed in these studies in relation to impact on patient outcomes.

As identified in our systematic review, there is a wide range of intervention studies in which compassion has been addressed in a variety of ways, including through staff training, staff support or introducing a new care model. However, the overall strength of work in the field limits the conclusions that can be drawn to inform health-care policy and practice. No study reported in the systematic review included sufficient detail of its intervention to enable replication and further evaluation. This state of play limits the capacity of nurses and others to include effective strategies in their own practice, but also limits the construction of a coherent evidence base to guide managers and practitioners in improving services. 86–88

In relation to the nature and strength of the existing evidence base, overall quality of the evidence was low and it appears that the few higher-quality studies are less likely to report positive results. No intervention was tested more than once and the majority of studies used before-and-after designs that are intrinsically weak. Patient-based outcomes were not routinely included, especially in relation to the evaluation of training interventions and nurse support interventions.

Our analysis did, however, highlight the feasibility issues associated with this field of research that have formed the backdrop to the design and implementation of the study reported in subsequent chapters.

Conclusions

Although there have been many published studies that appear to offer potential solutions to deficits in compassionate care, this is a body of literature that does not offer a definitive way forward for policy and practice in this important area of health care. This is especially challenging in a context in which the need for more compassion in health care is professed from national government to frontline practitioners. Greater conceptual clarity, better designed and reported interventions, including a focus on implementation processes, and evaluations using stronger research designs are urgently required.

To date, no evaluations of initiatives of this kind have enabled a robust assessment of the effectiveness of interventions on patient care, linked with the use of theory-based interventions reported with sufficient clarity to support optimal implementation, impact and sustainability. The study reported in the following chapters is the first stage in designing a rigorous mixed-methods evaluation incorporating experimental design to understand what works best in improving care.

Study design

This mixed-methods study used two main approaches to assess feasibility: (1) a process evaluation to enable evaluation of the feasibility of implementing the CLECC intervention and (2) a pilot pragmatic CRT to lay the foundation for a future evaluation of CLECC’s effectiveness. The design draws on the Medical Research Council (MRC) guidelines for evaluating complex interventions. 89 The MRC guidelines highlight the importance of a robust theory-based intervention design coupled with a thorough understanding of the mechanisms of change and the contexts in which implementation is possible. 89 It is essential to undertake this design and evaluation prior to and during evaluation of effectiveness, so that the impact of the intervention can be optimised, and so that eventual findings on outcomes can be explained by how change happened (or not). In addition, the guidelines also highlight the importance of detailed groundwork on the implementation of experimental design and methods before a definitive evaluation of effectiveness is undertaken. This includes selection and testing of outcome measures; testing the feasibility of proposed methods for recruitment, data collection and analysis; and calculations of effect size to inform future sample size calculations.

Areas of feasibility that merited study here included implementation of the CLECC intervention into practice, contamination of practices from intervention to control wards, extent of intraward clustering of outcomes, rates of participation and attrition, and the performance of the selected outcome measures. Through piloting these procedures on six wards in two English hospitals, this study aimed to reduce uncertainty in designing and executing a definitive evaluation. We aimed to test a range of parameters to provide an evidence base from which to make decisions about evaluation design to be implemented in other centres in a future trial, including CLECC implementation, feasibility of ward-level randomisation, contributing to sample size calculation and selection of outcome measures.

The feasibility of implementing the CLECC intervention into practice was assessed through a process evaluation, using normalisation process theory (NPT) as a framework. 90 Qualitative interviews with nursing staff and managers during implementation and follow-up phases, observations of learning activities, and ward leader questionnaires aimed to identify and explain the extent to which the CLECC intervention was implemented into practice, enabling an assessment of its workability and integration into existing work practices. In addition, data were also gathered on the feasibility of conducting qualitative interviews with staff, patients and visitors to inform qualitative evaluation accompanying a future definitive trial, with the purpose of explaining trial outcomes.

An experimental design in the definitive evaluation is the most appropriate design to establish effectiveness. 91 In order to prepare for a definitive multicentre evaluation, we undertook a randomised pilot study to assess the feasibility and pilot procedures for a pragmatic CRT of effectiveness. In randomised pilot studies, all or parts of the future trial are conducted on a smaller scale to see if it can be done. 92 Given that the intervention was targeted at a group of staff rather than at the care of individual patients, cluster randomisation of staff and patients at nursing ward level was undertaken. 91

Ward teams (i.e. clusters) were randomly allocated following baseline data collection to participate in the intervention or act as control. We theorised that implementing CLECC on four wards across two hospital sites was likely to provide sufficient diverse contexts within which its feasibility could be assessed, leading to further refinement when needed. The inclusion of a small number of control wards aimed to generate insight into the likely acceptability of randomisation in the main trial and differential compliance with study procedures between intervention and control.

The CLECC intervention was implemented over a 4-month implementation period on four wards from June 2015, with two wards acting as control. Outcomes were assessed at baseline (2 months before intervention and before randomisation to groups) and 4 months after completion of the 4-month CLECC implementation period (i.e. follow-up was 8 months after randomisation). There is no single validated measure for compassionate care and we assessed its measurement across three complementary core outcomes: (1) researcher-rated observations of the quality of staff–patient interactions, (2) patient-reported evaluations of emotional care and (3) nurse-reported measures of empathy. Baseline and follow-up data were also gathered on individual and ward team characteristics.

The economic component of the study aimed to explore how costs and benefits might best be measured in a definitive evaluation. Standard health technology assessment economic evaluations rely on patient-based outcome measures, the feasibility of which was explored in this study by use of EuroQol-5 Dimensions, five-level version93 (EQ-5D-5L), (a health status measure) at the ward level. In addition, the study explored the likely costs of both the CLECC intervention (training) and its implementation (qualitative interviews with nursing staff on intervention wards).

Data collection took place between March 2015 and May 2016.

Process evaluation

The process evaluation aimed to identify and explain the extent to which the planned intervention was implemented into existing nursing practices, to enable conclusions to be drawn about how CLECC can be optimised to support sustained compassionate care delivery in acute care settings. NPT was used to guide the process evaluation, shaping the interview topic guides and informing the framework for analysis. 90 NPT focuses on the dynamic processes that support the integration of innovative practices into everyday work, and so is a helpful way to evaluate what actually happens when complex interventions are introduced into practice, and how and why the desired outcomes are achieved (or not). 94 It explores social processes and the work that people do individually and collectively, in terms of cognitive and behavioural work. NPT has four core constructs: (1) coherence – making sense of the intervention, (2) cognitive participation – investing in the intervention, (3) collective action – the practical work of implementation and (4) reflexive monitoring – modifying and embedding the intervention. 95 These constructs were used to define the areas that formed the focus of the evaluation.

The process evaluation focused on:

1. exploring how and in what ways the new practice was initially received, how individually and collectively people practically conceptualise and make sense of it (coherence)

2. assessing the degree of ownership of and participation in the new practice by key individuals and teams (cognitive participation)

3. identifying the work that individuals and teams do to enact the new practice (collective action)

4. exploring the perceived impact of the new practice on staff work and on patient outcomes (reflexive monitoring).

Exploration of these areas was aimed at informing conclusions about how CLECC could be optimised for impact and sustainability beyond the implementation period.

Pilot cluster randomised trial outcome measures

There is no single validated measure for compassionate care and we assessed its measurement across three complementary core outcomes at cluster level and at individual participant level: researcher-rated observations of the quality of staff–patient interactions, patient-reported evaluations of emotional care and nurse-reported measures of empathy.

Ward team characteristics

Baseline and follow-up data were also gathered to enable a description of the characteristics of the ward nursing teams involved in the study. These included qualitative interviews with nursing staff and the administration of a number of instruments through written survey. We assessed staff local working climate using Climate for Care (CC) and Factors that Enable Climate for Care (FECC) questionnaires,26 39-item and 19-item questionnaires, respectively, with answers on a 5-point Likert scale developed as part of a toolkit from a NIHR-funded project measuring culture change and quality of NHS acute hospital care for older people, with the ability to identify distinct nursing team climates. 26 We also administered the Matron’s Assessment of Quality of Care (MAQC) and the Carer Experiences of Care (CEC) questionnaires from the same toolkit. 26 We assessed nursing staff perceptions of workload using items from the International Hospital Outcomes Study (IHOS) battery,107,108 including (1) enough nurses on staff to provide quality patient care and (2) ratings of core care activities that were deemed necessary but left undone. This survey has been widely validated internationally and subjective ratings from it correlate with objective measures of both staffing and quality. We also measured levels of nursing staff burnout using the 22-item Maslach Burnout Inventory (MBI). 109

Pilot cluster randomised trial randomisation

Ward teams (i.e. clusters) were randomly allocated following baseline data collection to participate in the intervention or act as a control. Randomisation was stratified by hospital and by ward type [medicine for older people (MOP) or surgical]. Randomisation was accomplished using the ralloc command in Stata^® 12 (StataCorp LP, College Station, TX, USA) and conducted by the study statistician. We planned for three wards (one surgical and two MOP) in each hospital. Wards were allocated to achieve two intervention and one control in each hospital, with one of the surgical wards allocated to intervention and the other to control. This strategy was to ensure that intervention implementation could be tested in two ward contexts in each hospital, and also to ensure that we gained experience of intervention and control conditions in both MOP and non-MOP ward specialties. We performed simple randomisation. First, we allocated the surgical wards to control or intervention. The MOP wards of the hospital with the surgical ward allocated to control were both intervention wards and, therefore, we did a second randomisation to allocate the MOP wards in the second hospital to intervention or control.

Pilot cluster randomised trial allocation concealment

Ward teams (i.e. clusters) were identified and recruited before randomisation. Clusters were randomly allocated to a group following baseline data collection by team members not involved in data collection. At follow-up, researchers conducting observations of the quality of staff–patient interactions were recruited from outside the core research team and not informed of allocation. Researchers gathering questionnaire data at follow-up and involved in qualitative interviewing were aware of ward allocation. It was not possible to conceal allocation from ward team nursing staff. Patients and visitors were not informed of allocation. The success of allocation concealment was tested as part of the feasibility work.

Progression to a definitive evaluation

As outlined above, the study was designed to lay the groundwork for a future definitive evaluation of the CLECC intervention. In the first instance, we planned that the process evaluation would enable an assessment of the CLECC intervention’s workability and integration into existing work practices, providing important information to guide its further refinement and implementation. Second, findings from the study would enable assessment of the feasibility of a future definitive evaluation against a number of important parameters. Results from the assessment of these parameters would then inform the design and implementation of a definitive evaluation, including sample size, level of clustering and selection of outcome measures.

The study took place on busy acute care wards and our target samples included frontline nursing staff and patients and visitors during the period of their hospitalisation. We set an explicit goal of maximising the participation of patients often excluded from research, that is, older people with complex needs including cognitive impairment and communication difficulties, as it is often people with these characteristics who are in greatest need of compassionate care. The study enabled us to develop and test a number of approaches to successful recruitment and participation. However, there is little in the literature to guide the recruitment of these groups in acute care settings, especially in relation to experimental studies. It was therefore not possible at the outset to quantify fully the study’s success criteria in relation to the number of patients or staff to be recruited. We instead set target recruitment rates that were reviewed and refined as data were collected and analysed.

The success criteria set for the study were:

1. completion of a process evaluation into CLECC’s workability and integration into existing work practices, sufficient to inform refinement and future implementation, and to inform future process evaluations

2. recruitment of sufficient wards (n = 6) to assess the feasibility of a CRT design to inform the design of a definitive evaluation, including information on participation and attrition rates, blinding strategies, mitigation of contamination, baseline rates and intracluster correlations for core outcomes, data collection and analysis procedures

3. recruitment or refinement of target numbers of staff, patients and carers to enable collection of data estimated to be sufficient to inform the selection and use of primary and other outcome measures in definitive evaluation.

Study progress against success criteria was externally monitored by the Study Steering Committee. A Study Advisory Group advised on the CLECC intervention implementation and a PPI group also advised the study team.

Patient and public involvement

Our consultative PPI work over 3 years on this topic confirms public recognition of the need for research of this kind and that the research addresses a topic of primary concern to the general public.

We have consulted with service users about this research since 2013, while we developed the CLECC intervention, and throughout this feasibility study. We recruited older members of the public who had experienced a hospital admission, or had been a carer of someone who had experienced a hospital admission. A PPI group (n = 5) started meeting in February 2013 to help develop the application and plan staff training for the feasibility study, and two members sat on the Study Steering Committee. A leading figure (Lesley Carter) from national charity Age UK (London, UK) also sat on our Study Advisory Committee. The initial PPI group input helped to identify priorities for the research outcomes, clearly showing that improvements to compassion in hospitals, and the involvement of patients and family in care were important outcomes for the PPI group. They also advised on the development of the CLECC intervention and its evaluation. Changes made as a result of PPI input include extending classroom sessions (part of CLECC intervention) to include registered nurses as well as care assistants, developing researcher guidance on approaching family carers to participate in research, and developing ways to include people with communication difficulties in the research. We also included PPI input in training our core research team. One of our PPI representatives (Jan Gollop) provided a half-day of training to research team members on conducting research on people with dementia. Her expertise from caring for her husband with severe dementia, and from her network of other carers, and experiences visiting hospitals gave her a unique insight into what researchers needed to know.

We also held a public consultation event in November 2015 to get feedback on the work to date, test our plans for definitive evaluation and recruit more PPI input. People who attended this event (n = 6) provided advice regarding the dissemination of feasibility findings, which informed the dissemination plan for both the feasibility study and the definitive evaluation proposed as the next step. People who came discussed the ‘value’ of a definitive study of CLECC, in particular its value for money in times of austerity, and concluded that compassion in hospitals was of such personal significance that a definitive evaluation did merit funding. These consultations have confirmed strong public support for the CLECC programme of work. The study addresses a topic of primary concern to the general public and our consultees have demonstrated very strong support for this work. Involving PPI representatives from both hospital sites has been particularly helpful in involving members of the public whose locality and experiences of the hospital in question help ensure not only patient involvement but also local involvement.

Ethics considerations

Our concern in this study was to keep the best interests of participating patients, visitors and NHS staff at the centre of what we did. We included a number of measures to help ensure this and carefully consulted with our PPI group and nursing representatives about our proposals over a 2-year period. We aimed for a proportionate approach that did not place undue burden on participants at any part of the process and also represented what we judged to be achievable as a research team working within limited resources. There was also the opportunity to pilot and adapt procedures within the feasibility framework.

One key ethics issue was the recruitment of and proposed data collection from patients, staff and visitors at what can be an already stressful time. We addressed this by developing concisely written information accompanied by verbal explanation and the chance to ask questions. We ensured that we included in written and verbal information the clear statement that people were not obliged to take part and that their care or treatment would not be affected in any way if they declined to take part or withdrew. We allowed people as much time as they needed to make up their minds about taking part. We ensured that research team members had the skills to identify distress caused by recruitment and/or data collection processes and had clear plans of action to follow if this happened.

Another ethics issue was the desire to include people who lack the capacity to make the decision to take part in the research. This is important because this group is often excluded from research and yet evidence suggests that they are most vulnerable to not experiencing compassionate care. We developed and implemented clear procedures to ensure that the principles of the Mental Capacity Act 2005 Code of Practice 110 and process consent were adhered to. 111,112

A further potential issue was the participation of ward staff as research subjects and the concern that may be raised about their rights to refuse to take part or to withdraw from the study. Our communication strategy aimed to ensure that everyone who should know about the study was informed about it and their right to not take part. Researcher training emphasised issues relating to the anonymity of staff, visitors and patients. Clear procedures were developed to guide the reporting of unsafe practice.

Ethics approval for the study was granted by the national Social Care Research Ethics Committee (REC) (reference number 14/IEC08/1018) in December 2014. We originally applied for NHS REC approval, but were advised by the manager for the local REC that the study did not merit NHS REC scrutiny because the proposed intervention was not clinical. We appealed this decision, but it was upheld by the Health Research Authority regional manager. We requested an alternative, but equivalent form of review and were referred to the Social Care REC.

Research team and training for data collection

The core research team involved in data collection were the chief investigator (JB), two research fellows (LG and WW), the senior research assistant (HB) and the research assistant (PL). They conducted the qualitative interviews and were supported in screening, recruitment and data collection by others. Research nurses and clinical trials assistants (n = 18 over the course of the whole study) at the participating NHS trusts screened and recruited patients for observations. Research nurses, clinical trials assistants and eight other research assistants screened and recruited patients and visitors for the questionnaire survey and helped to complete questionnaires when this was needed. Core team members and research assistants undertook the patient observations. All staff involved in the research received classroom and field training, as set out below:

• recruiting for patient observations – 4 hours in the classroom and 4 hours in the field

• conducting patient observations – 4 hours in the classroom and 6 hours in the field

• recruiting for and conducting patient and visitor questionnaires – 4 hours in the classroom and 2 hours in the field.

Changes from original protocol

Two key changes were agreed with the Study Steering Committee and implemented. The first change was to the target sample sizes specified for questionnaire surveys with nursing staff, patients and visitors. This was made after baseline data collection when the feasible recruitment rates became clear. Tables 6 and 7 illustrate original and revised target recruitment rates.

The second change was to the economic evaluation. As the study progressed and the membership of the health economics team developed, it became clear that the health economic dimension to the evaluation would be a more helpful foundation to the definitive evaluation if it focused on how the costs and benefits of CLECC could be measured in a future definitive evaluation.

The original study objective related to the economic evaluation was:

To estimate the costs of the intervention and quality of life.

This was changed to:

To inform the measurement of costs and benefits of CLECC in a definitive evaluation.

Chapter summary

This chapter has included information on the study design, the feasibility parameters being tested and the outcome measures being assessed. PPI, ethics considerations and changes from the original protocol have also been explained.

Ward sampling and recruitment

Six ward teams were included in the study. The sample size was determined by funding availability and the plan to run the study in at least two hospital organisations, and at least two ward specialties. Our previous research had highlighted the importance of organisational and ward context in determining nurses’ relational capacity, and this informed a sampling strategy aimed at diversity. 15 Senior nurses at two English NHS trusts were invited, on behalf of their organisations, to take part in the study and agreed. They were invited to nominate three acute inpatient units each to take part in the study. They were guided to select three adult inpatient wards in each trust with the highest proportion of patients aged ≥ 65 years. In both trusts, wards with the highest proportion of older patients were MOP, so, to aid diversity of the sample, we also specified that one of the wards needed to be a surgical ward with a high proportion of older patients. Senior nurses were also encouraged to include those wards perceived as less effective in their general performance in addition to those perceived as high-performing wards, because there is a need to know if interventions of this kind can also work in contexts in which staff may not recognise the potential benefits of the intervention and/or do not prioritise improving compassionate care.

Critical care units were not eligible for inclusion, as previous research indicates that nurses’ experiences of providing relational care in critical care environments are very different from their experiences of providing care in more general wards. 15 Wards were also excluded if the departure of the ward leader was anticipated in the subsequent 6 months of attempted recruitment, as stable ward leadership was theorised to be an important influencing factor on implementation.

Following the circulation of written information about the study, the chief investigator met with the ward leader for each of the nominated wards. They were given a verbal explanation about the study and a chance to ask questions and discuss the implications of taking part. At the close of the discussion, they were invited to put their ward forward for the study or not, or, if they preferred, to take more time to make up their minds.

Once the wards were recruited, all nursing staff, including registered nurses (RNs) and health-care assistants (HCAs) employed to work in the participating ward teams, were eligible to take part in the study and recruitment processes were designed to maximise their inclusion. Meetings with ward leaders were designed to inform staff about the study. Researchers also offered to visit the wards to talk to the teams about the study and this took place on three wards. Written information sheets about the study were also given to ward leaders to distribute to staff. Posters about the study were displayed on the participating wards in public and staff areas.

Addressing the process evaluation and the pilot CRT in turn, the following sections outline sampling, recruitment and data collection.

Process evaluation

The process evaluation aimed to identify and explain the extent to which the planned intervention was implemented into existing nursing practices on the four intervention wards. Data were collected using a variety of methods in order to gather insights into different aspects of implementation from different viewpoints. The data collected were:

1. quantitative ward profile data generated by each ward leader at the outset of the intervention

2. qualitative interviews with nursing staff and ward leaders from the participating wards, the PDNs leading CLECC implementation, the matrons overseeing the wards and hospital senior nurses

3. field notes made by PDNs delivering the intervention

4. observations of a sample of CLECC classroom training days and CLECC action-learning sets for ward leaders

5. quantitative records of training delivered.

Data were collected between May 2015 and May 2016 to capture the period just before the implementation period, with follow-up for up to 12 months. Three main phases were used to guide the scheduling of data collection in the process evaluation: period 1 (baseline/early CLECC active training; May–August 2015), period 2 (mid-to-late CLECC implementation period; July–October 2015) and period 3 (after implementation period; October 2015–May 2016).

Pilot cluster randomised trial

There were two main data collection phases for the pilot CRT and data were gathered from patients, staff and visitors. To enable us to gather data on baseline characteristics and assess against the selected outcomes, we used the following methods:

• observations of staff–patient interactions using QuIS42

• patient questionnaire survey comprising PEECH,100 Picker Patient Experience Questionnaire (short-form)100,115 to measure general care quality, EQ-5D-5L93 and participant demographic details100–102

• nursing staff questionnaire survey comprising JSE, MBI, CC, FECC, selected items from the IHOS battery and participant demographic details26,104,107–109

• visitor questionnaire survey comprising CEC and participant demographic details26

• matron questionnaire survey comprising MAQC26

• ward leader ward profile

• qualitative interviews with nursing staff.

Copies of the questionnaires used are available on request.

We also piloted qualitative interviews about relational care on the wards with a small number of patients (n = 12) and visitors (n = 12) to inform a future process evaluation. An overview of the method and findings from the patient and visitor qualitative interviews is reported in Appendix 5 .

Following baseline data collection in March and April 2015, the ward teams were randomised to intervention or control conditions. The CLECC intervention was implemented on four of the six wards from June 2015, starting on each ward with a 4-month implementation period. Follow-up data were collected on all six wards during February and March 2016.

Pilot cluster randomised trial recruitment

The procedures outlined here mirrored our envisaged procedures for a definitive trial. The opportunity was taken during the feasibility study to evaluate these procedures and further develop them when needed. Pre-screen and screening logs were developed to enable assessments of:

• the timeline of the introduction, approach, discussion and consent process

• the number of people assessed for eligibility

• the number of people approached to join the study

• the number of people recruited into the study

• the number of declined offers and the reasons for these decisions

• participation rates of older patients and patients with cognitive impairment

• the achievement of targets.

Patient recruitment to observations

All adult patients on participating wards were assessed for eligibility to be included in observations of care. Patients were excluded if they were unable to communicate their choices about taking part in the research and a consultee (as defined by the Mental Capacity Act 2005 Code of Practice)110 could not be consulted. Patients who indicated either verbally or non-verbally that they did not wish to take part were excluded, as were patients who were unconscious or those for whom there were clinical concerns that may preclude them from being approached. Patients excluded for clinical reasons included people who were critically ill, at the end of life, or isolated because of a high risk of infection.

The patient sample for observations was determined by randomisation. Up to 24 hours in advance of each scheduled observation, all eligible patients on the ward at the time an observation was scheduled were identified. Patients were placed in a list in alphabetical order of names and each patient was then allocated a number in sequence. A random number generator was then used to select an index patient from the pool of eligible patients.

Each index patient was then approached and informed (verbally with accompanying written information and with aids when needed) about the planned observations. Recruiting researchers were trained to be person-centred and patient, allowing sufficient time for successful communication, and to make environmental modifications to optimise communication. If the index patient indicated verbally or non-verbally that they were happy for the observation to proceed, other eligible patients in the researcher’s field of view were approached, informed about the planned observations, and if they indicated that they were happy for the observation to proceed their care was included in the observations. We did not ask for written consent from patients, but consent was instead recorded by the researcher. If the index patient declined to take part, another index patient was randomly selected, and approached and invited as before. The observations proceeded with data being collected on interactions with all patients who had agreed that the observations could proceed. When an assessment was made that a potential participant did not have capacity to make a decision about taking part in the research, advice was sought from a consultee, in line with the requirements of the Mental Capacity Act 2005 Code of Practice. 110 Figure 4 outlines these processes.

FIGURE 4.

Recruitment process for observations.

Patient sample and recruitment to questionnaire survey

An adapted census approach was used with the patient questionnaire survey. Researchers aimed that all eligible patients on the ward on the scheduled day for questionnaire data collection would be invited to complete a questionnaire. Questionnaire data collection days were planned in advance for each ward to ensure researcher availability, but timing of planned data collection during that day meant that patients would be least likely to be involved in other activities, such as washes, meal times or doctors’ rounds. Before distributing questionnaires, researchers ascertained with the nurse-in-charge which patients were able to be directly approached by a researcher to complete the questionnaire. Patients were excluded if they were critically ill, in receipt of palliative care or at high risk of infection. All eligible patients were then approached and assessed further for eligibility, particularly mental capacity. If the researcher was able to confirm eligibility, patients were informed about the research and invited to take part.

Staff sample and recruitment to questionnaire survey

We planned a census approach to the nursing staff survey in that all RNs and HCAs employed to work in the participating ward teams would be invited to complete a nursing questionnaire. Ward leaders were asked to provide a list of RNs and HCAs employed to work on their ward. Nursing questionnaires were then placed into individually named envelopes and given to ward leaders to distribute to staff.

Visitor sample and recruitment to questionnaire survey

An adapted census approach was used with the visitor questionnaire survey. Researchers aimed that all eligible visitors on the ward on the scheduled day for questionnaire data collection would be invited to complete a questionnaire. Questionnaire data collection periods were planned in advance to ensure researcher availability but also to coincide with visiting time on the individual wards. No exclusion criteria were set for visitors, and so any adult visitors on the ward during a data collection period were approached, informed about the research and invited to take part.

Staff recruitment to qualitative interviews

In addition to the qualitative interviews conducted as part of the process evaluation (which focused on the CLECC intervention implementation), intervention and control ward nursing staff members were also interviewed about ward characteristics, specifically relational care, teamwork and leadership at baseline and at follow-up. The purpose was to provide summary baseline data of group characteristics to inform the pilot CRT. Intervention ward staff recruited to interviews as part of the process evaluation were also questioned about these characteristics during their interviews (see Pilot cluster randomised trial recruitment and Qualitative interviews ). A group of control ward staff was also sampled and recruited in the same way, with their interviews focusing solely on relational care, leadership and teamwork. All RNs and HCAs on control wards were eligible to take part in the qualitative interviews and were invited to take part through posters, presentations and e-mails. We purposively sampled from those who volunteered in order to capture variations in staff grade and ward. Written consent was sought for the staff interviews and staff were given information about not being obliged to take part and their right to withdraw their consent at any time. We offered a payment of a £15 shopping voucher to individual staff who completed an interview. Individuals recruited at baseline were invited to a second interview at follow-up. When such individuals could not be contacted or declined further interviews, new individuals were recruited at the same grade and from the same ward to ensure that variation by ward and grade was maintained.

Economic evaluation

The economic evaluation drew on two main data sources outlined above. These were the EQ-5D-5L data from patient questionnaires and the qualitative interview data from staff that focused on the resource implications of CLECC. In addition, the cost of providing CLECC as an intervention was explored.

Chapter summary

This chapter has provided information about the key data sources used to address the study objectives, in particular sampling, recruitment and data collection in relation to the process evaluation, the pilot CRT and the economic evaluation. Chapter 6 provides detail on data analyses.

Process evaluation

Data were analysed using systematic reading, familiarisation and open coding, undertaken independently by research team members and then in collaborative data analysis workshops. Coding discussions led to the development of two coding frames: the first to enable the exploration of themes related to relational care, ward leadership and teamwork across all six wards (for the CRT element), and the second, from intervention wards only, to focus on CLECC implementation and mechanisms of impact. All qualitative interview data from nursing staff were coded against these frames, with the use of constant comparative methods enabling the generation of new categories and the comparison of data in relation to these categories.

The relational care coding frame ( Figure 5 ) was designed to support comparisons across the wards and, so, analysis focused on summarising and describing what people had to say about relational care, leadership and teamwork.

FIGURE 5.

Relational care coding frame.

A deeper level of analysis was undertaken in relation to the CLECC implementation frame and this part of the analysis was the focus of the team’s work. There was a conscious decision to avoid prematurely ‘fitting’ the data into NPT domains, and the use of the coding frame shown in Figure 6 enabled preliminary descriptive coding in relation to implementation and mechanisms of impact. The constant comparison method was used to examine the codes generated against the framework, which was further developed as analysis progressed. Narrative data summaries and matrix/charting techniques were then used to facilitate comparison with the NPT framework and test and refine emerging theories of implementation processes.

FIGURE 6.

Coding frame for analysis of CLECC implementation. Based on MRC guidance89 and Pawson. 116 a, Anything external to CLECC that impedes/strengthens implementation.

Findings from the analyses of the qualitative interview data were triangulated with qualitative data from field notes kept by the PDNs leading the delivery of CLECC and researcher field notes from observations of CLECC training activities.

Most, but not all, of the team were researchers with a nursing background and had not met the research participants prior to the study. All were involved in interviewing and/or observing the staff in the study, and in data analysis. In early stages, more than one researcher coded the same data set, with comparisons between coding decisions creating opportunities for team discussion and the development of shared approaches. The deliberate use of reflective techniques during collaborative analysis workshops enabled individual views and assumptions to be surfaced and explored with a view to enhancing the quality of analysis. NVivo11 (QSR International, Warrington, UK) was used to support preliminary coding and consistency across the team.

Pilot cluster randomised trial

Data were analysed using SPSS 22117 and the significant threshold (alpha) was set at 0.05 (two tailed). Stata 14 was used for multilevel logistic model. Exploratory data analyses were performed to check the data and identify inconsistencies. The primary purposes of tests of effectiveness were to pilot procedures for analysis and to inform effect-size calculations for a future study.

Economic evaluation

The cost of the CLECC intervention comprised one-off training costs and ongoing implementation costs. Training costs were agreed with senior managers from participating hospitals, using standardised models for calculating the cost of staff time when relevant.

The CLECC implementation costs focused on the ward-based cluster discussions and the extent to which these required additional nursing staff time, with potential implications for nurse staffing ratios. Twenty-one transcripts from T3 qualitative interviews with intervention ward nursing staff were reviewed and annotated by James Raftery with a focus on data on resource implications of CLECC. These analyses were discussed with interviewers and with Jackie Bridges to validate emerging hypotheses as regards the implementation costs of CLECC.

To aid calculation of the benefits of CLECC in a future trial, the utility of the most used quality-adjusted life-year (QALY) outcome measures was explored, namely the EQ-5D-5L. The EQ-5D-5L was administered to patients in the relevant wards at baseline and at follow-up. Completion rates were estimated. EQ-5D-5L baseline and follow-up scores were translated to quality-of-life scores using the national tariff and compared by ward at each time point. 93

Chapter summary

These chapter has outlined the various methods used for analyses of the process evaluation, pilot CRT and economic evaluation.

Recruitment and flow of ward teams (clusters)

Six ward nursing teams across two hospitals took part in the study, with three ward teams in each hospital. Ward leaders in all of the three nominated teams in Hospital A agreed to take part on behalf of their team. Two ward leaders in Hospital B agreed to take part, but a third ward leader nominated in Hospital B declined. Their team was about to embark on a quality improvement project and the leader was concerned that participating in the CLECC study as well would be too burdensome for the team. The matron concerned nominated an additional ward team caring for a high proportion of older patients, and this team’s ward leader agreed to take part. Each of the ward leaders was consulted about the prospect of randomisation to intervention or control conditions and no concerns were raised about this feature of study participation, indicating that the randomisation strategy was acceptable to participating staff. Thus, six ward teams entered the study and all remained in the study until all data collection was complete.

Both participating hospitals were NHS trusts located in separate urban areas in the same geographical region in England. Hospital A is a university hospital and foundation NHS trust employing between 7000 and 8000 staff and providing services to > 1 million people plus a wide range of specialist services to > 3 million people. Hospital B employs > 6000 staff and provides acute services to > 0.5 million people living locally and provides a smaller range of specialist services than Hospital A. The urban areas served by both hospitals have pockets of high deprivation. Life expectancy at birth is marginally higher than the national average in Hospital A locality and lower in the Hospital B locality. Hospital A locality has a lower percentage of the population than the English average who are classified as being from the white ethnic group and Hospital B locality has a higher than average white population.

Individual ward characteristics

Table 8 shows an overview of the characteristics of individual wards in the study, taken from the ward profiles completed by each of the ward leaders at baseline. Four wards were MOP wards and two were surgical wards. All the wards had a similar number of beds, with Hospital A wards having fewer beds than Hospital B wards. There was more variation between the wards in mean length of stay, ranging from 6 days on one of the surgical wards to 19 days on the other surgical ward. All four MOP wards had a similar mean length of stay (13–14 days). Planned full-time equivalent (FTE) staffing levels (RNs and HCAs) and nursing skill mix (proportion of registered nurses in total of RNs and HCAs) varied between the wards. The two surgical wards had higher planned staffing levels and skill mix than the MOP wards. In each ward in Hospital A, the numbers of actual staff in post were lower than planned, whereas for all wards in Hospital B, the staff numbers in post were higher than planned. All the wards used a mixture of short (approximately 8 hours) and long (approximately 12 hours) shifts for nursing work during the day.

There was substantial variation between the wards in the proportion of patients on the ward who ward leaders judged needed help with all of their activities of daily living (ADL), an indicator of patient dependency and thus need for help from nursing staff with these activities (0–45%). Wards in Hospital A had fewer beds and lower numbers of actual staff in post than Hospital B. Wards leaders in Hospital A also identified a higher proportion of patients needing help with all their ADL than Hospital B.

Ward leadership characteristics

All wards had a ward leader (senior sister) and deputy in post. Table 9 shows the length of experience for each ward leader in that role on that ward, as a ward leader overall and in any role on that ward. Wards A, D, E and F all had ward leaders with at least 9 years’ experience as a ward leader, whereas ward leaders for wards B and C were new or relatively new to the role. The ward leader for ward B took up post, their first at that level, at the beginning of the study. Only the ward F ward leader worked in a totally supernumerary role, with the others regularly or often being counted in the staffing numbers on a shift.

Quality of care

Table 10 shows the results of the MAQC completed through written matron’s survey. A completed matron’s assessment was received for each ward (n = 6). Higher scores indicate more favourable ratings. The wards have similar scores across both subscales with the exception of ward F, which had markedly lower scores for both subscales.

Relational care

A sample of nursing staff (n = 29) from all wards was interviewed using qualitative methods to ascertain their understanding of relational care on the ward where they worked. There was no significant difference between wards and hospitals in the way that nurses generally described relational care. The term compassion was used with examples given of interactions that staff perceived as conveying compassion in their care delivery, for example offering a relative a cup of tea to make the patient more relaxed because patients see we are looking after their family as well (N002 staff nurse). There was a recognition among nurses of interactions that patients do not perceive as compassionate, for example getting on with a nursing intervention with a patient without first introducing yourself and what you intend doing (N001 staff nurse). Staff explained that this happened because of time pressure to complete all expected nursing activities. Nurses who admitted to not always introducing themselves or actively getting to know the patient perceived these episodes of care as lacking compassion. Time was reported as a significant factor in delivering compassionate care, with staff challenged by patients who appear to need more time than others to express their needs or concerns. However, staff felt that compassion can be transmitted even in few words, for example when someone is clearly distressed because they have been incontinent, taking care with tone of voice, saying ‘it’s OK’ (N011 staff nurse).

Ward leaders reported incorporating discussion about compassionate care delivery with their ward team. They reported that a number of wards had patients who needed individual care, patients who tried to walk but, because of cognitive impairment, were unaware of their physical limitations to do this effectively and safely, and so their care required an increase in the usual staff complement. Agency staff were employed to provide this individual care and ward staff reported seeing them work without compassion, for example sitting and watching the patient rather than interacting with them. Ward leaders reported actively engaging with all nursing staff who interact with patients on the ward, including agency staff, to promote staff expectations of compassionate care delivery.

Care on the ward was reported not to be focused solely on medical management of a patient but also on the incorporation of other interventions seen as important to an individual’s well-being. Staff saw their role as offering choice to patients and this was reported to be led and supported by ward leaders, for example by offering the use of a dining table for patients who are able with or without assistance to get there (N034 ward leader). Relational care was talked about as an important dimension to the nursing role, and this was reflected in stories about connections developed between individual patients and individual staff members, and the sadness felt by staff when these patients died. This was especially the case when the staff member felt that they had been successful in providing good relational care, and that the relationship had been strong. All of the ward leaders considered role modelling and educating the nursing team to deliver good relational (compassionate) care to be an important part of their role.

Nurses on all wards shared an understanding of good relational care and felt that the general public perceived nurses as being less compassionate than they used to be. Examples were readily available of what constituted good relational care, as well as the barriers to and facilitators of the achievement of this standard. There was consensus that the degree to which care was compassionate varied, but that it was not ward specific, rather it was both time and staff dependent. Nursing staff were aware of a variety of initiatives that had been put in place to address the perceived lack of compassion in care delivery, citing the Friends and Family Test119 most frequently. Staff generally were keen to find or be offered strategies to assist them in delivering good relational care more consistently. The following quotation reflects what ward leaders expected and what nurses expected to be able to deliver:

I’m expecting them to look after the whole person, see what their needs are, help them with those needs and make them feel like the most important person they’re dealing with at that point in time.

N031 ward leader

Staff well-being

Table 11 displays findings from the baseline ward profile completed by the ward leader in relation to indicators of staff well-being. All the wards had new staff members joining over the previous 6 months, with wards D and E having the highest number of new starters. Sickness absence rates varied considerably, from 0.6% on ward A to 10.7% on ward D. All the wards used bank or agency staff regularly, at least several times per week. On ward A, staff very rarely missed their breaks, whereas missing breaks happened more frequently on the other wards and on a daily basis on ward C.

Table 12 indicates the degree of burnout taken from baseline nursing survey responses (n = 91). Higher mean emotional exhaustion and depersonalisation scores, and lower personal accomplishment scores, indicate less favourable conditions and higher burnout. Wards A and E had a mean emotional exhaustion score lower than the overall mean, and wards A, D and F had a lower mean score for personal accomplishment. Wards A and D had a higher personal accomplishment mean score.

Table 13 shows the proportion of staff on each ward experiencing burnout calculated from nursing survey responses (n = 91). Emotional exhaustion varied from 27% of 22 staff on ward E to 50% of 18 staff on ward C. Depersonalisation ranged from 0% of 12 staff on ward A to 40% of five staff on ward B. Personal accomplishment ranged from 10% of 21 staff on ward F to 22% of 18 staff on ward C.

Other questionnaire results

Baseline and follow-up results for all questionnaire data are reported in Appendix 8 .

Individual participant flow

This section and the accompanying CONSORT diagram ( Figure 7 ) focus primarily on individual participant flow in relation to the pilot CRT. Later, in Nursing staff qualitative interviews , but not reflected in the CONSORT diagram in Figure 7 , data are also shared about participant flow in relation to nursing staff in the qualitative elements of the study.

FIGURE 7.

The CONSORT flow diagram for pilot cluster CRT. Reproduced from Gould et al. 2018. 120 © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

Reproduced from Gould et al. 2018. 120 © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/.

All four wards randomised to the intervention went on to receive the intervention. Figure 7 shows the flow of clusters and participants through the pilot CRT. Randomisation took place after baseline data collection, but the results are presented by allocation for baseline and follow-up data to enable comparisons between groups to be made.

A number of data from nursing staff, patients and visitors were gathered at baseline and repeated at follow-up to enable study outcomes to be assessed. Follow-up was at the cluster level rather than at the individual participant level. For instance, eligibility of patients and visitors was dependent on their being an inpatient during both assessment periods, which was an unlikely scenario given that they were 1 year apart. In addition, some turnover of staff employed on the ward teams was anticipated. Therefore, to maximise recruitment at each assessment point, recruitment for baseline and follow-up assessment periods was independent.

Baseline measures

This subsection reports the baseline scores for each ward on the study’s core outcomes: quality of staff–patient interactions, patient evaluation of emotional care and nursing self-reported empathy. It also compares these outcomes by experimental group in the study, that is, intervention or control.

Chapter summary

Six out of seven ward teams invited to take part participated in the study and all remained in the study until data collection was complete. Ward teams were randomised to intervention (n = 4) or control (n = 2), and staff were amenable to the prospect of randomisation to either experimental condition.

This chapter has described the six participating wards at baseline assessment across a range of characteristics including specialty, patient dependency, staffing, ward leadership, ward climate, staff well-being, care quality, patient safety and core study baseline measures. The findings reflect a range of ward contexts, with similarities across some dimensions, such as bed numbers. Some differences between wards were evident, including staffing levels and duration of ward leadership. Matrons’ assessments of ward quality indicate that for all but one low-scoring ward, matrons assessed the care quality as good but not of the highest possible quality. Our qualitative interviews with staff did not identify particular differences between the wards, and, although all talked about the value of relational care, staff also reflected that limited time with individual patients constrained their capacity to be compassionate. Most staff–patient interactions observed were rated as positive. However, a proportion of interactions on all wards were rated negatively. In total, 40% of patients observed for 2 hours were rated as having one or more negative interactions during the 2-hour period. This proportion varied widely between wards (with intervention wards having a higher proportion than control wards) but indicates room for improvement on all participating wards. Staff survey results indicated that staff on all the wards showed signs of burnout; this was the case for over one-third of respondents. However, low response rates reduce certainty in the representativeness of these proportions in the staff group as a whole. Some individual staff on all the wards rated their empathy levels as low, others as high, and mean empathy levels for ward teams varied, with intervention wards having higher reported empathy. Low staff survey response rates suggest that these results need to be treated with particular caution. Overall, although these findings indicate some differences between the wards at baseline, there are also some remarkable similarities, particularly in relation to the quality of relational care, which is largely positive, yet with clear scope for improvement, and in relation to the presence of burnout in the staff group.

Although we were not successful in recruiting high proportions of nursing staff to take part in the written survey, the staff that did take part represented a wide range of characteristics in terms of work role, seniority, professional experience, age and ethnic group. Recruitment of patients and visitors was more successful, with the observations of staff–patient interactions attracting a particularly high proportion of recruits. Although not as high, recruitment to patient survey was still at a good level. Most patients involved in observations and the survey were older and female. One-quarter of patients who were observed had evidence of cognitive impairment, compared with 12% of questionnaire respondents. A high proportion of visitors approached to complete a questionnaire agreed to do so, with an average age of 62 years. The findings also indicate that the characteristics of participating staff, patients and visitors were equivalent across the experimental groups.

Implementation overview

The CLECC intervention was introduced to four ward teams in two hospitals. Its implementation in each hospital was led during a 4-month implementation period (beginning June 2015) by a PDN who was recruited and employed by the hospital and seconded from another role. Over the course of the implementation period, the role of the PDNs was to work with the ward leaders and the wider team to promote ownership of CLECC, to develop the learning activities into ways of working and learning that were seen to be feasible and helpful to the team and to develop a plan for sustaining what was good about CLECC after the departure of the PDN. The PDNs were mentored in their preparation for the implementation period by Jackie Bridges from the research team, working with a range of learning materials to prepare them for their role. The PDNs from each hospital communicated with each other regularly and also met up with Jackie Bridges for mentoring every 2 weeks at the outset, and then less frequently as time progressed.

Three ward teams ran the implementation period at the same time, and a fourth team started a few weeks later. The implementation period began for each ward team with the study days. Study days were exclusive to each ward team (i.e. they were not shared by participating teams from the same hospital). Each ward team had three study days led by the PDN and one-third of team members (RNs and HCAs, day and night staff) attended each of the days. This arrangement was intended to ensure that all team members had a chance to attend a study day. At both hospitals, a final ‘mop-up’ study day was held to include staff from both wards who had not made it to any of the previous study days. Ward leaders were invited to participate in all three study days and all did, with the exception of one ward leader who arranged for a deputy to attend. Study days included an introduction to CLECC principles. Staff were invited to complete CC and FECC questionnaires on the study day and to give a written response to the phrase ‘I feel valued when . . .’. They also undertook a values clarification exercise on compassionate care, but the main focus of activities on the day was to create opportunities for staff to talk together and listen to each other about what is was like to work on the ward and to develop a shared vision for care.

Ward-based learning activities followed the study days. The PDNs initiated and trained staff in the cluster discussions. PDNs trained staff nominated for peer observations, arranged the observations of care and analysed the results. PDNs also analysed the material that staff had completed on the study days and they encouraged meetings between the ward leaders and matrons. Reflective learning sessions for staff were planned by the PDNs who arranged for a senior nursing manager to visit the ward at the end of the implementation period to hear about how the team had used CLECC and what their plans were for taking their ideas forward. They also arranged and facilitated the action-learning sets for ward leaders. The action-learning sets were held for staff from both hospitals jointly so that the set was larger than two members and the PDNs could share the facilitation.

The process of normalising Creating Learning Environments for Compassionate Care into practice

The findings that follow illustrate the amount of cognitive and behavioural work that staff needed to do, individually and collectively, to normalise CLECC into practice. Although many of the individual elements of CLECC were possible to implement during the 4-month implementation period, sustaining this work beyond this time was difficult for some ward teams to achieve, and the findings that follow explain why this was the case. Findings are presented under the four NPT constructs and draw together data from all data sources.

Chapter summary

This chapter shared findings aimed at identifying and explaining the extent to which the planned CLECC intervention was implemented into existing work practices, to enable conclusions to be drawn about how interventions of this kind can be optimised to support sustained compassionate care delivery in acute settings. Findings reflect that some but not all CLECC activities were feasible to implement into practice, with a variety of factors influencing their impact and sustainability. Staff were generally keen to participate and valued the positive contribution of CLECC to their own well-being, to more cohesive team working and to supporting good patient care. Many original CLECC practices did not continue beyond the implementation period, but staff reported that the philosophy and associated culture that CLECC had nurtured continued to guide their practice. Sustainability was strongly linked by staff to the extent to which the ward leader understood and valued CLECC.

Findings indicate that CLECC had some coherence for staff in that they appreciated its potential value, but their understanding was often limited to the concrete activities of which they had direct experience. In terms of cognitive participation, staff were keen to participate, but ward teams varied in the extent to which individual members saw it as their role to make sure that CLECC happened, and there was uncertainty about the role of matrons in supporting CLECC. There was strong evidence of collective action to implement CLECC, with team members generally participating when the opportunity arose. However, the ability to act or not was shaped by the extent to which activities harmonised with the priorities of the wider organisation. Although staff valued CLECC, its sustainability was linked to factors outside the direct control of the nursing team. These findings are discussed further in Chapter 11 .

Pilot cluster randomised trial assessment of bias

Pilot cluster randomised trial outcome measure performance

This section reports on the performance of the three core outcome measures: (1) quality of staff–patient interaction using QuIS observations, (2) nursing staff self-rating of empathy using JSE in the nursing written survey and (3) patient evaluation of emotional care using the PEECH instrument in the patient written survey. It also reports on the performance of the EQ-5D-5L in measuring benefit as part of the economic evaluation.

EuroQol-5 Dimensions, five-level version, health status

The EQ-5D-5L was administered as part of the patient questionnaire survey to inform the economic evaluation. The instrument was initially located on the patient questionnaire, before the last item which asked for demographic details, but researchers found that starting the interview with asking about demographic details relaxed patients and was a good way to assess mental capacity. As a result, the EQ-5D-5L was often the last part of the patient questionnaire to be completed when researchers were helping with questionnaire completion.

Researcher field notes reflect that patients had difficulties understanding the EQ-5D-5L measure, especially the visual analogue scale, which requires subjects to mark on a numbered scale (0 to 100) how good their health is today. These concerns are reflected to an extent in the analysis of missing EQ-5D-5L data shown in Table 27 . In total, 89% (i.e. 150 out of 168) of baseline patients and 76% (i.e. 141 out of 186) of follow-up patients gave responses in all five domains of the EQ-5D-5L.

TABLE 27 - Summary of EQ-5D-5L data (at baseline and follow-up)

Time	Participants	EQ-5D-5L domain (n)					All (n)
		Mobility	Self-care	Activity	Pain	Anxiety
Baseline	Missing	11	10	16	9	9	18
	Recorded	157	158	152	159	159	150
	Total	168	168	168	168	168	168
Follow-up	Missing	19	17	23	17	18	27
	Recorded	167	169	163	169	168	159
	Total	186	186	186	186	186	186

The EQ-5D-5L scores ranged widely among patients, from 0.39 to 1 as shown in Figure 8 . A total of 13% (i.e. 19 out of 149) of patients had a negative EQ-5D-5L score (< 0). There was no strong correlation between EQ-5D-5L score and age or gender.

FIGURE 8.

Scatterplot of EQ-5D-5L scores (at baseline and follow-up).

Mean EQ-5D-5L index values across six wards and two experimental groups ( Tables 28 and 29 ) are lower than general population by age (0.75 for age > 75 years). A lower index value represents a lower quality of life, with 1 representing the best health possible and 0 representing death. Lower than average values in this population are expected given that respondents were in hospital. The EQ-5D-5L score (0.412) in the intervention group was lower than that in the control group (0.502), but there was no statistically significant difference in EQ-5D-5L score between both groups (mean 0.09, 95% CI –0.026 to 0.205). EQ-5D-5L scores were different at follow-up. Given that different patients were involved at each point, these changes, which were both positive and negative, cannot be interpreted as changes attributable to CLECC. Only if the different groups of patients in each ward had the same ailments and severity at baseline and follow-up could changes be attributed to CLECC.

TABLE 28 - The EQ-5D-5L scores by ward and experimental group (at baseline)

EQ-5D-5L score	Trial group
	CLECC					Control
	A (n = 25)	B (n = 19)	D (n = 31)	E (n = 30)	Total (n = 105)	C (n = 20)	F (n = 43)	Total (n = 63)
Mean	0.337	0.472	0.452	0.395	0.412	0.551	0.476	0.502
Lower 95% CI	0.189	0.281	0.332	0.262	0.344	0.376	0.353	0.404
Upper 95% CI	0.485	0.663	0.571	0.527	0.48	0.726	0.599	0.6
Minimum	–0.283	–0.436	–0.223	–0.2	–0.436	–0.352	–0.23	–0.352
Maximum	0.877	1	0.877	1	1	0.879	1	1

TABLE 29 - The EQ-5D-5L scores by ward and experimental group (at follow-up)

EQ-5D-5L score	Trial group
	CLECC					Control
	A (n = 32)	B (n = 33)	D (n = 29)	E (n = 29)	Total (n = 123)	C (n = 31)	F (n = 32)	Total (n = 63)
Mean	0.314	0.545	0.463	0.419	0.433	0.651	0.572	0.602
Lower 95% CI	0.195	0.4	0.337	0.285	0.368	0.479	0.466	0.513
Upper 95% CI	0.433	0.69	0.589	0.553	0.497	0.823	0.677	0.692
Minimum	–0.2	–0.166	–0.2	–0.358	–0.358	–0.307	–0.209	–0.307
Maximum	0.846	1	1	1	1	1	1	1

The results of the QALY analysis based on the EQ-5D-5L data showed large differences between wards at baseline and follow-up. This was not surprising given that different patients with different ailments and severity were involved in each ward at baseline and follow-up. Consequently, although patients could be classified into Healthcare Resource Groups (HRGs; the basis on which NHS hospitals are paid), this does not include any measure of severity. Data on the quality of life of patients would need to be adjusted for severity if changes in EQ-5D-5L were to be interpretable. No agreed severity measurement exists and none is routinely collected in the NHS. Apart from diagnosis, procedure and demographic data, no other relevant data are collected. As a severity casemix measure was not used in this study, it is not possible to explain the differences in QALYs that were found.

Feasibility of estimating costs of Creating Learning Environments for Compassionate Care

Initial training costs were feasible to calculate and primarily focused on the cost of ward staff time attending the study day and the cost of PDN time to facilitate the implementation period ( Table 30 ).

Future calculations of cost-effectiveness will depend on the length of time for which any benefits of CLECC are sustained beyond the initial investment of these training costs. If, for instance, the training costs are one-off costs with the effects sustained over an infinite period of time these costs become negligible when assessing cost-effectiveness and only ongoing implementation costs are relevant over time. Qualitative findings suggest that CLECC practices can be sustained for up to 12 months beyond the start of the implementation period. The cost of attaining and sustaining the benefits of CLECC over a 12-month period are therefore £53 per day per ward (£19,503/365), or £2 per bed-day (based on 30 beds per ward) [£19,503/(365 × 30)]. Were the benefits sustained over a 24-month period with no further training required, these costs would be halved.

In terms of ongoing costs of implementation, the results from qualitative interview data indicated that nursing staff implemented CLECC cluster discussions on the ward in short 5- to 10-minute sessions. Those who commented on whether or not extra time needed to be scheduled for such sessions suggested that it could be met within existing time schedules. Sessions remained within the short 5- to 10-minute slots, which was not seen as onerous. When wards became particularly busy, the sessions did not take place. CLECC sessions were popular with all the nursing staff interviewed. CLECC ‘champions’ were seen as important in ensuring that sessions took place. The nursing staff interviewed reported that the implementation of CLECC did not require additional nurse time. They considered that ward leadership was more important to ensure that cluster discussions happened, especially at busy times.

Chapter summary

This chapter has reported a number of important findings that can inform the design of a future evaluation. Findings indicate no selection bias inherent in our study recruitment processes and methods. The methods employed were inclusive of all staff levels and also inclusive of older patients with a range of cognitive abilities, ensuring excellent representation from a traditionally hard-to-reach group who are often excluded from research but who are prone to more negative experiences of hospital care. There was some evidence that observer researchers could find out the experimental group of the wards on which they were observing and this will need careful attention in a future trial. In addition, the findings of pathways for contamination beyond the intervention wards mean that any future trial design will need to avoid running intervention and control conditions in the same organisation at the same time. A more positive interpretation of these same findings is that the CLECC intervention appears to have the potential for impact beyond the target ward teams and future evaluations should aim to explore this potential.

The QuIS tool was highly acceptable to patients recruited and had the highest rate of participation by patients with cognitive impairment of all methods. We demonstrated that our recruitment and data collection plans for QuIS use were feasible in busy ward environments. The validity and reliability of QuIS was acceptable. The QI Tool software performed well, enabling straightforward data collection and upload.

Patient questionnaires, despite being very acceptable to patients, had a lower participation rate than QuIS for people with cognitive impairment. Most patients needed researcher help and there were complaints that it was too long. The PEECH and EQ-5D-5L scales were hard for some patients to use.

The response rate to nursing questionnaires was very low, particularly at the follow-up stage on the intervention wards, perhaps owing to research fatigue. There was a perception that questionnaires were lengthy to complete and that staff were too busy. The low response rate means reduced certainty that the responses to the questionnaire represent the views of the nursing staff as a whole.

A patient outcome measure, such as EQ-5D-5L, does not appear to be feasible to use to evaluate value for money for two reasons. First, given that some two-thirds of patients needed help to complete the questionnaires, it could be used in a full study only as a proxy measure. Second, and more importantly, interpretation of EQ-5D-5L scores as measures of health improvement is not possible as different patients, with different ailments and severity were involved at baseline and at follow-up. Without an adjustment for the various ailments and their severity, differences in EQ-5D-5L scores cannot be interpreted.

Through qualitative interviews with staff, we were able to establish that, aside from initial CLECC training costs (cost of staff time at team study day and cost of employing CLECC PDN), the implementation of concrete CLECC activities by ward teams was not associated with additional resource use. Nursing staff reported that the CLECC cluster discussions typically took 5–10 minutes each shift. They were dropped when wards became too busy to have them. We were also able to identify other candidate activities associated with supporting CLECC that merit attention in a future evaluation.

Quality of staff–patient interactions

The observed quality of staff–patient interactions, assessed using QuIS scores, between experimental groups at follow-up is shown in Table 31 . The distribution of interaction scores across the five available categories is broadly similar to baseline distribution, with most interactions rated as positive care and fewest interactions rated as negative protective for each experimental group.

There are more positive (social plus care) and fewer negative (protective plus restrictive) scores for intervention wards than for control wards at follow up (78% vs. 74%, 8% vs. 11%). Chi-squared testing of these results suggested a significant difference between experimental groups (p = 0.017), but the results shown in Table 32 indicate that, once other variables are taken into account in the analysis, the odds of a negative interaction are not significantly reduced because of the effect of the CLECC intervention. Results are in the direction of an effect favourable to CLECC, that is, there were fewer negative interactions on intervention wards, but this was not a statistically significant difference.

The proportion of negative interactions per patient was calculated on all of the QuIS data (n = 140 patients at follow-up). The results in Table 31 indicate that, although some patients had no negative interactions, others had up to 67% of their interactions rated negatively. On average, 10% of interactions per patient at follow-up were rated as negative.

In total, 92% of patients (i.e. 129 out of 140) at follow-up were observed for a full 2-hour period. For patients observed for the full 2-hour period, the number with one or more negative interactions during that 2-hour period was calculated ( Table 31 ). Findings did not differ significantly by experimental group (chi-squared p = 0.744).

Table 33 illustrates the total negative interactions by individual ward. At the individual ward level, some wards (n = 3) appear to have improved their QuIS ratings and some appeared to have deteriorated (n = 3). The proportion of patients experiencing at least one negative interaction in a 2-hour period varies between individual wards from 25% to 64%. Two of the intervention wards showed an improvement between baseline and follow-up (i.e. the proportion of patients with negative interactions decreased), two showed a deterioration and the two control wards also showed deterioration.

Patient evaluation of emotional care

Patient evaluations of emotional care using the PEECH tool, administered through written patient survey, did not differ significantly by experimental group (Mann–Whitney U-test, p > 0.05), although small non-significant differences in total score and three out of four subscales favoured CLECC ( Table 34 ).

Fewer patients in the CLECC group than the control group had low scores in the connection subscale (63% vs. 79%) ( Table 35 ), but this was not adjusted for any potential differences at baseline or in patient characteristics or ward effects.

The results shown in Table 36 indicate that, once other variables are taken into account in the analysis, the odds of low connection scores are lower on the intervention wards, but not significantly so. Model 2 has been adjusted for ward, Model 3 for ward and baseline and Model 4 for ward, baseline and patient characteristics (age, gender, ethnicity and education level).

The results are in the direction of an effect favourable to CLECC, that is, CLECC may be associated with a reduction in the odds of having a low score in the connection subscale of PEECH. However, this association is no longer significant when we adjust for baseline and patient characteristics.

Patient evaluations of emotional care in each ward at follow-up are shown in Table 37 . Higher scores indicate better patient-reported experiences. As at baseline, connection consistently scores lower than the other subscales. Total mean PEECH scores range from 47.6 on ward C to 53.8 on ward B (out of possible total of 66). There was a small improvement in total mean PEECH score summed for all the wards from baseline to follow-up [48.9 (SD 11.7) to 49.9 (SD 10.8)]. There are variations in the direction of change over time between individual wards, with ward E showing deterioration across all subscales and total PEECH score, and wards B, D and C showing improvement in all subscales and total PEECH score.

Nursing staff self-reported empathy

Levels of self-reported empathy using the JSE scores from the nursing staff survey (RNs and HCAs) varied across the individual wards at baseline and at follow-up. Higher scores indicate greater empathy.

Empathy scores by experimental group at follow-up are shown in Table 38 . Mean and median empathy scores were similar across experimental groups at follow-up. A Mann–Whitney U-test confirmed no significant difference between groups (p = 0.800).

There was a small reduction in mean empathy score for all wards from baseline to follow-up (113 vs. 112). Mean empathy scores decreased on four wards from baseline to follow-up and increased on two wards ( Table 39 ).

Intracluster correlation

At ward level, the ICCs for QuIS, PEECH (subscales and total) and JSE were low (< 0.1). ICC was high at the observation session level for QuIS ( Table 40 ).

Economic evaluation

The implications of the findings reported in this chapter are briefly explored in relation to an economic evaluation. None of the outcomes reported here seems appropriate for use in cost-effectiveness analysis. Such analysis typically takes the form of cost per unit of a particular outcome, usually the primary outcome. The primary outcome here, QuIS, is not expressed as a single number, which rules out any use of cost per QuIS unit. The same broadly applies to PEECH. Although nurse empathy is expressed as a single score, interpretation of a cost per unit of nurse empathy would be difficult. Given these difficulties, the most productive way forward would involve an impact inventory of the sort recommended in a recent authoritative review. 123 Besides being good practice for all economic evaluations, this would ensure that relevant data were presented on the range of costs and benefits involved.

Chapter summary

This chapter has focused on a comparison between intervention and control wards on the core outcomes at follow-up for the study, namely quality of staff–patient interaction, patient evaluation of emotional care and nursing self-reported empathy. The results suggest that CLECC may have a favourable effect in reducing negative interactions between staff and patients, and in reducing patients’ experiences of lack of emotional connection with staff, but, as expected, we did not detect significant differences once other variables were accounted for. We found no evidence that nursing staff empathy improved because of CLECC, but these results in particular have to be viewed in the context of the low response rate to nursing surveys. Improving staff survey response rates in a future evaluation will improve confidence that bias is not skewing the results.

We reported between-ward differences, but differences at this level and on this small scale are as likely to be explained by random variation as by any other cause.

This chapter also reported on intracluster correlation for the three core outcome measures. All measures showed low variance at the ward level. However, there was a clear design effect apparent with QuIS at an observation session level, and this will need to be accounted for in the design of a future trial.

These results reported here will enable sample size calculation and inform outcome measure selection and use in a future trial, and are followed up further in Chapter 11 .

Feasibility of implementing and sustaining Creating Learning Environments for Compassionate Care121

The process evaluation aimed to identify and explain the extent to which the planned intervention was implemented into existing nursing practices and to draw conclusions about optimising the sustainability of future CLECC intervention. We found that, on all of the wards, many of the individual elements of CLECC were welcomed by staff and were possible to implement during the implementation period. We also found that sustaining this work beyond this time was difficult for some ward teams to achieve. Findings point to refinements needed to CLECC to improve the prospect of its impact and sustainability. The findings applied across all the ward contexts, in spite of some important differences between wards at the outset including specialty, staffing levels and ward leader experience.

Although CLECC had limited coherence for some staff, it was welcomed by teams and served as a broader stimulus to collective action. CLECC developed cultures in which reflection, learning, mutual support and innovation were legitimised within the work team, and in which expertise was seen to be distributed more widely between managers, RNs and HCAs. CLECC moved all the teams further along the continuum to becoming more expansive learning environments,29 but implementation was mediated for all by the context of working in an acute hospital environment. Staff highlighting what they valued about CLECC illuminated the stark realities of team working in such settings. The struggle to find the time to participate in CLECC reflects the pressure on staff to be constantly engaged in material patient care activities. Staff valued the cluster discussions because, otherwise, there was little opportunity to support each other’s well-being. They appreciated the study days because they could get to know each other as people. They valued CLECC because otherwise they were lone workers, sharing working time and space with other team members, but not actually working as a team at all. The intensification of nursing work owing to rising patient complexity, in parallel with the application of increasingly stringent financial efficiency quotas, is well documented. 124–126 These findings paint a rich picture of the consequences for staff experiences at work and explain associations between hospital work team climate and staff well-being reported elsewhere. 26,27

Staff at all levels were able to identify the benefits to patient care of ward staff engaging in CLECC activities, echoing other findings that the creation of unmanaged spaces for work team members to ‘take shelter’ provides the potential for valued learning and social support for difficult work with clients. 39,125 The findings confirm that intervening at the work team level can be successful, confirming an association conjectured from other research. 26–28,127 In spite of high workloads, CLECC empowered staff to reflect on local norms governing team practice, and on the relationships and resources that aligned with them, and to make some changes, confirming the original programme theory (set out earlier in Chapter 1 , Creating Learning Environments for Compassionate Care ) and indicating that collective agency can play a part in shaping relational capacity at the individual and work team levels. 125,128 However, we also found that implementation was uneven between teams, particularly over the longer term, reinforcing the value of paying attention to the sustainability of complex interventions beyond initial set-up. 129–131

Factors outside the direct influence of the ward teams mediated the impact and sustainability of the intervention, in particular the institutional norms that legitimated staff’s participation (or not) in CLECC activities and the interpretation by more senior figures (including PDNs, matrons and senior hospital managers) of what CLECC was and their role in supporting it. Although CLECC draws on the principles of democratic working, and we saw how HCAs in particular were enabled to take a lead in some CLECC activities, its longer-term success relied on cognitive participation from more senior members of the hierarchy. The authority that ward staff had to control how they spent their time, to innovate and to afford their own and colleagues’ well-being some priority varied between teams and was dependent on the signals, or ‘invitational qualities’,128 from these more senior figures as to what was legitimate or not. These findings that nurses do not control the conditions in which they work echo extensive research on the curtailment of professional autonomy in publicly funded health care, and the particular position that nursing as a profession occupies. 2,7,8,126,132–134 Matrons are the point at which organisational drivers, often business imperatives, must align with professional imperatives and the needs of frontline teams and their patients. The hybrid role and competing identities for nursing managers of this kind have been highlighted elsewhere126,135,136 and it is unsurprising that we identified different approaches to managing this key role. Although the current CLECC activities relating to senior manager participation appear to have aided coherence, findings suggest that additional activities targeted at improving their cognitive participation may be needed. Contextual features that appeared to be relevant to CLECC’s implementation journey included institutional norms regarding the legitimacy and nature of nursing work, staff learning and staff support; interpretation of key stakeholder roles (including nursing managers and PDNs); and ward-level characteristics, such as staffing levels in relation to patient workload, and stability of workforce and team leadership over time.

Our findings indicate that a higher and more sustained impact for interventions, such as CLECC, may be possible only through more substantial restructuring that reshapes the conditions in which people are able to act. 53,54 We support Parker’s39 assertion that caregiving organisations need to be designed to enable caregivers to access functional work teams within which they can interpret their experiences, and we have identified a number of concrete but modifiable barriers that merit attention in such design, including lack of time and institutional rules that undermine the value of staff-to-staff social support. They also include more clearly defining the role of nursing managers in signalling the legitimacy of staff providing each other with emotional support, supporting nursing teams to meet and learn together. Future versions of the CLECC intervention should include new activities to engage nursing managers in the implementation period, involve them in the learning activities and create opportunities for them to engage and reflect with frontline staff.

These findings reflect the fact that relational work in caregiving organisations depends not just on individual caregiver agency but also on whether or not this work is adequately supported by features of the wider system. Relational capacity may thus be regarded not only as a property of individual practitioners, but also as a modifiable and situated property of work teams. The limitation that institutional norms of legitimate nursing work placed on staff finding time to meet together raises the prospect of lack of relational capacity at a wider system level, and suggests that wider restructuring beyond middle manager roles may in fact be needed to effect substantial and sustained change.

These findings offer possibilities for actively restructuring work team conditions to enable the relational aspects of caring and working. Complex interventions, such as CLECC, can be theorised as ‘time-limited series of events, new activity settings and technologies’ in systems, the focus of the intervention being to generate new structures of interaction and new shared meanings. 53,54 Adopting this perspective means that the focus of change efforts is not just on the behaviour of individual staff, but also on restructuring of relationships, norms and resources in the wider system that may play their part in the success or otherwise of the intervention. The findings point towards a number of organisational conditions in which high-quality care is most likely and in which interventions of this kind are most likely to succeed, and these are set out in Chapter 12 . Further research across a wider range of organisational settings will enable these emerging theories to be refined to enhance their transferability. This further research will also enable us to more closely describe the links between context, implementation processes and outcomes associated with implementation of the CLECC intervention.

Informing future Creating Learning Environments for Compassionate Care evaluation

Our systematic review16 found that any of the interventions we investigated might be deemed worthy of further investigation based on their positive outcomes but that none could be recommended for routine implementation, given the lack of theoretical basis and description for many interventions, the pervasive positive bias that is associated with weak study designs, and the lack of evidence for impact on patient outcomes in most studies. Adherence to recognised and emerging standards for developing and evaluating complex interventions, such as the UK MRC framework,88 and fuller reporting of interventions and outcomes would address many of the issues noted in our review. We concluded that many researchers in this field have been unable or unwilling to use experimental designs within the context of mixed-methods approaches to evaluation.

The findings from this study indicate that the use of an experimental design to evaluate the effectiveness of compassionate care interventions within the context of a mixed-methods study is feasible, as is a focus on outcomes that are patient based. Ward teams were successfully randomised to intervention or control, and staff were amenable to the prospect of randomisation to either experimental condition. All wards recruited remained in the study throughout data collection and all clusters randomised to the intervention went on to receive it. Blinding of patients and visitors to ward allocation appeared successful, although strategies to blind researchers gathering data need further development in a future trial. Although we identified some differences between individual clusters and individual participants in the trial, none was sufficiently substantial to raise concerns of baseline imbalances between intervention and control conditions. 137,138

We found evidence of pathways through which the CLECC intervention had the potential to influence practice in other wards in both of the participating NHS trusts. This may be a sign of CLECC’s success in transforming and embedding in the wider system, but also indicated that future studies of effectiveness should not run intervention and control conditions in the same organisation over the same time period. This will be an important consideration in designing any future definitive experiments. 139,140

Older patients with a range of cognitive abilities are a traditionally hard-to-reach group in research, especially when they are unwell and in hospital. Even though they are more prone to negative experiences of hospital care, they are often excluded from research. 6,141,142 Although cognitive deficits may limit some people’s ability to share their experiences, our findings indicate that devising recruitment and data collection methods that maximise support and inclusion can be successful. It is estimated that at any one time, up to 25% of beds in acute hospitals are occupied by people with dementia, with the figure likely to be higher on specialist older people’s care wards. 143,144 Overall, 25% of patients observed in this study had evidence of cognitive impairment, suggesting that our sample was representative of the wider hospital population. Of all the patient questionnaires returned, 12% were completed by patients with cognitive impairment and so we were less successful in achieving representativeness here.

Our findings that structured non-participant observation appears to be the most promising method to describe the experiences of older people with cognitive impairment in the general hospital setting echo those of Goldberg et al. 141 Participating in an observation does not require any particular state of health, abilities or performance from the patient in question, whereas, for instance, answering questions in an interview about one’s care experiences requires as a minimum orientation to place, language skills and attention. 142 We did find that recruitment of people with cognitive impairment to the study took more time, as did completing questionnaires with them, and this has resource implications for future research with this patient group. In relation to patient involvement in general, this method overcomes the reluctance of patients to evaluate care while it is ongoing that was noted in other studies. 64

Our development of the QuIS tool for use in acute settings worked well and our earlier work confirmed an association between patient-reported experience and observed staff–patient interactions. 99 However, our earlier work did not include people with a cognitive impairment and validation of QuIS ratings with this patient group may be a necessary next step in the tool’s development. Although a clear design effect was apparent with QuIS at the observation session level that will need to be accounted for in the design of a future trial, benefits to the use of this tool were notable. 139 Acceptability of the tool to patients and staff was high, and reliability between observers was acceptable. We did not find any evidence that staff changed their behaviour as a result of being observed but this possibility cannot be eliminated. However, the findings across both assessment periods that a proportion of interactions observed were negative indicate that, even if staff planned to give consistently good care while being observed, they were not successful. Although we cannot eliminate the possibility of observer effects, the effect in a future trial will be present on all wards in all conditions and so differences between wards can still be attributed to the intervention. We know from other work that the quality of interactions with staff is very important to older people and shapes their experiences in hospital settings and so its successful measurement is a good indicator of compassionate care. 6 Our findings from this study confirm that observation-based measures are more inclusive of patient groups vulnerable to negative experiences in hospital. Overall, they support the selection of the quality of staff–patient interactions, measured by an observational tool (such as QuIS), as a candidate primary outcome in a future trial.

The response rate to nursing questionnaires (36%) was very low, particularly at follow-up stage on the intervention wards, perhaps owing to research fatigue. There was a perception that questionnaires were lengthy to complete and that staff were too busy. The low response rate means reduced certainty that the responses to the questionnaire represent the views of the nursing staff as a whole, although the findings of burnout and low empathy within the staff groups on every ward were important. The response rates are typical for surveys of this kind with this population. For instance, in a European study145 of nurse staffing levels, an estimated 39% of 2917 registered nurses working on NHS medical and surgical units in England completed a questionnaire similar to the one used in this study. Deploying a shorter questionnaire and negotiating time for their completion with managers may enhance response rates in a future study.

Small but non-significant differences between experimental groups at follow-up in quality of interaction and patient evaluation of emotional connection with staff are promising findings, particularly in the context of qualitative findings that indicate benefits to patient care perceived by staff. Data were gathered ≥ 4 months after the end of the implementation period, indicating that if there is an effect that can be attributed to CLECC, it is sustainable beyond the period in which CLECC was being actively facilitated by PDNs. We found no evidence that nursing staff empathy may be improved because of CLECC but these results in particular have to be viewed in the context of the low response rate to nursing surveys.

These findings are an important contribution to a field in which the use of experiments is relatively rare, and the results reported here will inform study design, sample size calculation and outcome measure selection and use in a future trial.

Our qualitative findings illuminate the importance of context in shaping implementation and outcomes and strongly indicate that future measurement of benefit should be part of a mixed-methods evaluation.

Informing measurement of Creating Learning Environments for Compassionate Care costs and benefits

Our findings have established the feasibility of estimating the cost of a CLECC-type intervention. Intervention costs were calculated as training costs (PDN time and staff time attending study day) and ongoing implementation costs (cost of staff engaging in CLECC activities on the ward). The extent to which training costs would be an additional cost to existing training would need to clarified in a future study, including any further training costs incurred from shifting team membership and the need for refresher sessions for existing staff. Through qualitative interviews with staff, we were able to establish that, aside from initial CLECC training costs, the implementation of concrete CLECC activities by ward teams was not perceived to require additional resource use. However, given the size of our sample and its qualitative nature, we consider that staff time spent on CLECC activities would still need to be recorded and costed in any larger trial. Data should also be gathered on the amount of training delivered through a register of attendance at classroom training, action-learning sessions and cluster sessions. The findings also indicate candidate activities associated with supporting CLECC that merit attention in a future evaluation.

A patient-level outcome measure, such as EQ-5D-5L, was shown not to be feasible, mainly because different patients with different ailments and severity were involved at baseline and at follow-up. Use of a patient-specific outcome requires either that the same patients in both intervention and control groups are measured over time or that adequate adjustments for the casemix of severity can be made. Neither seems likely. Neither QuIS nor any of the secondary outcomes were promising candidates for cost-effectiveness analysis, owing mainly to lack of single summary measures. Although QuIS is used as an audit tool in Scotland and has been applied to some services in the English NHS, it does not at present lend itself to use in cost-effectiveness analyses. Provision of data on QuIS and other ward-level scores may be of use in other evaluations or in developments towards a summary measure.

Consequently, we consider that any proposed economic evaluation of a CLECC-type intervention should comprise two elements. The first is an impact inventory, including comprehensive data on the costs and benefits of the interventions, distinguishing training and implementation. The second element proposed is a series of cost-effectiveness analyses linking cost to each of the primary and secondary outcomes. This approach has value for the evaluation of novel complex interventions with uncertain resource implications.

In summary, an impact inventory would provide a comprehensive listing of the resources, cost and benefits of CLECC with a focus on those to do with providing the interventions, but set within a wider context that includes effects on staff and on patients. Cost per change in each of the primary and secondary outcomes could also be estimated and compared with other later studies.

Strengths and limitations

This detailed and thorough mixed-methods study makes an important contribution to the evidence base on the design and evaluation of compassionate care interventions. Based on data gathered from a range of English NHS ward contexts, the findings are relevant to those seeking to influence and evaluate nursing practice in acute hospital settings in similar contexts. The qualitative findings indicate that staff welcomed CLECC and perceived positive benefits to their own well-being, to improved team working and to more compassionate patient care. As detailed earlier, we have theorised that the impact and sustainability of CLECC can be enhanced by attention to wider system restructuring, and refining these theories using data gathered in a wider range of organisational contexts will be an important next step. In particular, closer attention to defining the contexts in which CLECC is most likely to succeed will be a necessary focus in the next stage of this programme of research. We found variations in intervention fidelity attributable to a variety of contextual features. These factors merit refinement of elements of the intervention for future use, but also deserve continuing investigation in future studies because there may be other important features, identifiable only through a larger-scale study of context. Continuing qualitative investigation, in the form of interviews with frontline staff and service managers, and more detailed observation of CLECC implementation in practice, will enhance an understanding of the influence of context on implementation. It will also provide explanations for findings regarding CLECC’s efficacy.

The pilot RCT findings lend some support to the nurses’ views as regards the benefits to patient care, but larger-scale evaluation is needed before definitive claims are merited. Insufficient information was available at the outset of this study to enable power calculations that informed sample size and so there is no certainty that any apparent positive effects are not produced by chance alone, rather than the impact of the CLECC intervention. Potential issues of lack of researcher blinding to experimental allocation and contamination pathways between intervention and control wards also mean that caution should be applied in drawing conclusions of efficacy from the findings presented.

We have generated useful findings about the performance of a range of outcome measures in relation to compassionate care and have demonstrated the feasibility of using patient-based outcome measures in this field. Our findings indicate the strengths of observation-based evaluations of care delivered, but further research to assess the validity of these evaluations in relation to the experiences of people with cognitive impairment is warranted.

The research to date has focused on nursing teams in hospital settings and no claims are made about the generalisability of these findings to other types of team or other settings. We propose that, with some modifications to account for different contexts, the CLECC intervention may be of value to other teams in other settings but research of the kind reported here will be an important foundation to its use and evaluation in new contexts.

Our findings indicate that further evaluation is merited and point the way to how such future evaluation should be designed and carried out. Chapter 12 draws together the conclusions from the study and sets out implications for health care and recommendations for this future research.

Implications for health care

Our in-depth analysis of the process of implementing a complex intervention targeted at compassionate care raises questions about the extent to which interventions of this kind should in fact target and seek to influence and restructure relationships, norms and resources in the wider system. They suggest that health-care leaders who interpret their role as mobilising structural capacity to support the relational work of frontline staff may well improve the relational capacity of teams and their individual members. We have defined elements of this mobilisation in relation to the CLECC intervention, and our planned enhancements to the original intervention are set out in Box 2 . The enhancements clarify the role of leaders outside the ward team in supporting ward teams to implement CLECC by engaging leaders in the programme, involving them in the learning activities and creating opportunities for them to engage and reflect with frontline staff. We also propose tying in CLECC with wider staff education strategies in the organisation to help its wider integration, thus enabling the possibility that its goals become embedded and reflected across educational provision.

Proposed enhancements to the CLECC implementation period include appointing PDNs with solid experience as educators, and a more detailed specification for study days, to ensure consistency between sites in adhering to the CLECC principles. The appointment of team members as CLECC champions and inviting teams to develop sustainability plans will encourage staff to identify concrete contextually specific activities that will support CLECC going forward, along with an articulation of the roles, responsibilities and resources required to sustain CLECC.

Other proposed enhancements to CLECC focus on the period beyond the initial implementation period. Sustainability plans will encourage staff to agree on explicit expectations for discussing and developing an understanding of CLECC principles on an ongoing basis, and may embed this culture further. New team members will need induction to understand what CLECC is and what their role is in relation to CLECC. This particularly applies to ward leaders joining a ward where CLECC is in place, and careful attention and mentoring will be needed to support them as they develop their role.

Recommendations for research

The complexity of the intervention and the clear relationship between context and impact reflected in the findings, in addition to the continuing need to establish the efficacy of interventions of this kind, require a mixed-methods approach to future evaluation within the context of a programme of research to:

1. identify the organisational contexts in which optimal impact and sustainability of the CLECC intervention are most likely

2. further establish the feasibility and validity of the QuIS tool in relation to the experiences of patients in acute care settings

3. evaluate the effectiveness and cost-effectiveness of the CLECC intervention relative to usual care.

A programme of research would enable each of these objectives to be addressed. First, identifying the contexts in which optimal impact and sustainability are most likely is an important goal for the next stage of this research. Evaluating the processes of implementation in relation to contextual features in a wider range of acute care contexts will enable us to generate and test a typology of organisational types (at hospital and ward levels) that specifies their receptiveness to interventions of this kind. The nature of this type of evaluation is likely to be largely qualitative, drawing on observations of practice and interviews with key stakeholders over time to describe variations and identify relationships between implementation processes and context, and the resultant impact and sustainability. Qualitative and quantitative contextual data gathered from the feasibility study reported here (on, for instance, ward leadership, staff perceptions of care and staff well-being) could be added to equivalent data gathered from other organisations in a future study and combined to inform the development of the typology and the identification of contextual features relevant to implementation processes, impact and sustainability. The opportunity to investigate the wider dissemination and embedding of ideas and practices originating from the intervention but spreading beyond the target team to the wider system can also be exploited by qualitative exploration within clinical departments and within the wider health-care system.

Meeting this first objective is achievable through the study of context, implementation, impact and sustainability of the CLECC intervention in acute hospital nursing teams sampled to ensure heterogeneity. The findings regarding the influence of differences in contextual features at the institutional and team levels can be used to inform this sampling, namely institutional norms regarding the legitimacy and nature of nursing work, staff learning and staff support; interpretation of key stakeholder roles (including nursing managers and PDNs); and ward-level characteristics, such as staffing levels in relation to patient workload and stability of workforce and team leadership over time. Conducting this study over a 2-year period will enable us to capture the impact of variations over time and also to build a picture of longer-term sustainability.

Although our findings on the use of the QuIS tool are promising in relation to its inclusivity of hard-to-reach patient groups, further work is merited to inform its use as a primary outcome measure in future experiments in acute settings, in particular on its validity in relation to patient experience. Our work indicates that there is an association between QuIS ratings and patient experiences in acute hospital settings, but this relationship needs testing on a wider scale, in particular with patients who have a cognitive impairment. The second objective of the proposed programme of work can be met by a study that evaluates staff–patient interactions through QuIS rating, as used here, but also through patient ratings of the same interactions. These sets of ratings can then be compared. Further work will be needed to establish a means by which patients with cognitive impairment can rate interactions. If the proportion of negative interactions is the primary outcome measure in a future study, understanding which interactions are rated by observers (and, when possible, patients) as negative, and why, is an important next step, as is working with patient representatives to establish their views on the size of a meaningful reduction in negative interactions. Further study could also be used to develop more effective procedures to blind observers from experimental allocation in advance of an experimental study. In addition, the high intracluster correlation that we found at an observation session level merits the exploration of the cause of this variance and the feasibility of different approaches to data collection that might reduce its impact, for instance, shorter observation sessions. This further evaluation and testing of QuIS across these parameters would be a valuable foundation to its further use as an outcome measure in acute settings.

Drawing on our findings about the feasibility of experimental approaches to evaluating compassionate care interventions, our third objective for a programme of work focuses on the delivery of a definitive multicentre trial to establish CLECC’s effectiveness and cost-effectiveness. Our findings indicate that, given variations in the implementation journey over time, outcomes should be captured over a long period of time, at least 12 months, to ensure that sustainability is tested. Evaluation in a range of organisational contexts will improve the generalisability of findings and so a multicentre trial is merited. If QuIS is selected as the primary outcome measure, the high ICC at the observation session level indicates that it would be more efficient to conduct more observation sessions of shorter duration. The length of time taken to recruit a patient group with more complex needs militates against conducting sessions of < 1 hour. We can explore this further in the QuIS study proposed above but, for the calculations that follow, we assume that observation sessions are 1 hour in length. Detecting a 50% reduction in the rate (odds) of a negative interaction (i.e. a reduction from 10% of all interactions to 5%) at 90% power would require observation of 582 interactions per group, that is 1164 interactions in a parallel group trial. Allowance for clustering is achieved through use of a multiplicative factor [1 + (n cluster – 1) × ICC]. Patients in our feasibility study had an average of six interactions with staff per hour. If the cluster (observation session) is six (interactions), that is, the observation sessions are 1-hour long, the factor is [1 + (6 – 1) × 0.411] = 3.055. We would therefore need 1164 × 3.055 = 3556 observed interactions to detect a difference, that is, 593 observation periods of 1 hour each in total, rounded up to 300 per group.

If individual wards are observed for 20 hours each at each assessment period (the amount used in the feasibility study), 30 wards would need to be engaged in a trial (15 in each group). Our estimates of the costs and feasibility of implementing the CLECC intervention in each NHS organisation and the work required to set up and oversee the study in each organisation indicate that five wards each in six different NHS hospitals would enable this level of data collection to be achieved. These calculations assume that one patient is observed at a time but take no account of clustering within wards. In reality we were generally able to observe more than one patient at a time, which would provide additional data that would be more than sufficient to compensate for the relatively small increase in sample size required because of clustering at the ward level, given the low ICC at this level.

Our findings indicate that measuring patient views on care and staff self-rated empathy are useful and feasible as secondary outcome measures, although careful attention would need to be paid to maximise staff survey response rates. The study design would need to ensure that intervention and control conditions do not run in the same organisation at the same time. A waiting list control may be helpful here, with all study wards eventually receiving the intervention, but this would double the intervention costs and funding constraints are very likely to reduce the feasibility of this option.

As outlined above, cost-effectiveness can be evaluated through an impact inventory, including comprehensive data on the costs and benefits of the interventions, distinguishing training and implementation. We also propose a series of exploratory cost-effectiveness analyses linking cost to each of the primary and secondary outcomes.

To summarise, our findings indicate that further intervention development and evaluation work of the CLECC intervention through a programme of research is now merited.

Contributors to the study

Contributions of authors

Jackie Bridges (Professor of Older People’s Care) was chief investigator, leading the design and implementation of the study, the acquisition, analysis and interpretation of data, and the preparation and writing up of results.

Ruth M Pickering (Associate Professor of Medical Statistics) contributed to study design and oversaw the analysis of quantitative data and the presentation of quantitative results.

Hannah Barker (Senior Research Assistant, Ageing and Dementia) gathered qualitative and quantitative data, trained other research team members, prepared the quantitative data for analysis and contributed to analysis of qualitative data.

Rosemary Chable (Associate Director of Nursing) was based at a participating NHS site and oversaw intervention implementation and staff participation in the study.

Alison Fuller (Professor of Vocational Education and Work) contributed to the study design, the intervention implementation and the qualitative data analysis.

Lisa Gould (Research Fellow, Ageing and Dementia) managed the study day to day, gathered qualitative and quantitative data, trained other research team members, contributed to the analysis of qualitative data and helped to prepare the results for publication.

Paula Libberton (Senior Research Assistant, Ageing and Dementia) gathered qualitative interview and observation data as well as patient questionnaire data, contributed to the analysis of qualitative data and helped to prepare the results for publication.

Ines Mesa-Eguiagaray (Medical Statistician) conducted the analysis of quantitative data and the presentation of the quantitative results.

James Raftery (Professor of Health Technology Assessment) conducted the economic evaluation and helped to prepare the results for publication.

Avan Aihie Sayer (Professor of Geriatric Medicine) contributed to the study design and to leading the acquisition and analysis of the data.

Greta Westwood (Head of Nursing, Midwifery and Allied Health Professional Research) was principal investigator at a participating NHS site, overseeing study management, intervention implementation, recruitment and data collection.

Wendy Wigley (Principal Teaching Fellow, Research Associate, Complex Care) gathered qualitative interview and patient questionnaire data and contributed to the analysis of qualitative data.

Guiqing Yao (Associate Professor, Health Economics) contributed to study design, oversaw the economic evaluation and helped to prepare the results for publication.

Shihua Zhu (Senior Research Fellow, Health Economics) analysed the quantitative data for the economic evaluation.

Peter Griffiths (Professor of Health Services Research) was deputy chief investigator, contributed to the study design, and supported Jackie Bridges in leading the implementation of the study, the analysis and interpretation of data, and preparing the results for publication.

All co-authors contributed to drafting the final report and/or revising it critically for important intellectual content. All approved the final version of the submitted report.

The authors would like to express their sincere thanks to the staff, patients and visitors for their participation in this study, and to all the individuals listed below as contributors to the study.

Associated publications and presentations

Barker H, Griffiths P, Gould L, Mesa-Eguiagaray I, Pickering R, Bridges J. Quantity and quality of interaction between staff and older patients in UK hospital wards: a descriptive study. Int J Nurs Stud 2016;62:100–7.

Barker H, Griffiths P, Mesa-Eguiagaray I, Gould L, Bridges J. Quantity and Quality of Interaction Between Staff and Older Patients in UK Hospital Wards: A Descriptive Study. Paper presented at the Royal College of Nursing International Research Conference, Edinburgh, UK, 7 April 2016.

Blomberg K, Griffiths P, Wengstrom Y, May CR, Bridges J. Interventions for compassionate nursing care: a systematic review. Int J Nurs Stud 2016;62:137–55.

Bridges J, May CR, Griffiths P, Fuller A, Wigley W, Gould L, et al. Optimising impact and sustainability: a qualitative process evaluation of a complex intervention targeted at compassionate care. BMJ Qual Saf 2017;26:970–7. http://qualitysafety.bmj.com/content/early/2017/09/15/bmjqs-2017-006702.full

Bridges J. Creating Learning Environments for Compassionate Care (CLECC): Developing and Evaluating the Feasibility of a Complex Intervention. Paper presented at the International Learning Collaborative on Fundamentals of Care, Oxford, UK, 13 June 2016.

Bridges J. Culture of Care: Innovation in Practice. Paper presented at the Older Person’s Nurse Fellowship Conference, London, UK, 14 June 2016.

Bridges J. How Clinical Academic, Research Nurses and Ward Nurses Contributed to Make the CLECC Study Work. Paper presented at It’s okay to ask about . . . research nursing, Southampton, UK, 20 May 2016.

Bridges J. Intervention Studies in Relational Care in Acute Settings: Content, Context, and Consequences. Symposium presented at the Royal College of Nursing International Research Conference, Edinburgh, UK, 7 April 2016.

Bridges J, Fuller A. Creating Learning Environments for Compassionate Care (CLECC): a programme to promote compassionate care by health and social care teams. Int J Older People Nurs 2015;10:48–58.

Bridges J, Griffiths P, Barker H, Pickering RM, Mesa-Eguiagaray I, Libberton P, et al. Creating Learning Environments for Compassionate Care (CLECC): an Acute Care Feasibility Study. Poster presented at European Nursing Congress, Rotterdam, the Netherlands, 5–7 October 2016.

Bridges J, Libberton P, Barker H, Gould L, Wigley W, Griffiths P. Creating Learning Environments for Compassionate Care (CLECC): Developing and Evaluating the Feasibility of a Complex Intervention. Paper presented at the Royal College of Nursing International Research Conference, Edinburgh, UK, 7 April 2016.

Gould L, Griffiths P, Barker H, Mesa-Eguiagaray I, Bridges J. Creating Learning Environments for Compassionate Care (CLECC): Feasibility of Evaluating Impact on Patient Care. Paper presented at the Royal College of Nursing International Research Conference, Edinburgh, UK, 7 April 2016.

Libberton P, Bridges J. Older Adults in General Hospitals Want their Mental Health Needs Addressed as Part of Nursing Care: Why is this so Challenging? Paper presented at the International Network for Psychiatric Nursing Research (NPNR) Conference, Nottingham, UK, 15 September 2016.

Libberton P, Griffiths P, Gould L, Barker H, Mesa-Eguiagaray I, Bridges J. Relational Capacity of Nursing Teams: Exploring the Relationship Between Team Context, Relational Capacity and Caring Practices. Paper presented at the Royal College of Nursing International Research Conference, Edinburgh, UK, 7 April 2016.

McLean C, Griffiths P, Mesa-Eguiagaray I, Pickering RM, Bridges J. Reliability, feasibility, and validity of the quality of interactions schedule (QuIS) in acute hospital care: an observational study. BMC Health Serv Res 2017;17:380. https://doi.org/10.1186/s12913-12017-12312-12912

Mesa Eguiagaray I, Böhning D, McLean C, Griffiths P, Bridges J, Pickering RM. Inter-rater reliability of the QuIS as an assessment of the quality of staff-patient interactions. BMC Med Res Methodol 2016;16:171. https://doi.org/10.1186/s12874-016-0266-4

Mesa Eguiagaray I, Pickering R, McLean C, Griffiths P, Bridges J. An Application of Weighted Kappa in Nursing Research. Paper presented at the International Biometric Conference, Victoria, BC, Canada, 12 July 2016.

Mesa Eguiagaray I, Oliver E, Griffiths P, Bridges J, Gould L, Pickering R. A multilevel logistic model of poor quality staff-inpatient interactions accounting for variability across patients and observation periods. Poster presented at the International Society for Clinical Biostatistics Annual Conference, Vigo, Spain, 12 July 2017.

Oliver E, Griffiths P, Mesa Eguiagaray I, Pickering R, Gould L, Bridges J. Exploring nurse staffing skills mix and its effect on quality of interactions in UK hospital wards. Paper presented at the Royal College of Nursing International Nursing Research Conference, Oxford, UK, 7 April 2016.

Gould L, Griffiths P, Barker H, Libberton P, Mesa-Eguiagaray I, Pickering RM, et al. Compassionate care intervention for hospital nursing teams caring for older people: a pilot cluster randomised controlled trial. BMJ Open 2018;8:e018563.

Bridges J, May CR, Griffiths P, Fuller A, Wigley W, Gould L, et al. Optimising impact and sustainability: a qualitative process evaluation of a complex intervention targeted at compassionate care. BMJ Qual Safety 2017;26:970–7.

Data-sharing statement

Data can be requested from the corresponding author.

Patient data

This work uses data provided by patients and collected by the NHS as part of their care and support. Using patient data is vital to improve health and care for everyone. There is huge potential to make better use of information from people’s patient records, to understand more about disease, develop new treatments, monitor safety, and plan NHS services. Patient data should be kept safe and secure, to protect everyone’s privacy, and it’s important that there are safeguards to make sure that it is stored and used responsibly. Everyone should be able to find out about how patient data are used. #datasaveslives You can find out more about the background to this citation here: https://understandingpatientdata.org.uk/data-citation.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care.