A framework and toolkit of interventions to enhance reflective learning among health-care professionals: the PEARL mixed-methods study

Patient and staff experiences offer important insights into health-care quality

Patient and staff experiences offer important insights into the quality of health care, which complement organisational-level data on processes and outcomes. 7–9 Patient experience is an explicit outcome measure in the UK NHS10,11 and in the regulation of care quality. 12 All NHS trusts are required to collect patient experience data through surveys13,14 and additional insights are available through patient complaints. 15 In the USA, patients are surveyed through the Hospital Consumer Assessment of Healthcare Providers and Systems,16 whereas in Australia health organisations are required to involve consumers in accreditation processes. 17 Staff experience also provides insights into care quality: earlier action on staff concerns could have mitigated failings in care. 4 The NHS staff survey18 has been conducted annually since 2003. Patients and staff appear to share complementary insights into care quality: patient satisfaction is higher in hospitals in which nurses also reported better care quality. 19,20 Patient and staff perceptions appear to offer both overlapping and unique insights into safety in hospital. 21

Patient and staff experience is strongly influenced by staff attitudes and behaviours

The NHS National Inpatient Survey13 asks respondents to rate their overall experience on a scale of 0 (‘very poor’) to 10 (‘very good’). In 2017, 50% of respondents rated their experience as ≥ 9. 13 This indicates that there are substantial opportunities within the health system for ‘learning from excellence’. 22 However, ‘good’ ratings by patients (as opposed to ‘very good’) may also disguise important opportunities for improvement. 23 A survey by Healthwatch England (London, UK) reported that half of those experiencing substandard care do not report it. 24 Patients who respond to surveys with perfect ratings but with negative free text frequently describe lapses in staff behaviours and attitudes,25 such as communication, empathy, courtesy, consideration, compassion and patient focus. A study of patient-reported safety incidents26 found that 22% were related to communication failures alone, whereas a systematic review of patient complaints27 judged that one-third of complaints were related to staff–patient relationships. This may well be an underestimate given that staff behaviours are mentioned in up to two-thirds of all complaints (Queen Elizabeth Hospital Birmingham local data on file). Attitudes and behaviours of other members of staff are also important determinants of staff (dis)satisfaction, as demonstrated by the 487,727 respondents to the NHS staff survey18 in 2018. Taken together, patient and staff experience, both positive and negative, provide an important opportunity for improvement through behaviour modification.

Patient and staff experience data are not used optimally to change behaviours

Despite this investment in data acquisition, health systems have difficulty using patient and staff experience data to improve care, particularly the non-technical skills related to attitudes, behaviours and culture. 28–30 Reflective learning underpins approaches to improving non-technical skills, but the processes by which experiences are translated into reflection, and reflection into behavioural change, are not well understood. Using patient experience to improve care is not a trivial task. 31 Lapses in care are usually multifactorial, the product of interactions between the individual and the ‘system’; however, from the perspective of the patient, quality is largely about trust-based relationships with specific individuals,32 not ‘systems’. Trust boards and quality governance committees must contend with the competing priorities of hundreds of quality indicators each month and may prioritise avoiding falling below a quality threshold rather than achieving higher values of a performance standard once met. A review in 201233 that examined how hospitals had used research from the UK National Inpatient Survey concluded that ‘simply providing hospitals with patient feedback does not automatically have a positive effect on quality standards’. Even trusts with a tradition of collecting and using patient survey data may struggle to convert these data into tangible improvements. 34 In a study of 50 clinical and managerial staff in three English hospitals, Sheard et al. 35 found that ward staff had difficulty using patient feedback, and the collection of patient experience feedback was a ‘self-perpetuating industry’ conducted ‘at the expense of pan-organizational learning or improvements’. 35 They concluded that ‘macro and micro prohibiting factors come together in a perfect storm which [prevents] improvements being made’. 35 In a systematic review, Gleeson et al. 36 reported that ‘patient experience data were most commonly . . . used to identify small areas of incremental change to services that do not require a change to clinician behaviour’. 36 Institutional commitment to using patient feedback may not be reflected at the front line, where single individuals can adversely influence other members of staff. 37 Conversely, front-line staff can struggle to get their voices heard at senior levels; one of the recommendations of the Mid Staffordshire NHS Foundation Trust inquiry4 was to establish ‘Freedom to Speak Up Guardians’ in all NHS trusts to ensure that staff concerns are heard and acted on. 38 These findings indicate that changing behaviours requires a change in underlying attitudes at individual, group and organisational levels. How is this best achieved?

Reflection as behaviour

Behavioural modification is a universal preoccupation, and behavioural sciences now inform government policy. 39 A large number of techniques exist and a proposed behaviour change taxonomy has so far identified 93 different interventions. 40,41 However, evidence supporting the primacy of one technique over another is not strong. 42,43 Many interventions involve personal insight development through reflection, but few behaviour change theories express this explicitly. One which does is the Capability, Opportunity, Motivation – Behaviour (COM-B) framework,44 which assimilates 19 behaviour change theories into a single framework in which the behaviour of interest has three determinants, each with two subtypes: capability (physical and psychological), opportunity (physical and social) and motivation (reflective and automatic). The automatic subtype for motivation relies on heuristics, is engaged in conditions of complexity and stress, and maps to Daniel Kahneman’s ‘System 1’ thinking. 45 The reflective component of motivation is slower, more effortful and analytical (Kahneman’s ‘System 2’ thinking). These two subtypes of motivation map to the ‘peripheral’ and ‘central’ routes described in the elaboration likelihood model of persuasion. 46,47 Factors influencing motivation (particularly the automatic subtype) are summarised by the acronym MINDSPACE (messenger, incentives, norms, defaults, salience, priming, affect, commitment and ego). 48 As the behaviour of interest here is reflection itself, we need to consider both the automatic factors that influence the desire to reflect and the more effortful elements of ‘reflecting on the need for reflection’.

We next consider how theories of reflection as a behaviour link to theories of reflective learning as a tool for personal insight development.

How does reflection stimulate learning?

Kolb and Fry49 presented reflection as a four-stage model: experience, observation, analysis and recalibration. Schön50 described reflection ‘in-action’ and ‘on-action’. Others emphasise the importance of an emotional component to reflective learning,51 including the ‘disorientating dilemma’:52 the realisation that there is a gap between desired and actual behaviours. Sandars53 describes reflection as a metacognitive process that creates a greater understanding of self and situation to inform future actions: looking back to look forward. This involves a transition from tacit to explicit knowledge54 in which socially acquired norms of behaviour55 are modified either through individual reflection or, more powerfully, through group activities. 56 The process shares similarities with Broadwell’s57 four stages of competence: unconscious incompetence (unaware of problem), conscious incompetence (data received, now being processed), conscious competence (using data to improve or disseminate excellence) and unconscious competence (effortless excellence). Effective reflection appears to involve making this transition while recalibrating and reinterpreting experience in-action and on-action.

Reflective learning could be deployed more effectively to improve care quality

First described by Dewey in 1933,58 reflective learning is now a mandatory tool in the education, appraisal and revalidation of health-care professionals. In the UK, the General Medical Council (GMC) and others define reflective practice as:

The process whereby an individual thinks analytically about anything relating to their professional practice with the intention of gaining insight and using the lessons learned to maintain good practice or make improvements where possible.

General Medical Council. 59

Reproduced with permission from the General Medical Council. © 2020 General Medical Council. All rights reserved

They state that:

. . .  reflecting on . . . experiences is vital to personal wellbeing and development, and to improving the quality of patient care.

General Medical Council. 59

Reproduced with permission from the General Medical Council. © 2020 General Medical Council. All rights reserved

Reflection is incorporated in all UK health-care postgraduate training programmes and is evidenced in professional portfolios, which, for doctors, must include multisource feedback from patients and colleagues. The Nursing & Midwifery Council require five written reflections over 3 years by nurses for revalidation. 60

However, despite the widespread promotion of reflection as a tool for self-improvement, evidence that it actually improves performance is weak,61–63 as is the evidence that feedback from patient experience surveys promotes effective reflection. 64 We are biased towards favourable events and judgements65,66 and tend to reject adverse patient feedback,67 thereby missing the opportunity to learn from either through critical analysis. Poor performers additionally lack insight into their abilities. 68 It takes emotional strength and resilience to take a critical view of one’s own skills, particularly those relating to attitudes and behaviours given that these touch us personally; undesirable information is processed as a threat with physiological correlates that can actually impede learning. 69 In the UK, doctors are concerned that honest reflections documented in personal portfolios might incriminate them in a court of law. 66 Guidance on reflective practice for doctors70 proposes that solely negative events and errors should be used for reflection, rather than permitting a mix of both negative and positive events. The GMC also mandates reflecting on ‘significant events’, which are defined as ‘any unintended or unexpected event, which could or did lead to harm of one or more patients’59 (Reproduced with permission from the General Medical Council. 59 © 2020 General Medical Council. All rights reserved), in guidance on supporting information for appraisal and revalidation. 71 The emphasis on negative events could be viewed as diminishing the value of learning from positive experiences and role models. 22 The current emphasis on reflection for appraisal has had the effect of diminishing its potential for translation into lifelong reflective practice. 72 If reflective learning is to be effective, it must involve more than just completing a form and ticking boxes.

Could communication skills training substitute for training specifically in reflection? A review of 243 studies73 of teaching communication skills to medical undergraduates identified only 16 interventional studies, and of these only two reported behavioural outcomes, making it difficult to determine whether or not ‘communication skills’ are sufficient or even if such courses are effective at all. At postgraduate level, the effects of communication skills training appear to be weak or evanescent. 74–79 In one randomised controlled trial (RCT) of staff training in end-of-life care,80 communication skills were associated with worse depression among patients in the intervention group.

By contrast, interventions that focus on engaging staff in workplace-based activities, which improve teamworking,81 insight, patient-centred care and empathy, may be more effective and more durable. 82–93 The Cleveland Clinic offers short videos on empathy,94 and similar internet-based resources demonstrate how emotion may be engaged to stimulate reflection and promote mutual understanding. 95,96 For reflective learning to improve patient and staff experience, it probably needs to influence hearts as well as minds. Techniques for reflection need to be based on insights from behavioural sciences and evaluated using relevant process and outcome measures. In summary, the work of reflection must become a social enterprise rooted in a community of learning. This is the ethos of the Patient Experience And Reflective Learning (PEARL) project.

Location

The project was located in the acute medical units (AMUs) and intensive care units (ICUs) of the four participating hospitals.

Acute medical units are generally the first point of emergency admission to hospital for patients, usually from the emergency department (ED) or following direct referral by general practitioners in the community. Of those admitted, around 40% of patients may be discharged home from the AMU within 48 hours and 60% transferred to general or specialist wards for continuing care. Specialist acute physicians will generally work 12-hour daytime shifts in the AMU, providing twice-daily ward rounds; weekend cover may be provided by acute physicians or by a general physician on duty. Nurse-to-patient ratios may be in the region of 1 : 6. The clinical team has to cope with large numbers of patients, a rapid turnover and making quick and accurate diagnostic and therapeutic decisions in conditions of uncertainty.

Intensive care units care for the most severely ill patients in the hospital using highly trained staff and complex equipment. Nurse-to-patient ratios are usually 1 : 1 or 1 : 2, providing care to one of the most vulnerable populations in health care, requiring burdensome technical support for multiple failing organ systems and experiencing a mortality rate of around 30%. 98 Intensive care consultants usually conduct two ward rounds per day, 7 days per week, and provide continuous care for periods of several days, or whole weeks, as well as undertaking regular night duties to maximise continuity of care. Providing reliable care in this complex environment requires both high-level technical and non-technical skills. Non-technical skills include effective teamworking, situational awareness and leadership, and the ability to integrate information across locations and over time to co-ordinate communication and actions, and to provide care with compassion.

Participants

Each participating unit established a local project team consisting of clinical, managerial and administrative staff as well as patient and relative representatives with experience of the AMU or ICU. Teams held bimonthly local project team meetings that were chaired by a non-executive director (executive director at one trust).

Staff on the units participated in interviews and observations (carried out as part of WS3) as well as the co-design work to develop interventions to enhance reflection (WS4).

Participants in the experiential data collection workstream (WS2) were patients and relatives who had spent time in the AMU or ICU. Their experiences were collected through the PEARL project patient and relative experience survey. Staff experience data were collected from staff whose primary working area was the AMU or ICU at the participating trusts, using the PEARL project staff experience survey.

Project set-up

Local project leads for each participating unit were appointed. The leads then established a local project team, which consisted of medical, nursing, allied health professional (AHP), administrative and managerial colleagues from their units. The teams were also encouraged to invite patient and relative representatives. Once established, the team agreed local arrangements for project delivery following confirmation of capacity and capability for their site. Existing opportunities for feedback and reflection were identified. With direction from the central project team, local teams held meetings once every 2 months to review project outputs, encourage team reflection (e.g. through existing meetings, team briefs and formal reports) and consider methods for incorporating feedback in routine practice.

In addition, we held four plenary workshops (three of which were co-design activities), nine facilitated local co-design meetings (three in each centre) and one further local co-design meeting without central facilitation. Our ethnographers observed all of these activities and provided feedback on each.

General aspects of survey development

The aim of the PEARL project surveys was to acquire experiential data from patients, relatives and staff that could be fed back to the local teams as evidence of the need for reflection, and as a motivating stimulus. We were sensitive to the need to avoid duplicating existing survey work and to minimise the burden on recipients of the surveys and the administrative load for local teams. We describe first the generic considerations for both surveys, followed by the specific development of each survey.

A survey subgroup was established, which consisted of project management committee members (including patient representatives) and the local project leads. A scoping review was undertaken of English-language surveys that had been used in the UK and focused on the experience of care of patients or relatives, or the experience of providing care by health-care staff.

Data acquisition varied between these surveys: some employed questions and others used statements with strength of agreement ratings. The working group, therefore, prioritised a standard approach for the patient and staff surveys, expressing a preference for the use of statements with strength of agreement ratings using a 5-point Likert scale. A limited number of direct questions were also included in the patient and relative survey.

Survey statements were selected or excluded through iterative discussion. The focus of attention was to identify those aspects of health-care behaviours that were within the competence and capability of individuals to address. We aimed as far as possible to preserve the original wording of the source surveys to retain questionnaire validity; where questions had been used, these were edited to statements. For the staff survey, we also conducted a modified Delphi process to prioritise the final set of questions (see Staff experience survey).

For the format of the surveys, we considered at an early stage whether these should be paper based or web based. A study from the Netherlands comparing both formats in colorectal registry cancer patients found a preference for paper surveys among respondents aged > 70 years, which was offset by a trend for web-based responses among those aged < 70 years. 99 A study from Denmark100 found lower response rates, higher completion rates and much lower administrative costs for an e-mailed web-based survey than the postal paper-based format; however, they did not factor in the costs of developing or hosting the web-based survey. Patients may not possess, and medical record departments do not universally record, e-mail addresses or mobile telephone numbers (for social media applications) for sending out invitations to complete the survey, and fewer young people now use e-mail as a primary mode of communication. The local project teams and the patient representatives were of the view that paper-based surveys were preferable at this stage, but recognised that these would in the future be supplanted by web-based approaches.

Participant information sheets accompanied each survey, explaining their purpose and that completion was optional, not mandatory. Questionnaires were anonymous; no person-identifiable information was collected.

The research ethics committee approved implied consent for those respondents who returned the completed questionnaires using the free-post return envelopes.

Analysis was undertaken by the central project team. Each completed survey was reviewed, focusing particularly on the free text to ensure that if serious concerns had been raised the relevant local project lead could be informed for subsequent investigation.

Responses were allocated subject numbers and reported to the Clinical Research Network using the National Institute for Health Research’s (NIHR’s) Central Portfolio Management System for uploading research accruals. Dissemination logs outlining the numbers of surveys disseminated were kept locally, with figures reported to the central team to allow calculation of response rates for each unit.

Surveys were scanned using Formic software (Formic Solutions, Uxbridge, UK) and the data uploaded to a spreadsheet for analysis. Data were presented to the units as standardised reports that showed the proportion of respondents selecting each level of strength of agreement with each statement. The data management team checked a random sample of 10% of the responses to ensure that there were no problems with the scanning. When issues were identified the data management team would undertake further action: either check all data or request a new scan of surveys. Free-text responses were transcribed manually and listed verbatim in the reports. They also underwent thematic analysis using NVivo software (QSR International, Warrington, UK).

Patient and relative experience survey

Questions from the following existing UK-validated questionnaires were reviewed for inclusion by the survey subgroup:

• NHS National Inpatient Survey13

• Jenkinson et al. ’s Picker Patient Experience Questionnaire101

• Sullivan et al. ’s Reduced NHS National Inpatient Survey AMU questionnaire102

• Family Satisfaction in ICU survey103

• NHS Friends and Family Test. 14

From these surveys, we extracted those questions that were most relevant to attitudes and behaviours and were applicable to any acute care area in the hospital. We excluded questions that related to logistics (e.g. waiting list delays), related to technical aspects of care (e.g. complications of treatment) or were location specific (e.g. outpatient clinic attendance). This allowed us to use the same survey for the AMUs and the ICUs. We focused on questions relating to non-technical aspects of care (e.g. care delivery and communication), which could be influenced by individual members of staff.

The survey that best met our criteria was the Family Satisfaction in ICU survey,103 which provided the majority of the questions in the final set and obviated the need to undertake a prioritisation exercise for question selection. An additional question was from the Friends and Family Test. 14 Two free-text options were included to ask which aspects of care could be improved and what went well. Limited demographic information was requested. Respondents were given the option of providing their contact details if they wanted a response from the local project lead.

We chose not to offer the survey in languages other than English, partly on grounds of cost–benefit (the process of translation, reverse translation and validation for multiple languages would have added considerable complexity) but mainly because the aim of the survey was to obtain sufficient information to provide material for staff reflection, not to provide a comprehensive assessment of the clinical service, which was the responsibility of the trust.

The final survey that was developed was reviewed and piloted by the whole collaboration prior to implementation.

Staff experience survey

Development of the survey: Delphi method for prioritisation and selection of survey items

The aim of the PEARL project staff survey was to gain insights into individual, contextual and organisational influences on staff behaviour that might affect patient experience. This included attitudes towards reflection and to the use of patient experience data.

As with the patient and relative survey, it was felt that there was no single existing survey that met our requirements for the project. Therefore, we assembled material from four existing UK-validated surveys that focused on staff experience and patient safety: the NHS Staff Survey18 from 2016, Hospital Survey on Patient Safety Culture,104 Teamwork and Safety Climate Survey105 and the GMC Trainee Survey. 106 To these we added questions derived from the PEARL project ethnography topic guides. These were based on the COM-B44 framework and the theoretical domains framework. 107 Where the original surveys employed questions, we converted these to statements using a 5-point strength of agreement Likert scale, retaining as much of the original phrasing as possible (as with the patient and relative experience survey).

We used a prioritisation method (Delphi) for statement selection because of the diversity and range of options available. The statements were circulated by e-mail to the collaboration and the respondents were asked to rate each statement using a four-point scale: ‘definitely include’, ‘probably include’, ‘probably exclude’ and ‘definitely exclude’. Each e-mail ‘round’ carried forward the outputs from the preceding round.

Responses were categorised as follows:

• Statements that achieved ≥ 75% agreement to definitely include did not go into the next round, but were extracted and reserved for the final survey.

• Statements that achieved ≥ 50% agreement to either definitely or probably exclude were removed.

• Statements that achieved < 100% but ≥ 50% agreement to either definitely or probably include were retained for the next round.

Following three rounds of Delphi prioritisation, a final set of survey questions emerged to form the final PEARL project staff survey. The Delphi process and outputs are presented in Figure 1. Basic demographic questions were included along with one free-text question asking ‘Do you have any additional comments about working in this organisation?’.

FIGURE 1.

The PEARL project staff survey Delphi methodology flow chart. Italic text indicates an item developed from PEARL ethnography topic guides.

Eligibility

The PEARL project staff survey was offered to all clinical, administrative and managerial staff whose primary working area was the AMU or ICU at the participating hospitals. This included doctors (consultants to foundation grade), nurses (bands 5 to 8), administrative and clerical staff (including operational managers), health-care assistants, maintenance and ancillary staff, AHPs and pharmacists. Dissemination logs were maintained locally, with figures reported to the central team to allow calculation of response rates for each unit.

Sampling and consent

The staff survey was distributed twice for a 2-month period during the PEARL project (January to February 2018 and January to February 2019). As with the patient survey, implied consent applied and the surveys were anonymous.

Dissemination

Local units were advised to use the method of dissemination most appropriate for them. Options included via staff pigeon holes, attachment to monthly payslips, at weekly/monthly staff meetings, in staff/locker rooms or posting out. Questionnaires were disseminated with a pre-paid return envelope for completed responses to be sent back to the central project team for processing and analysis.

Processing and analysis

As with the patient and relative survey, once the completed staff surveys were received by the central team a random sample of 10% was reviewed by the data management team and any issues identified were resolved before further processing. All free text was transcribed and classified using NVivo; any serious concerns were flagged to the appropriate local project leads for investigation.

The responses were digitised, analysed and reported to each local project team as standardised reports that showed the proportion of respondents selecting each level of strength of agreement with each statement. In addition to the stacked bar charts and pie charts, an overall domain score was calculated for staff experience. The domain score was calculated using the responses to statements relating to perceptions of care quality and the working environment and excluded those statements relating to personal reflection: a strong positive statement was weighted with a score of 100, a positive statement with a score of 80, a neutral statement with a score of 50, a negative statement with a score of 20 and a strong negative statement with a score of 0. Data from the NHS staff Friends and Family Test for the corresponding time period were obtained for the corresponding hospital sites or trusts. National comparative data for AMUs and ICUs were estimates because different naming conventions for these units between trusts meant that some may have been omitted. For the second survey, comparisons were made with the results of the first survey.

Survey validity and reliability

Before disseminating the first survey, we decided to exclude any question with any of the following response characteristics:

• a non-response rate of > 10%

• a lack of discrimination (defined as > 70% of all respondents ‘strongly agreeing’ or ‘strongly disagreeing’ with the statement)

• redundant (defined as questions with an item-scale Cronbach’s α of < 0.7).

Based on the initial analyses of the first survey, no question met any of the above criteria; therefore, all questions were retained for the second round. Following the second round the analyses were repeated and again no question met any of the above criteria for removal.

Phase 1

Ethnographic research that was conducted in phase 1 focused on gaining insight into attitudes and practice relating to feedback and reflection on patient experience, through observing day-to-day practice and interviewing a purposive sample of staff across the sites. Topic guides were developed to ensure consistency between ethnographers for phases 1 and 2.

Our key aims for phase 1 were to:

• systematically describe the approaches used to feedback experiential and quality and performance metrics in ICUs and AMUs

• obtain staff views about credibility, utility and impact of different approaches to feed back patient experience data

• explore staff engagement with reflective learning based on patient experience data and identify how individual and group reflection can be promoted in practice

• identify the factors that affect the extent to which reflection is translated into action at the front line of practice

• inform the co-design of the RLF and toolkit.

We conducted observations and interviews in eight acute care units (three AMU-type and five ICUs) in the three NHS trusts participating in the PEARL project, between May and December 2017. Characteristics of the sites were as follows:

• Site P1 had an ICU and a high-dependency unit (HDU), referred to jointly as ‘critical care’, as well as a separate large AMU.

• Site P2 had an ICU and a separate clinical decisions unit (CDU) (equivalent to an AMU) with an acute medical clinic attached to it.

• Site P3 was split over two sites: one site had two ICU wards with attached HDUs and an AMU (referred to as the ‘assessment suite’) and the other site had one ICU with an attached HDU.

We conducted > 140 hours of observations in ICUs and AMUs, including 81 informal documented conversations with a wide range of staff. We collected relevant documents, such as newsletters and photos of patient experience displays, in the units. Observations were recorded as written field notes and audio-recorded debriefs. We undertook 45 formal, semistructured interviews with a purposive sample of staff members, including administrative, nursing and medical staff, sampled to include diversity in their role and level of involvement in trust activity around patient experience data. Interviews and observational visits were conducted in two rounds (May to June 2017, prior to the introduction of the PEARL project patient experience questionnaire, and November to December 2017, after data collection had started). We conducted 36 interviews in round 1, which were focused on the collection and use of patient experience data and engagement in reflection, and nine interviews in round 2, which focused in more detail on reflection in practice. Interviews lasted between 10 and 90 minutes (mean 38 minutes). Informed consent was obtained for interviews. Breakdown by site is shown in Table 1.

Phase 2

Phase 2 involved observations and interviews. The main aim of phase 2 was to gain insight into staff experiences of using tools to select and design interventions to promote reflection in practice in order to inform the development and implementation of the reflective learning toolkit.

We conducted observations of the local project team meetings in which sites piloted components of the draft toolkit that had been designed to support the selection of reflective practice interventions, and planned for implementation (April to May 2019). We conducted interviews with members of staff at each site who had attended these meetings in order to gain feedback on the approach taken in the meetings and how well the draft toolkit components had worked as well as to capture their experiences of developing and trialling their selected interventions (June 2019). We observed four meetings in total (site P1, two meetings; sites P2 and P3, one meeting each). The first meeting at site P1 lasted approximately 1 hour and the second meeting was approximately 2.5 hours long; the meetings at sites P2 and P3 lasted around 2 hours. Observations were recorded as written field notes and audio-recorded debriefs. We conducted 15 interviews (June to July 2019) with a range of staff (five per site) about their experience of choosing interventions during the meeting and the progress in implementing any chosen interventions on the ICU or AMU wards. Informed consent was obtained for all interviews. Interviews lasted between 26 and 85 minutes (mean 49 minutes). Breakdown by site is outlined in Table 2.

Analysis of phases 1 and 2

Interviews and observation debriefs were recorded and professionally transcribed verbatim; anonymisation took place during transcription. We conducted regular team debriefs during the data collection period to reflect on emerging findings and guide ongoing data collection, and we discussed findings with the wider team. Analysis of data from phases 1 and 2 was conducted separately.

For phase 2 we conducted an analysis to characterise the features of meaningful patient experience feedback and to explore the barriers to and facilitators of effective reflection on patient experience in practice. A thematic analysis approach was taken to analyse the transcripts of the interviews and observational data. 108 A subset of interviews was read in close detail and open-coded to create a code frame and initial thematic categories; these were discussed with the wider study team. The coding frame was then applied to the remaining interviews and observational data transcripts. As new themes arose, there was flexibility to extend the coding frame to capture these. We used case studies, narrative reports and visual data displays as ways of facilitating comparison across sites and synthesising findings. Findings from the analysis of phase 1 data were used throughout the co-design process to inform the development of reflective practice interventions and the toolkit as a whole. Phase 2 data were subject to a separate thematic analysis closely linked to the aims of identifying lessons for development of the toolkit. NVivo 11 software was used to support the management, coding and querying of the data.

Introduction

Co-design refers to a set of practices that can be used to allow diverse stakeholders to come together to address a shared issue. The ‘co’ prefix refers to the participatory nature of the activity and ‘design’ refers to both the technical skills and practices of design and the unique approach to problem-solving approach used by designers.

Co-design has previously been used in health care. The most widely published approach is experience-based co-design, which accesses the experiences of patients and relatives and staff from videoed interviews109 or directly in interactive group discussions. The PEARL project co-design approach was supported by the Translating Knowledge into Action (TK2A) theme110 of the NIHR Collaborations for Leadership in Applied Health Research and Care Yorkshire and Humber. The TK2A theme is a multidisciplinary team of health service researchers who have developed over the last 10 years111 methods drawn from design practice, with a focus on the use of creative practice to support the successful sharing and synthesising of knowledge. 112

The reflective learning framework

The RLF is the programme theory or logic model for the PEARL project. It is the health services equivalent of a systems specification in computing or a drug’s mechanism of action. It describes how reflective learning might ‘work’. We developed the RLF iteratively throughout the project, making modifications to the theory as we gained insights from the ethnographic work and from our observations of the co-design process for the toolkit. We based our approach on that recommended by Davidoff et al. 113

We defined reflective learning as an experiential process of personal insight development in which one’s own and others’ experiences are used to produce a change in attitudes and behaviours. We characterised the features of effective reflective learning based on Dewey’s58 three phases of reflection (retrospection, self-evaluation and reorientation), to which we subsequently added Gibbs’ six-step cycle114 and Schön’s50 categorisation of reflection occurring in-action or on-action.

In mapping out the factors affecting engagement in effective reflective learning, we drew on the COM-B framework. 44 With effective reflective learning as the behaviour of interest, the COM-B framework identifies three domains of determinants, each with two subtypes: capability (physical and psychological), opportunity (physical and social) and motivation (reflective and automatic). As the COM-B framework was both parsimonious and evidence based, we decided to incorporate it in our preliminary version of the RLF. An important element of our conceptualisation is that the ‘B’ in COM-B signifies engagement in effective reflective learning: completing the cycle of insight development, attitude change and resulting change in interpersonal behaviour and practice.

This framework incorporates the dynamic in which an input (e.g. experiential information from the surveys) would, depending on the ‘COM’, result in the behaviour of effective reflective learning (primary output). This would generate measurable changes in attitudes and practices, and enhanced care delivery and teamworking (proximal and distal outcomes).

Iterative versions of the RLF were developed by the core project team and presented for critique at the co-design workshops. Participants at the co-design workshops reviewed the RLF for both understandability and scientific credibility and provided feedback that was used in the next iteration. The COM-B framework was used to classify interventions to drive engagement in effective reflective learning as these emerged from the co-design process. We show the evolution of the RLF in Chapter 3.

Toolkit co-design

Facilitation

Facilitation of the nine co-design local meetings and the three workshops was provided by the Art and Design Research Centre, Sheffield Hallam University (Sheffield, UK). Skilled and sensitive facilitation was essential to promote the flow of conversation, manage emotions, permit periods of silence, allow humour to release tension and ensure that everyone had an opportunity to express their ideas and opinions, regardless of status, seniority or specialty. The facilitation team comprised three members of the TK2A team. The theme lead and clinical researcher are a nurse and physiotherapist, respectively, both having worked on creative co-design projects across a broad range of topic areas for the last 10 years. The third member of the TK2A team is a designer who has worked in health-care contexts and co-design since completing their doctorate. The team were able to function as experts in the co-design process but could claim independence from the team, unit and hospital. A scribe recorded key points, conclusions and actions, and how these were derived. Each meeting was observed by two ethnographers for subsequent distillation and analysis.

The tools and techniques used for co-design were based in part on the NIHR’s Better Services by Design website. 115 The process was underpinned by the ‘double diamond’ approach (‘discover, define, develop, deliver’) in which the first workshop focuses on the key issue, the second expands the discussion to consider the wider context, the third refocuses on developing specific interventions and the final workshop allows participants to report their piloting of the interventions.

Each session started with a brief summary of the purpose of the project and a recap of the preceding activities. The facilitator then invited everyone to introduce themselves, followed by an ‘ice-breaker’ exercise to promote the conditions required for co-design to take place, after which the group undertook the activities described below. Each small working group (between two and four for each meeting, dependent on numbers) was supported by the facilitators and team. Plenary sessions allowed sharing of discussions and outputs. A summary was constructed at the end from all of the participants.

Co-design 1: reflectable moments

The aim of this first workshop was to allow the group to explore the meaning of reflection and how it ‘works’. Participants were invited to describe ‘reflectable moments’: events or activities that stimulated reflection in themselves or others, what feelings were aroused, how these feelings were used to gain insight into oneself or others and how those insights could stimulate generalisable learning. This process aroused emotions that the facilitator channelled into creative energy.

Participants modelled reflectable moments using cut-out figures (Figure 2) to recreate the characters and situations that they had encountered. This allowed the participants to tell their story with both realism and a degree of detachment, recognising the benefits of making the intangible tangible through creative practices. 116

FIGURE 2.

Co-design meeting 1 resources. (a) Cut-out figures; and (b) reflection wheel.

Analysis of the thought processes used by the participants allowed us to combine the three-stage Dewey model58 and six-step Gibbs cycle114 into a single ‘reflection wheel’. It was also clear that individuals varied in their approaches to reflection: in-action, on-action, solitary and group.

Co-design 2: reflection personae

The second workshop gathered participants’ opinions and experiences of reflective behaviours as demonstrated by others, by creating reflective character types or personae. Individuals may accommodate more than one reflection stereotype, but isolating these in caricatures permits participants to focus on specific issues and promotes examination of one’s own attitudes to reflective learning.

The participants were invited to form four groups to consider one each of the following types: the ‘groupie’, the ‘ruminator’, the ‘loner’ and the ‘non-reflector’ (Figure 3).

FIGURE 3.

Co-design meeting 2 resources. (a) Reflection personae; and (b) reflective learning activities table.

For each of these personae:

• Small group session 1 – participants created a fictitious but believable character based on that reflection type using the template (20 minutes).

• Plenary session 1 – participants described that character to the other groups (5 minutes for each group).

• Small group session 2 – participants discussed how each character would respond to a good feedback scenario and a negative feedback scenario (20 minutes).

• Plenary session 2 – participants considered what actions or interventions might promote effective use of feedback by each character (5 minutes for each group).

The scribe classified outputs from plenary session 2, which were subsequently transcribed using the 2 × 2 table to categorise actions and interventions according to whether they were predominantly in-action or on-action interventions and for use by groups or individuals.

Co-design 3: designing interventions to incorporate reflection in everyday activities

The third meeting was conducted twice: the first time it was facilitated and supported by the PEARL project management committee and the second time it was conducted independently by the local teams using additional tools. We describe here the second of these two meetings.

Participants were asked to examine the list of prioritised reflection tools and activities (see Table 9).

In step 1, participants were given polling cards to give their private views on the extent to which the unit staff had the capability, opportunity and motivation to reflect on patient and staff experience (Figure 4). The anonymous responses were collected, the scores for each element (capability, opportunity and motivation) were summed and the percentage agreement for each proposition was determined by dividing the sum of the scores for each element by the number of respondents and multiplying by 100.

FIGURE 4.

Co-design meeting 3 resources.

In step 2, the difference between the actual percentage agreement and the maximum percentage agreement (100%) provided an indication of the perceived gap between the current position and the ideal position for each element of the framework. Participants were asked to discuss the possible reasons for each gap, for example by asking ‘What would be needed for you to give this element a top rating of 5?’.

In step 3, based on the gap analysis, from the list of prioritised interventions (see Table 9) and working in small groups the participants developed a maximum of three using the reflection specification template (Table 3). Participants were asked to focus on actions or activities that promote reflection, not on those that ‘fix the problem (identified using the feedback collected)’. For example, when considering how to enhance reflection during handovers or ward rounds, using a checklist is unlikely to promote reflection (although it will be highly beneficial for many other reasons), whereas starting the handover or round with some feedback from one of the surveys, encouraging input on the round from the relatives or ending the round with a quick debrief, could stimulate reflection by giving everyone a voice.

In step 4, in plenary session, participants presented their activity to the whole group. A task management and delivery plan was completed for each activity.

In step 5, teams were asked to pilot the intervention using rapid cycle approaches (test, evaluate, retest and spread) and provide feedback to the final plenary workshop.

TABLE 3 - Reflection specification template

This specification template was used by the local teams during the final stage of the co-design process to structure their planning of activities to promote reflection in practice. They were asked to maintain a focus on the primary task of promoting reflection rather than on merely describing ‘better care’, thus enhancing awareness of how important communication is for the patient and relatives rather than mandating that staff must communicate with patients or use a checklist.

Reflection intervention
Describe briefly here the nature of the intervention (e.g. ward round, handover, debriefing, posters, coffee mornings, induction training) and then complete the right-hand column below:
What will you do?	•
Why will this activity help staff be reflective?	•
Who can participate in this activity?	•
When will it occur? In-action (‘hot’), near action (‘warm’), or on-action (‘cold’)?	•
Where will the reflection take place/be located?	•
How can the reflection be structured or supported?	•
Barriers: what might limit the effectiveness of this reflection?	•
Enablers: what factors might make the reflection more effective?	•
Outputs: what impact will this particular act of reflection have?	•

Prioritisation of the reflection activities and tools

We identified potential candidate tools and interventions for inclusion in the toolkit in several ways. During the co-design meetings and workshops, staff identified numerous methods for promoting reflective learning. Some of these activities were already in routine use, others were aspirational and some were unconstrained by practical considerations. The ethnographers also identified and described approaches for reflection in practice from their observations and interviews.

We combined the outputs of these nine workshops with the findings from the observational and interview work undertaken by the ethnographers to assemble a list of tools and activities that had the potential to enhance reflective learning. This list was then circulated to the local project teams as a questionnaire, with the request that each member rated each of the tools or activities for effectiveness and feasibility using a 5-point Likert scale (5 = strongly agree, 4 = agree, 3 = neutral, 2 = disagree and 1 = strongly disagree). Effectiveness was defined as ‘the likely impact of the activity if it were to be introduced to your unit’. Feasibility was defined as ‘the likelihood that it could be introduced and would be acceptable to most staff’.

Responses were aggregated and median ratings were determined. The following cut-off points were used to determine inclusion, exclusion or further consideration:

• inclusion – effectiveness median of 4 or 5

• exclusion – effectiveness median of 1 or 2, or effectiveness median of 3 and feasibility median of 1, 2 or 3

• further consideration – effectiveness median of 3 and feasibility median of 4 or 5.

The outputs from this process are described in Chapter 3.

Respondents and response rates

The PEARL project patient and relative survey was distributed continuously for 2 years from June 2017 to May 2019. A total of 18,616 surveys were disseminated and 4747 responses were received (overall response rate 25.5%). Response rates were higher in ICUs (35.4%) than in AMUs (20.1%) (Table 4).

Approximately 60% of surveys were completed by the patient, 20.5% by family members and the remainder by both the patient and the relative. Given the difference in the severity of illness, it was expected that more ICU than AMU responses would have been completed by the relative; however, this was not the case: the distribution of respondents in the categories was similar for both AMUs and ICUs (Table 5).

The self-identified ethnicity of the respondents was predominantly white British (88.8%) (Table 6).

Age characteristics of the respondents are shown in Figure 5. In total, 59.8% (n = 2796) of respondents were aged ≥ 66 years.

FIGURE 5.

The PEARL project patient and relative survey: respondents by age.

Quantitative analysis of survey responses

Respondent ratings from ICU patients and relatives were more positive for all questions than for AMUs. In response to the NHS Friends and Family Test question ‘How likely are you to recommend our unit to friends and family if they needed similar care or treatment?’, 93.5% of ICU patients and relatives selected ‘extremely likely’ or ‘likely’, compared with only 74.3% of AMU respondents (p < 0.0001). This difference was also evident in response to the question ‘How would you rate the overall quality of care you/your relative received in the unit?’: 93.1% of ICU respondents selected ‘excellent’ or ‘good’, compared with 72.0% of AMU respondents (p < 0.0001). There was concordance between patients and relatives in the ICU in their responses: 93.9% of ICU patients selected ‘extremely likely’ or ‘likely’, compared with 92.0% of ICU relatives in response to the Friends and Family Test; 93.4% of patients rated care as ‘excellent’ or ‘good’, compared with 92.3% of relatives. In AMUs, however, patients rated care more positively than relatives: 76.2% of patients compared with 66.7% of relatives selected ‘extremely likely’ or ‘likely’ to the Friends and Family Test question, and 74.4% of patients but only 62.6% of relatives rated AMU care as ‘excellent’ or ‘good’ (p < 0.0001).

All other questions in the PEARL project patient and relative experience survey (excluding question 5, ‘were you ever in pain?’) were phrased as statements with a 5-point Likert scale of agreement for responses (strongly agree to strongly disagree). Notable differences were seen between ICU and AMU respondents, with higher ratings for perceptions of being treated with dignity and respect in ICU respondents (97.0%) than in AMU respondents (88.9%) (p < 0.0001). No other differences were evident between responses from patients and responses from relatives. Confidence in staff was high for both the ICU and the AMU, but again differences were apparent: ICU respondents were more likely to express confidence in doctors (97.7%) and nurses (96.5%) than AMU respondents (88.8% and 88.3%, respectively).

All respondents stated that they were more likely to get answers that they could understand from both doctors (89.2%) and nurses (92.7%) in the ICU than in the AMU [77.6% (p < 0.0001) and 80.2% (p < 0.0001), respectively]. Relatives stated that they were more likely to understand the responses they received from nurses than responses they received from doctors (ICU 93.2% vs. 86.9%, p < 0.0001; AMU 78.6% vs. 72.7%, p < 0.0001).

Significant differences were seen in response to the statement ‘I was able to speak to a doctor when I wanted to do so’: 75.4% of ICU respondents agreed or strongly agreed with this statement, compared with 53.7% of AMU respondents (p < 0.0001). No difference in response was seen between ICU patients and ICU relatives; however, there was a notable difference between AMU patients and AMU relatives: 57.1% of patients agreed or strongly agreed with the statement, compared with 45.8% of relatives (p < 0.0001).

Analysis using performance–importance plots

To illustrate the relationship between the level of satisfaction with each item in the questionnaire and how that particular item influenced overall satisfaction, PIPs were generated. As described in Chapter 2, the PIP compares the percentage of ‘best response’ ratings for each survey item with the correlation coefficient of each item and the overall weighted satisfaction score. Figures 6 and 7 include guidance on interpreting such plots.

FIGURE 6.

Performance–importance plot: aggregated ICU patient survey responses (n = 2322).

FIGURE 7.

Performance–importance plot: aggregated AMU patient survey responses (n = 2425).

The results for ICU respondents are shown in Figure 6 and for AMU respondents in Figure 7. Each survey statement is identified by a number. The survey statement responses relating to communication are coded dark blue and those relating to clinical care are coded light blue. The two questions relating to overall satisfaction are coded orange.

Three conclusions can be drawn from inspection of these plots. The first is that all of the responses relating to communication are clustered in or near the left-upper quadrant, indicating that perceptions of the adequacy of communication strongly determine overall satisfaction. The second is that ICU respondents have a higher median best satisfaction rating (54%) overall than AMU respondents (31%). The third is that, despite this difference between AMUs and ICUs, the pattern is retained: patient and relative experience of communication with staff remains the main driver of (dis)satisfaction.

There are several plausible explanations for these differences between AMUs and ICUs. Compared with ICUs, AMUs have much lower nursing and medical staffing intensities, a higher patient throughput, shorter lengths of stay and, therefore, less time for staff to establish fiduciary relationships with patients and families. It is also likely that, given the high throughput and short length of stay, the AMU data are ‘contaminated’ by patient and relative experiences acquired before AMU admission (e.g. in the ED or a ‘holding area’ awaiting admission) and after transfer out of the AMU to other wards in the hospital with even lower staffing levels.

However, the purpose of the survey is to provide staff with quantitative and qualitative data that can be used as the raw material to promote reflective learning at both the individual and the organisational level. Here, a very clear message emerges: the technical aspects of care are by and large well delivered and it is the non-technical, behavioural aspects of care that affect patient and relative experience.

Analysis of free-text responses

Of the 4747 survey responses received, 3275 (69%) had additional comments relating to how care could be improved and what was done well. Free-text responses from patients and relatives were substantially positive (≈ 60% of comments).

Response rates

Two rounds of the PEARL project staff survey were conducted in January and February 2018 and 2019. In 2018, 1694 surveys were distributed to AMU and ICU staff, of which 681 completed questionnaires were returned (response rate 40.2%) (Table 7). In 2019, 1541 surveys were distributed and 870 completed questionnaires returned (response rate 56.46%). Surveys were supplied with a freepost return envelope, allowing respondents to complete the survey anonymously before returning it to the central project team for processing and analysis. Aggregated reports that outlined findings for each unit were provided.

Quantitative analysis of survey responses

Analysis of free-text responses

Of the 1551 PEARL project staff experience survey respondents, 35% provided additional free text about working at their organisation. The proportion of positive to negative free-text comments was the reverse of that seen with the patient survey: approximately 78% of staff free text contained negative comments. These predominantly related to lack of training and progression opportunities, issues with teamworking, staff feeling unsupported in their roles, communication and the overall impact of working on staff well-being.

Patient experience data

Patient experience data are collected in different forms, which fall into three broad categories. Formal data are collected and collated at the organisational level, such as questionnaires/surveys (including the PEARL project survey) and the Friends and Family Test, and through non-solicited mechanisms, such as complaints, contacts via the PALS or recognition of individual members of staff through local awards. Informal data available to individual front-line staff about patient experience come from items such as ‘thank you’ cards and gifts from patients/relatives and bedside comments. In between these formal and informal sources of feedback are other forms of feedback that front-line staff receive that are generated as part of organisational activities; however, these types of feedback are not collated and used formally for organisational activities. These include patient stories generated when patients are invited to come back and share their experiences, and individual-level feedback that is collected for mandatory revalidation or appraisals. We refer to these three different types of feedback as happening ‘above-the-line’, ‘below-the-line’ and ‘between the lines’, respectively (Figure 8).

FIGURE 8.

Forms of feedback. M&M, morbidity and mortality. a, Facebook, Inc., Menlo Park, CA, USA; URL: www.facebook.com.

If all types of data are taken into account, there are currently a broad range of sources of patient experience data potentially available as a stimulus for reflection. Sites varied in their approaches to actively gathering formal data on patient experience and in the extent to which staff had access to the different forms of feedback. In terms of formal sources of data, where questionnaires were being used these were usually administered during the patient’s stay, but were often not used systematically, and the approach was widely described as being ‘hit and miss’. The Friends and Family Test was used in some of the AMUs but did not tend to be used in the ICUs at any site. A few ICUs, or sections of ICUs, did try to collect regular patient or relative data through the use of questionnaires/forms. Not every ICU had formal, regular collection of patient experience data; some ICUs collected patient experience data on an annual or 2-yearly basis, whereas others used forms, such as the ‘Take two minutes’ form, or ‘suggestion boxes’ in the relatives’ room to collect relatives’/patients’ experiences. All sites had systems for collating and feeding back complaints.

Doctors who collected patient experience data for the purpose of their own appraisals were able to collect from any ward in the hospital including the AMUs and the ICUs; however, it seemed that the doctors felt that it was difficult to hand out patient surveys in these locations.

In terms of organisational opportunities for staff to hear patient stories, in some participating ICUs (usually those with extended stays) staff had the opportunity to attend follow-up clinics or coffee mornings to hear about patients’ experiences from patients and relatives/carers; however, these sessions were primarily designed to help patients’ reorientation as opposed to a means of generating patient experience feedback. AMUs did not have a similar scheme.

All staff had the opportunity to receive informal patient feedback, for example at the bedside or through receiving thank you letters, but this was more likely to occur on ICUs, where patients had extended stays. In addition, some staff felt that nurses and other ward-based staff were more likely than doctors to receive verbal patient experience feedback because they were more of a consistent presence on the ward.

Reflective practice

The co-design meetings

At the start of the first meeting in each centre, following introductions, participants were invited to describe touchpoints or ‘reflectable moments’ that had happened during clinical practice and stimulated reflection and personal insight in themselves or in others. Many of these accounts, from both staff and patients, were emotionally charged and at times transforming, in the sense that there was little need thereafter to explain why effective reflection was important. It also demonstrated that, once emotional engagement had occurred, it needed to be balanced by sufficient distance to permit objective analysis and the transition to ‘conscious competence’. This was achieved in the second half of the meeting by asking participants to create a reflective learning scene or event using pre-prepared cardboard cut-out figurines and ward furniture, which participants could personalise. There was excellent engagement by team members in this activity, but the team members varied in the extent to which the events were used to describe personal learning. The clinical staff in particular tended to focus their attention on producing better care rather than ‘better reflection to produce better care’. Indeed, throughout the project we had to emphasise the need for clinical participants (including core project team members) to focus on reflection as the behaviour, not the secondary outcome of better care.

In the second set of local co-design meetings, participants used stylised characters (‘personae’) to discuss individual differences in reflective learning. Themes that started to emerge from these discussions included the need for organisational support for reflective learning, integrating reflection in continuing professional development activities, training in reflection, senior role models, reflection mentors or ‘buddies’ to provide individual support and prompts for reflection. ‘Loners’ or ‘ruminators’ were perceived as needing 1 : 1 support to encourage participation in group activities. Non-reflectors were perceived as problematic if this was associated with dysfunctional behaviours and a lack of insight, in which case forms of mandated feedback (multisource/360-degree feedback), senior engagement and sanctions and challenges could be deployed if there was organisational support for speaking up. There was a desire for methods of evidencing reflection that would not compromise confidentiality or diminish authenticity, and for demonstrating that reflection had resulted in learning, either individually or at an organisational level.

Teams were invited to the third set of co-design meetings to test a method for developing two reflection interventions. In the first half of the meeting, teams were asked to identify and discuss the various opportunities for reflection that they had experienced, or would have liked to be made available, in their units; 31 opportunities were identified, ranging from solitary reflection on-action at home to formal reflection meetings or reflection in-action. Teams then selected the two opportunities that they considered the most effective, and in the second half of the meeting they developed these ideas using the reflection specification template (see Table 3). This provided a ‘dry run’ for the final team meetings that were held locally without external facilitation (described in detail below and in the second phase of the ethnography observations).

Plenary workshops

The four workshops were held on 14 June 2017, 15 February 2018, 20 December 2018 and 4 June 2019. The workshops were held in the conference park of the University of Birmingham, Birmingham, from 10.30 to 16.00, with a break for lunch. There was variation in attendance, which was driven in large part by the constraints of the clinical service. There was PPI representation at each workshop.

Following the first scene-setting plenary workshop, the second, third and fourth workshops were interdigitated with the local co-design meetings. For each workshop, the morning session enabled teams to report on progress with reflective learning activities and on how they had used the patient and staff survey reports. This allowed teams to share experiences and learn from each other. The central project team members then provided de-identified feedback on ethnographic observations, discussed pedagogic considerations (methods of reflection and learning) and reviewed tools and techniques for behaviour change. In the afternoon sessions, teams undertook practical exercises and the day concluded with a summary from the chairperson of the steering committee.

In the second and third workshops we introduced to the local teams various models of COM-B as a framework linking forms of reflection (on-action and in-action, group and solitary) to determinants of reflection (capability, opportunity and motivation) and their subdomains, and thence to factors influencing those domains. We show this in Figures 9–12. This framework was considered too complicated to be used as a practical tool by the local teams but functioned well conceptually and for diagnostic and developmental applications by the central project team. The high-level elements of capability, opportunity and motivation proved to be useful classifiers of the reflective learning interventions that were prioritised by the local teams following workshop 3. In workshop 4, the participants presented the reflective learning tools that they had selected, developed and piloted. We describe this in detail in the next section.

FIGURE 9.

The PEARL project COM-B reflection framework. CPD, continuing professional development; M&M, morbidity and mortality; MSF, multisource feedback; QI, quality improvement.

FIGURE 10.

The PEARL project COM-B reflection framework: motivation to reflection (MINDSPACE). MSF, multisource feedback.

FIGURE 11.

The PEARL project COM-B reflection framework: opportunity to reflect. MSF, multisource feedback.

FIGURE 12.

The PEARL project COM-B reflection framework: capability to reflect. MSF, multisource feedback.

Meeting logistics

As described in Chapter 2, a meeting was planned by each hospital taking part in the study to select up to three interventions for each unit to aid or promote reflection. The core project team provided guidance on how to run these meetings, with a pre-designed slide set and a pre-meeting telephone call with the project leads. The meetings were observed by an ethnographer. Interviews were conducted with a sample of meeting participants, who were chosen to access the views of people with a variety of roles; interviews were conducted by telephone following the meetings. The focus was on optimising the approach for sites to select and implement interventions: key lessons learned about how to optimise the approach are included in Selecting interventions.

A variety of staff from the ICUs and the AMUs attended these meetings including, among others, consultants, registrars, matrons and nurses; one hospital site also had PPI representatives. There was variation among the participants in terms of seniority within the hospital and familiarity with the PEARL project. For all of the meetings there was a good mix of staff present; however, it was felt that attendance by more senior staff and a wider mix of hospital staff, such as physiotherapists and health-care assistants, would have benefited the discussion and selection of interventions:

Because obviously I’m quite a junior sister to the unit . . . I think from both our point of view, and just seeing who was involved in the other teams at the other trusts, I feel like maybe we weren’t represented enough from more senior staff.

Int 017, nurse

The obvious staff groups that weren’t represented would be allied health professionals, so I’m talking about physiotherapists . . . and pharmacists and people like that. They’re not kind of absolute core members of the team . . . but they are there a lot of the time, and should probably be represented as far as possible.

Int 038, doctor

The meetings took place at different times during the day at the hospital sites, with breakfast meetings and lunchtime meetings being two of the options. There were disadvantages to both time slots, with respondents generally feeling that people were rushing to leave the breakfast meeting and that lunchtime meetings were delayed because morning activities were over-running. Although all the sites had scheduled different lengths of time for the meeting, ranging from 1 to 2.5 hours, all respondents commented that more time was necessary and that the meeting felt rushed at the end:

There’s something about that there should have been more time, definitely and there should have been more time then to go through these activities contributing to reflection because that was not discussed. It was literally they were given the list, quickly choose one, quickly discuss it.

Observation

There were staff attending the meetings who were less familiar with the PEARL project and the methodology; although time was given to provide background information, some respondents felt that this was inadequate. This was particularly the case for one of the exercises that required staff to judge the ‘capability’, ‘opportunity’ and ‘motivation’ for reflection already present, so that a ‘gap’ could be identified. It was felt that this exercise was overly complicated and, although it did create discussion, it was not used by all to select interventions:

I think I added two slides in, and I did make some changes. I think, because at the meeting there were some people who were new to PEARL. Completely, they needed at least one slide that actually explained the capability, opportunity, motivation model. And what sort of things we were looking for.

Int 035, doctor

The meetings were viewed positively on the whole; they were described as being very constructive with lots of enthusiasm from the participants. During the meetings there were occasions when groups were working together; this may have been a unit with different disciplines discussing ideas or it may have been those from a similar discipline but different grades working together. What was observed was that these groupings gave an opportunity for exchanges of ideas and learning from others, be that a different unit, discipline or grade. Further discussions, through e-mail or formally planned meetings, took place after the initial meeting to help develop ideas:

It was actually one of the, I felt, one of the better meetings that I’d been to regarding PEARL. I think it was very well attended, people were very enthusiastic, and seemed keen and interested in it. There was quite a lot of revelation during our meeting, you know, the ITU [intensive therapy unit] nurses discovering that the CDU nurses had no staff room.

Int 036, nurse

Overall, negative comments about the meeting that highlighted areas for improvement included the length of time allowed for the meeting, there not being enough background for new people, the need for a broader spectrum of staff to attend and the complexity of some of the exercises. There were different views expressed about the selection process for the interventions, with the view being that the key criteria should be that the interventions should be practical and feasible, whereas some felt that this potentially restricted their creativity:

I think we were quite constrained to what was said, and it may have been that more ‘out of the box’ suggestions might have come up, but because of the format we did just come up with things that . . . were not particularly ‘out of the box’, were not particularly novel.

Int 038, doctor

Selecting interventions

There were a variety of approaches taken to select the interventions; very few of the groups used the capability, opportunity, motivation exercise, described above, that focused on identifying the gaps to then inform the selection of interventions:

I think from our point of view, because we were already in the process of certainly starting the coffee mornings, they were already running, that one seemed very obvious, because actually we could build upon it and do it within the reflective PEARL framework.

Int 020, nurse

I think we went through them and looked at what was practical and what might not be practical.

Int 057, doctor

I think it almost gave it a bit more of an impetus to sort of go OK, this might have a bigger impact, but a bit trickier, but actually why don’t we go with it.

Int 039, doctor

Approaches taken to select the interventions included:

• voting on the supplied list of interventions

• developing an idea that was already ongoing

• selecting an intervention that was feasible to implement in a short time period

• choosing interventions that were felt would have a lot of impact.

After the interventions had been selected, the teams had to outline the plans for implementing them. Some found this harder than others, depending on whether the intervention was already in place to some degree or if it was a completely new idea. There were tensions evident in some groups in which the discussion about the potential intervention focused on the potential problems without taking a solution-focused approach.

The respondents expected the interventions that they had selected to have an impact on reflection, but could not always say how much impact or how long this might last. Some commented that they hoped the intervention would last beyond the life of the project. There did appear to be a trade-off between impact and feasibility, with interventions that were seen as easier to do expected to have less impact than other interventions that required more time and resources. One issue that caused problems for groups was finding suitable metrics that would give evidence that reflection had taken place. Concern was raised regarding how impact would be measured and, therefore, how staff would know that the intervention had made a difference:

I think the main issue with us was like that it was . . . about the metrics, wasn’t it? I think that was my main issue . . . we had patient feedback from like coffee mornings, and that was given to staff as well who came along, but we have no official way of measuring [impact].

Int 017, nurse

Will the intervention make a difference to certain individuals? Yes, I’m sure it will. And therefore in that respect, it’s worth doing. Will it make a difference that’s measurable in any way, whether, in terms of staff morale or staff sickness rates or patient experience? I’d love to think it will but I think it will be very difficult to show.

Int 037, doctor

Implementing interventions

Many of the selected interventions had been implemented following the meeting; therefore, it was possible during the interviews for the respondents to comment on the ones that they were involved with. Some interventions had been quite successful and were starting to become embedded in the unit or used beyond the original plan; this was often the case when the intervention was already partly under way and some staff members had taken ownership of the intervention (e.g. patient coffee mornings). Other interventions proved more problematic and either had not been started or had been tried with the realisation that amendments were needed if the implementation was to continue. Reasons for the intervention not working included practical limitations; for example, the discovery that reflecting during handover was ineffective because there was not enough time, or that staff at the end of their shift were exhausted and were, therefore, not engaging well. A common feature of the interventions chosen, however, was that sites had primarily chosen interventions that provided improved access to patient feedback, put prompts in the environment to encourage reflection or provided an opportunity for reflection to take place, such as through time in meetings or interaction with reflection mentors:

As I said, the mandatory training days slot, we’ve almost embedded that actually, it’s almost become a permanent fixture.

Int 059, nurse

A notable omission from the interventions selected was the use of frameworks or models of reflection to ensure that reflection was effective, in particular ensuring that staff ‘closed the loop’ on reflection to ensure that there are changes in attitudes and behaviours. One exception to this was one site that developed and displayed a poster illustrating the steps involved in completing a reflective cycle. This was developed further by the project management team, with the centre involved to create the PEARL project reflective cycle.

Respondents indicated that they would continue to work on the interventions that they had selected, as well as look to implement other ideas. There also seemed to be a ‘knock-on’ effect, with some interventions being used more widely as other members of staff became aware of them and saw other opportunities for their implementation:

The education side, we were just literally launching them [feeding back about patient experience from the coffee mornings] . . . and as a result of the PEARL, [person 5], our matron, has actually asked, after listening the other day to what we were actually doing with the role play and things, she’s actually asked us to present it to band 6 and band 7 meetings and things like that.

Int 017, nurse

Staff suggestions for the final toolkit

Staff were asked about the main challenges to be overcome in promoting the take-up of the toolkit in hospitals. These included the following:

• Reflection was seen as very important by the respondents but was also acknowledged as easy to avoid when there were intense time pressures.

• The ability of staff to have the time and opportunity to go and reflect if it meant them leaving their patients.

• Staff may not feel motivated to reflect, for whatever reason, and, therefore, culture change may be necessary to motivate these staff to reflect – this is challenging.

• There was concern raised about the legal implications of doing written reflection and how that may impact the staff member and how they choose to reflect.

• Staff identified that having a means of capturing that reflection had taken place through some kind of metric as an important but challenging element of the toolkit.

• Staff felt that it would be challenging to introduce the toolkit if there was not enough support from the organisation and buy-in from senior staff.

• It was seen as important to make sure that the ‘right’ people were involved from the start of introducing the toolkit to promote its use.

• For the toolkit to be successful it was felt that it needed to be introduced with lots of promotion and maybe a social media campaign, so that all staff were aware that it was taking place.

• A further challenge was having interventions that were not heavily reliant on one member of staff to make them successful.

• Tools need to be sustainable beyond the initial PEARL project support.

Staff were asked in interviews to give feedback on what they thought was important to include in the toolkit and implementation plan. One of the main responses was to have examples in the toolkit of how the intervention had worked in other units and enough guidance to be able to replicate the intervention but with the flexibility to adapt it to their specific requirements. Staff wanted prompts that they could use to help direct their reflection and exercises they could do to help facilitate discussion and reflection. There was also a request to have some background information on how the PEARL project developed, details on the ‘science’ behind behaviour change and a glossary (because it was felt that some staff may not be familiar with all the terminology) included in the toolkit:

So I think a little bit of guidance, which is where I see the reflection mentors coming in, so that people are reassured that what they’re doing, that they are sort of doing it in a beneficial way. Because we can all just chat about things, but I think to really actually, if we’re going to use this toolkit to benefit the staff, maybe just a little bit of guidance about the background to why we did PEARL, what we found from PEARL, this is why we’ve come up with this toolkit, and you know, we want it to help you, and this is how we envisage it helping.

Int 035, doctor

Key features that staff wanted to be included in the toolkit and in the plan to support its implementation were:

• examples of where the tool has worked in other units (this could be built into an electronic repository or hard-copy examples)

• background on the PEARL project and details on the ‘science’ behind behaviour change and reflection

• glossary of terms used

• simple, straightforward tools that could easily be picked up and used

• access/pointers to other useful resources

• tools that could be completed in a few minutes

• ways of measuring the impact so that there is evidence that reflection has happened or that the tool has worked

• strong branding in the toolkit and maybe some publicity to support its launch

• guidance/tips on how to use the tool – maybe also a video guide.

What excellent reflection would look like

For most staff, a successful outcome from the PEARL project would be reflection becoming normalised and something that happened naturally on the unit and did not feel like a tick-box exercise. It was felt that staff would feel more valued if they were given time and space to reflect and that this was an activity that was encouraged by more senior staff. Excellence would also be indicated by having measurable changes happening as a result of the interventions:

If you get PEARL embedded into the culture of a unit in the same way, that it’s just something we do, I think that would be the ideal utopian state.

Int 038, doctor

I think excellent would look like . . . that people’s reflection would be both a personal thing and an individual thing, but also a team dynamic and a change. I think personally for it to be excellent we need to demonstrate change. Whether that change is of attitudes, behaviours, changes of practices or changes of the environment.

Int 019, nurse

How do local cultures influence reflective learning?

A collaborative local culture with quality-focused inclusive leadership is essential for the success of quality improvement interventions,131 and we would propose to characterise trusts and units in terms of organisational subcultures6 before and towards the end of the intervention. We would propose to evaluate two hypotheses among others. The first is that compliance-orientated organisations may be less susceptible to behavioural modification than collaboration-orientated organisations. The second is that, within whole organisations, multiple subcultures (cliques/clans, hierarchies, developmental and rational) can be identified, which are driven largely by the impact of single individuals.

How do social media create communities of learning?

We would wish to establish web-enabled surveys and to create moderated social media support groups in parallel with the roll-out of the toolkit. Disseminating outputs and sharing experiences in a concise and targeted manner is essential for reaching busy staff members, who may respond differently depending on the way information is presented (‘narrative for nurses’ and ‘data for doctors’ would be one such meme).

Are the results of locally driven experiential surveys more potent influencers than standardised national surveys?

We know that survey results are more influential the closer they are to the subjects themselves. Data and narrative are also complementary. We would, therefore, propose to compare the PEARL project surveys with the results from national surveys (NHS Inpatient Survey, NHS Staff Survey and the GMC Trainee Survey) in terms of how these various sources are used to influence practice and learning, and the extent to which their results influence staff.