Quality and safety between ward and board: a biography of artefacts study

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its preceding programmes as project number 13/07/68. The contractual start date was in July 2014. The final report began editorial review in January 2017 and was accepted for publication in June 2017. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

none

Copyright statement

© Queen’s Printer and Controller of HMSO 2018. This work was produced by Keen et al. under the terms of a commissioning contract issued by the Secretary of State for Health and Social Care. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

2018 Queen’s Printer and Controller of HMSO

High-quality data

There are different views, in the NHS and in academic circles, on the data that are needed to monitor and manage the quality and safety of health services. One is that doctors, nurses and other clinicians are always concerned with quality and safety. Patients’ records, updated in the course of clinicians’ work, reflect the quality and safety of services provided. Thus, albeit at the risk of oversimplification, we might take the following scenario as an example: if a patient’s temperature and blood pressure are normal, the most recent pathology tests are normal and the patient says that she is happy with the care she has received, then the treatment and care were of reasonable or good quality. There is no distinct category of data that reflect quality and safety; rather, the data are part and parcel of everyday service delivery.

A second view is held by practitioners and academics with interests in service improvement. They argue that data play a vital role in improvement projects, perhaps most obviously in the ‘study’ phase of ‘plan-do-study-act’ cycles. That said, few authors talk in any detail about data or information systems. 8,9 The 2013 report by Donald Berwick, the respected US physician, is one of the exceptions to the general rule. He identified a range of routine data that ward staff – and wider clinical teams – needed to investigate unwarranted variations in services and to support service improvement (Table 1). 10 Berwick argued that these data were not routinely available to ward or directorate staff in NHS trusts, and that neither did they have appropriately trained staff to analyse and present the data – the staff lacked expertise in data analytics. His arguments are based on quality management principles, and, as such, stress the importance of learning from adverse events through root cause analyses or ‘deep dive’ reviews. Accounts of adverse events are, therefore, an important source of evidence to inform service improvements.

The second Francis report provides an example of a third view, which is that the quality and safety of services should be monitored by trust managers and external agencies,3 who therefore need appropriate performance data. In recent years, the NHS has required all hospital trusts to provide an increasing number of indicators. Current submissions from trusts include the number of complaints and of incidents (where a patient has been harmed in the course of treatment and care), and data for the NHS Safety Thermometer. The Thermometer includes the number of patients who have developed severe pressure ulcers, experienced a fall, experienced a venous thromboembolism (VTE) and developed a urinary tract infection following the use of a catheter.

There have been a number of studies on the use of data to manage hospital performance, including the recent HSDR study by Mannion et al. 11 There have also been reports that draw attention to the large number of routine data that are captured, and the limited use of these made by the trusts or organisations that have access to them. 12 There are also many studies of individual indicators, notably those on mortality. 13 We have not, however, found any published studies of the processes involved in producing data sets for NHS performance management purposes. Indeed, given the marked differences in the views about the types of data that reveal the quality and safety of services, and the people who are in a position to review and act on those data, it was not at all clear what data trusts were actually capturing and using, or whether or not their uses of data were changing over time.

Information technologies in acute hospitals

Investments in IT systems were recommended by the Bristol Inquiry in 2001,14 which argued that they were a prerequisite for providing the data that clinical teams needed to manage services, and for managers to monitor those services and ensure that they were safe. The Wanless review15 on the future of the NHS, published in 2002, also recommended substantial increases in IT investments. This led to the creation of the NHS National Programme for IT, which had an initial budget of around £7B, eventually increasing to > £10B. The programme had a vision of making patients’ records available anywhere in England: if a Manchester resident fell ill in Norfolk, NHS staff in Norfolk would be able to access that person’s clinical records. However, a succession of National Audit Office reports between 2006 and 2011 reported that billions of pounds were largely wasted. 16 In particular, new, integrated IT systems for acute hospitals were not delivered on time, and even today only a minority of trusts have implemented them. The vision of an integrated, England-wide IT service has never been realised.

The result was that, at the beginning of this decade, most hospitals found themselves with ageing systems and a pressing need to upgrade and replace them. 17 Many trusts made rapid progress once the National Programme was officially abandoned in 2011, and implemented systems across most wards and departments. Even so, having been held back for so long, most trusts needed to make further substantial investments at the time this study started in 2014.

The literature: evidence and theory

Our interest in this study is in the data and IT systems that are used to monitor and manage the quality and safety of services. There are a number of literatures that could be drawn on. Taking our cue from the ‘views’ outlined above, one focus of current investments is on acute hospital wards. There is a rich literature on human–computer interaction in health care. 18 However, most studies focus on accident and emergency departments and intensive care units rather than on acute wards. 19,20 There is also a growing literature on the use of mobile devices, including laptops and tablets, on wards. 21 This study contributes to the literatures on whiteboards and mobile devices on acute wards. It is worth noting that wall-mounted systems are typically referred to as ‘dashboards’, or ‘clinical’ or ‘quality dashboards’, in the literature. There is a risk of confusion in this report, because the term ‘dashboard’ is also used to refer to the graphical displays used in board reports in the NHS. Following the use of terms in the NHS, we refer to whiteboards on wards, and dashboards in board reports, from Chapter 2 onwards.

We are also interested in hospital-wide systems: in the kinds of systems that NHS trusts need in order to provide data to managers. Most trusts have eschewed the single hospital-wide – or ‘big bang’ – solutions favoured under the NHS National Programme. They have instead opted to develop communication networks and integration engines – in effect, software that supports access to several systems via a single screen – that allow clinicians to access data from several departmental systems via a single screen. They build on what is there, rather than replacing existing systems in a single hospital-wide implementation.

There are a number of traditions of studies on these systems. 22 For example, some writers have drawn on structuration theory to understand the deployment of IT systems in organisations. Barrett et al. 23 and Beane and Orlikowski24 built on this work, and critiques of it, and used practice theory to explore the ways in which health-care organisations and IT systems shape one another. Other writers work in a very different tradition, whereby organisations are viewed as complex systems, and IT systems are technologies that can be used to address complexity. 25 Additionally, other writers have taken a more instrumental view of organisations, arguing that it is possible – and useful – to identify barriers to, and facilitators of, organisational change. In this study, our approach has more in common with the tradition represented by practice theory than with the view that uses complexity or barriers and facilitators. On the latter, we note that our fourth study objective refers to barriers and facilitators. For reasons we discuss in Chapter 11, we did not find thinking in terms of barriers and facilitators helpful in this study.

A science and technology study

We made our decisions about the nature of the study in the context of these literatures, and in the light of the comments we have made about data and IT systems. Specifically, our task was to investigate and understand developments at the levels of both the hospital ward and the board. This ruled out a narrow focus on individual technologies such as electronic whiteboards. More positively, we were able to draw on the conceptual and methodological developments in institutional approaches, such as practice theory, and on the critiques of those theories that have developed over the years. We opted for the Biography of Artefacts approach, which is located in the tradition of science and technology studies. 26,27 This approach appears to have developed in parallel with structuration, practice and other institutional theories, with which it shares a number of features. It starts from the observation that IT systems in organisations are implemented over many years. New functions are added periodically, and existing systems are linked to one another and to new systems. Systems thus develop in piecemeal fashion, producing digital infrastructures that are deeply embedded in the day-to-day work of an organisation.

We use the term ‘infrastructure’ throughout this report. An infrastructure is an amalgam of a number of previously separate components. Siskin provides a useful description:

. . . infrastructures are never built from the centre with a single design philosophy. Rather they are built from the ground up in modular units, their development an oscillation between the desire for smooth, system-like behaviour and the need to combine capabilities no single system can provide.

Siskin, p. 628

As the quotation implies, the ways in which infrastructures are used can change over time. As a result, if we want to understand any IT system in an organisation – understand why it looks the way it does today – we need to understand its history. This led us to design a study that adheres closely to the technologies of interest and to the working practices associated with them. It is, in part, a ‘hidden history’ of the people who capture and validate data, who prepare dashboards for board papers and who upload data sets to NHS Digital. It is also a story of what happens when the data arrive in different places: hospital directorates, board meetings and national and local agencies. We are not aware of any previous studies of the NHS trust infrastructures that produce the data used to monitor and manage the quality and safety of services.

Aims and objectives

In our proposal, we described our research aims and objectives as follows:

The research has two aims. The first and principal aim is to establish whether or not ward teams in acute NHS trusts have the information systems they need to manage their own work, and to report on that work to trust boards and other stakeholders, in the post-Francis environment. The research will focus on the design and implementation of a key technology, ward-level dashboards. Dashboards are already the preferred NHS vehicle for integrating information from diverse sources, are being actively promoted by NHS England and NHS Digital (formerly the Health and Social Care Information Centre). As earlier sections show, dashboards will need to be redesigned in the light of the Francis, Keogh and Berwick recommendations. The second aim is to establish the extent to which ward-level dashboards provide a basis for achieving the openness, transparency and candour envisaged by Francis, and supported by Keogh and Berwick. As the reports emphasise, although there are examples of excellent practice, the NHS as a whole needs to undergo a culture change, moving from low to high trust working practices. The extent of sharing of detailed information about performance will be an important source of evidence about that culture change.

Keen et al. 29

There are four research objectives:

1. Design. We will assess the extent to which trusts are able to integrate activity, quality, outcome and cost information in dashboards, to enable ward teams to manage their services effectively and to improve services over time.

2. Implementation and use. We will evaluate the impact of the use of dashboards on clinical and management practices at ward level.

3. Governance. We will assess the extent to which dashboards provide data that are valuable to other local stakeholders, including trust boards, Healthwatch and commissioners.

4. Barriers and facilitators. We will identify the barriers to, and facilitators of, the effective redesign and use of dashboards.

We referred to dashboards in the text reproduced here, and throughout the proposal, but, as we noted above, we make a distinction in this report between whiteboards used at ward level and dashboards used at board level. It should also be noted that we address both aims and the first three objectives in the report, but we do not discuss barriers and facilitators. We explain our technical concerns with these terms in Chapter 11.

The structure of this report

In Chapter 2, we set out the study design and methods used in the study. In Chapter 3, we outline the development of national arrangements for data collection and IT over the last 30 years. In Chapter 4, we present the methods and findings of the first phase of the study, a 2014 telephone survey of senior trust nursing managers about their use of whiteboards and dashboards, and of trust board papers in January 2015. In Chapters 5–10, we set out the main findings of the study, starting with a brief overview of the four study sites and following with mini-biographies of developments on wards, in data and technology infrastructures, of board quality committees, of directorates (also termed clinical service or business units in NHS trusts) and of external bodies including commissioners and regulators. In Chapter 11, we discuss our findings and identify implications for practitioners and recommendations for researchers.

Key points

• A telephone survey of 15 acute hospital trusts and a survey of board papers of all acute hospital trusts in England were undertaken in 2014 and early 2015 respectively.

• We then observed the use of information systems in four acute hospital trusts over an 18-month period in 2015 and 2016.

• We used a number of methods, including the direct observation of the use of whiteboards and other technologies on wards, an observation of board quality committees, semistructured interviews and an analysis of board papers.

• Normalisation process theory was used to direct our fieldwork.

• The Biography of Artefacts approach was used to analyse our findings.

Introduction

This chapter describes the study design and methods. It has three main sections, reflecting the three phases of the study: scoping and selection, field research at four sites, and analysis and interpretation. As Figure 1 emphasises, the fieldwork and analysis were undertaken iteratively: they are presented separately here for clarity.

FIGURE 1.

Sequence of research activities.

Scoping and site selection

The study was approved by the University of Leeds Faculty of Medicine and Health Research Ethics Committee (see Appendix 1). We identified 18 acute trusts that were within a reasonable travelling time – 75 minutes each way – of our university base, and thus were potential sites for the in-depth phase of our study. Research governance approval was successfully obtained from 15 of the 18 trusts. We were unable to obtain research governance approval in the other three trusts. We undertook one interview in each trust, 11 with chief nurses and four with senior colleagues whom chief nurses nominated.

Field study in four acute hospital trusts

We designed this phase of the study as a prospective, iterative case study design, focusing on two wards – one surgical and one medical – in each of the four acute NHS trusts. Given the focus of the ‘After Francis’ HSDR call, and of Sir Robert’s second report, we excluded intensive care units from the study. More positively, the fact that this was an ‘After Francis’ study focused our attention on the management of the quality and safety of services on wards, and upwards ‘from ward to board’. We named the four trusts Solo, Duo, Trio and Quartet to protect their identities.

There are many types of case study approaches, ranging from Yin’s scientific to more naturalistic ones. 33 Our approach was towards the naturalistic end of the spectrum, on the basis that the best way to understand implementation is to spend time observing the people doing the implementing. 34 As a result, we were principally interested in the working practices of nurses and nurse managers, although we did also observe and interview other clinicians on wards, and we observed and interviewed a range of staff elsewhere in the four trusts. We judged that focusing on wards alone would be outside the scope of the call. Accordingly, we did not propose to observe clinician–patient interactions or detailed data capture in medical and nursing records; that would be an interesting study, but one undertaken in response to a different call for proposals.

Ethics approval was obtained from the University of Leeds Faculty of Medicine and Health Ethics Committee in July 2014 (see Appendix 5). The observations and interviews undertaken at the four sites are set out in Tables 2 and 3. We were able to interview almost everyone we approached. We were not, however, able to interview one trust informatics director, one medical director, one member of an information team, two junior doctors, two nurse managers (both towards the end of the fieldwork) and representatives of one Healthwatch and three clinical commissioning groups (CCGs). We do not think that the omission of most of these interviews had a significant impact on the conduct of the study or on our analysis. We do, however, note that reliance on one CCG interview limits what we can say about the role of CCGs. Two CCGs refused to talk to us – one instructed us to ‘remove us from your records’ – and we were not able to get a response from a third. We do not know why some CCG staff responded in the way that they did; their reactions to our approaches stand in marked contrast to the positive responses from the great majority of people whom we approached.

Normalisation process theory

A number of established theoretical frameworks have been used to guide the design of studies of technologies in organisations, including health-care organisations. Our interest was in the implementation of new ways of working associated with IT. In our proposal, we argued that normalisation process theory provided the appropriate focus and had a track record of successful use in case studies in the NHS. 35,36

We used the theory to guide us towards phenomena of interest and, thus, to design our data collection methods. The theory rests on three main arguments. First, it proposes that practices become embedded in social contexts as a result of people working, individually and collectively, to implement them. Second, implementation is operationalised through four generative mechanisms – coherence, cognitive participation, collective action and reflexive monitoring. 35,36 If those involved in the implementation of dashboards can identify coherent arguments for adopting them, are engaged in the process of implementation, are able to adapt their work processes to use dashboards (or dashboards to fit in with practices) and judge them to be valuable once they are in use, then the dashboards are more likely to become embedded in routine practice. Third, embedding new ways of working is not a ‘one-off’ process, but requires continuous investment by the parties involved in implementation.

Figure 2 shows how the components of the theory are related. The lower half of the figure emphasises the importance of understanding working practices in local contexts. The upper half draws attention to institutional arrangements: the ways in which prevailing values and norms influence the behaviour of trust staff. The theory, and in particular the four generative mechanisms, provided a framework for designing the fieldwork. For example, it focused our attention on the value of observing ward working practices and, thus, evaluating the extent to which whiteboards were integrated into them (collective action). Similarly, it encouraged us to ask interviewees about their experiences of implementation and of using new technologies (cognitive participation and reflexive monitoring).

FIGURE 2.

Model of the components of normalisation process theory. From May C, Finch T. Implementation, embedding, and integration: an outline of Normalization Process Theory. Sociology, vol. 43, iss. 3, pp. 535–54,37 copyright © 2009 by SAGE Publications. Reprinted by permission of SAGE Publications, Ltd.

copyright © 2009 by SAGE Publications. Reprinted by permission of SAGE Publications, Ltd.

The theory also encouraged us to consider what it would mean to move from left to right across the diagram. The main point here, we felt, was that new technologies would become progressively embedded in people’s working practices. For example, as we describe in Chapter 7, when electronic whiteboards were introduced at Solo, some staff were initially sceptical about them. On one ward, technological problems led to the whiteboards being withdrawn and later reintroduced, with mixed results. Viewed in the light of normalisation process theory, the initial experience involved limited movement from left to right, and the later experience was one in which different members of staff moved different distances to the right. One would not say that the whiteboards had been fully embedded in local working practices by the end of the observation period. In Trio, in contrast, the whiteboards were fully embedded throughout the observation period; staff frequently used them and had found ways to integrate them into routines.

Biography of Artefacts and practices

We assumed at the start of the study that we would be observing the design, implementation and use of discrete technologies over time. We further assumed that ward whiteboards and board-level dashboards would be part of the same IT systems, with data captured on wards being available to trust managers. However, the findings of the telephone survey and the early observations at the four sites indicated that the situation on the ground was more complicated than we had anticipated. In our early fieldwork, we found that:

1. There was a separation, technologically, between real-time ward management systems – systems that make data immediately available to all users of a system once they have been entered – and the systems used to manage data for management meetings. The ward systems did not, as we had assumed, provide the primary ‘feeds’ for management reports.

2. There was also a distinction between data used on wards and elsewhere, and the IT used to capture, store, manage and present them. The same data were available on different technologies – magnetic whiteboards, digital whiteboards and tablets – across the four trusts.

3. National bodies – including NHS England, the Health and Social Care Information Centre (HSCIC; now known as NHS Digital) and regulators – play an important role in determining what data are captured by trusts and how they are structured and submitted to NHS Digital. That is, it did not make sense to assume that trust informatics teams – and suppliers – had a free hand to develop discrete systems; they had to take account of existing data and IT infrastructures.

Normalisation process theory had proved to be valuable in the design of our fieldwork. We realised, however, that it assumes that a single, discrete technology is being implemented. As the points above imply, this was not the case in this study: there were a number of technologies being implemented. The theory is not suited to the complicated mix of technologies that we found. Moreover, we were dealing with a large-scale technology, and thus groups of people who were engaging with it in very different ways – at ward and board levels – in different parts of an organisation. We therefore needed to identify an analytical framework that allowed us to address our aims and objectives, and that retained the key features of the approach set out in our protocol – but that was suited to the nature of the technologies that we found at the sites.

We considered a range of alternatives, but the Biography of Artefacts approach, developed by Pollock and Williams26 at Edinburgh, was the only one that we felt dealt with the development of large-scale systems over time. The approach stems from the observation that many IT systems in organisations have been implemented over many years. New functions are added periodically, and existing systems are linked both to one another and to new systems. Systems thus develop in piecemeal fashion, producing digital infrastructures that are deeply embedded in the day-to-day work of an organisation.

Pollock and Williams argue that it is simply not possible to study any large-scale organisational technology in its entirety. The pragmatic solution is to observe developments at a number of ‘key points’ where significant things happen, for example meetings in which system developers meet users to discuss the design of a system, or in which ward nurses use systems in the course of their work. The usual field methods are ethnographic, with researchers effectively undertaking ‘mini-ethnographies’ at each of the key points. In this study, we drew particularly on a narrative approach, whereby informants’ stories provide insights into how they are achieving or getting to achieve their aims, and how the narrative unfolds over time. 41 The authors further argue that if we want to understand any IT system in an organisation, and understand why it looks the way it does today, we need to understand its history. The method therefore involves the development of a number of ‘mini-biographies’ based on observations made, over time, at each of the key points. A Biography of Artefacts is, then, made up of a number of mini-biographies. The result is an account of developments from an unusual perspective. For example, there are a number of studies on the use of performance management data by trust board. However, we are not aware of any that, as in the current study, ask how the data get there and why those arrangements have developed, as well as how they are used.

We decided to use the Biography of Artefacts approach in the autumn of 2015. We were committed in our original protocol to understanding developments over time, and the decision to produce mini-biographies did not have a substantial effect on our fieldwork. That said, our realisation that the technologies were more complicated than we had expected led us to expand the scope of our fieldwork in two directions. The first was to ‘go behind the scenes’ to understand how trusts transformed data from a range of systems into consolidated reports for boards and other audiences. The second was to explore what happened to data sets when they were submitted to NHS Digital and sent onwards to other organisations, including the CQC and CCGs.

Key points

• Technologies for handling national data submissions have developed over the last 30 years.

• They were initially developed to manage routine activity data but, more recently, have been used to manage the increasing number of quality and safety data.

• Developments have been piecemeal, resulting in hybrid infrastructures: amalgams of different technologies.

• Current IT arrangements support the submission of data sets to national bodies, rather than supporting trusts to manage their own affairs effectively or share data with one another.

• Some recent policies advocate ‘real-time management’: these, too, envisage national monitoring of the quality and safety of services.

• Nurses’ data and IT needs have been barely visible during the whole period.

Introduction

This chapter discusses the government policies, and the national data and IT infrastructures, that have shaped the development of information systems in the study sites. It became apparent early in the study that national policies had had a very significant influence on the technologies and practices that we were observing. There were, for example, many data sets on the quality and safety of services that trusts were required to submit routinely to NHS Digital and other national bodies.

We present developments in national infrastructures, unfolding over a period of 30 years, albeit with the major changes occurring since the early 2000s. This is consistent with the Biography of Artefacts approach, which we outlined in Chapter 2. Developments are presented in three ‘clusters’, which have emerged at different points in time and are characterised by different ways of thinking about the nature and purposes of data and technology infrastructures. The three clusters are, performance management from the late 1970s onwards; a hybrid of New Public Management-style inspection and advocacy of local governance of quality and safety from 1997 onwards; and ‘real-time management’ since 2012.

First cluster: managerialism, 1979–97

The Conservative administration that came to power in 1979 argued that traditional bureaucracies, including the NHS, were inefficient. They believed that efficiency could be improved by the introduction of a range of measures, some of which were borrowed from private firms, including the use of performance management, the devolution of authority to local managers and the use of market-like mechanisms to commission public services. 42,43 These policies led to the introduction of institutional arrangements that are familiar today, including the separation of commissioning and provision of services. In the context of this study, a key development concerned national data collections. Hospitals submitted a Hospital Episode Statistics (HES) data set and a set of performance indicators (the latter on paper) from 1987 onwards (Professor Paul Aylin, based on evidence paper submitted to Bristol Inquiry, personal communication, Imperial College London, 2015). 44 HES included mortality data, as hospitals had to record all discharges, and death was one category of discharge. HES data sets were initially extracted monthly from hospitals’ Patient Administration Systems (PAS) onto tapes or compact discs (CDs), which were sent (by post or courier) to Regional Health Authorities. 45

There was one other policy that led to the routine collection of data on quality in this period: the Patient’s Charter of 1991. This introduced a maximum waiting time target of 18 months from GP referral to hospital admission. Hospitals were required to report the number of patients who were waiting more than 18 months.

The first significant national IT infrastructure investments were made in the 1990s, prompted in part by the perceived need to move large data sets between locations for contract monitoring in the then-new NHS internal market. 46 Contracts for a national internal network, NHSnet, were signed in 1994. From 1996, hospitals submitted data sets to Regional Health Authorities electronically via a national service, the NHS-wide Clearing Service, called ClearNET.

Second cluster: centralising performance management, 1999 onwards

A Labour government was elected in 1997. From 1999 onwards it decided to pursue managerial policies, continuing in the broad direction set by the previous Conservative administration. At the same time, policy-makers became concerned about the quality and safety of NHS services, prompted by official reports that highlighted far higher hospital mortality rates than anyone had realised, as well as a scandal in cardiac surgery services at Bristol Royal Infirmary. 1,2,47,48

There were a number of policy responses, notably in the creation of national bodies, that would need data on trusts to do their work. Two new bodies were announced in The New NHS:49 the National Institute for Clinical Excellence and the Commission for Healthcare Inspection (later renamed the Healthcare Commission and then the CQC). The latter was to be responsible for governance of the quality of services in NHS organisations. These organisations were subsequently joined by Monitor, which started its work in 2004, as the regulator of the (then) new foundation hospitals. 50

The National Patient Safety Agency was created in 2001; NHS organisations were required to report clinical incidents to it. There was a substantial overhaul of professional regulation, which, among other things, led to a revised system for the licensing of doctors and a National Clinical Assessment Authority for investigating poor performance. The net effect of policies in this period was twofold: a step change in the number of centrally mandated data ‘returns’ in the years after 1997, and the creation of new agencies that would use them (Table 4).

Third cluster: real-time governance

At the same time as later ‘second cluster’ policies were published, the government published policies with two distinctive characteristics. The most important of these is the Five Year Forward View,77 published in 2014. Rather than emphasising competition, performance management and other managerial policies, the Five Year Forward View talks of the NHS as a ‘social movement’, and of the importance of engaging with local communities, with organisations in a locality working closely together. It talks, too, of the devolution of authority to groups of organisations working together, with the freedom to develop new working practices appropriate to the populations they serve. This implied distinctive data flows between organisations rather than upwards from trusts to national bodies. Subsequently, in early 2016, the Secretary of State for Health announced IT investments totalling £4.2B, with a significant proportion of this to be used to ensure that systems could share data with one another. 78,79

A second new theme concerned the use of clinical data in ‘real time’: using data on services while memories of those services are still fresh, as opposed to using them for retrospective reviews weeks or months after the activities described. Thus, the second Francis report,5 introduced earlier, stated that:

All healthcare provider organisations, in conjunction with their healthcare professionals, should develop and maintain systems which give them:

1. Effective real-time information on the performance of each of their services against patient safety and minimum quality standards;

2. Effective real-time information of the performance of each of their consultants and specialist teams in relation to mortality, morbidity, outcome and patient satisfaction.

Recommendation 262, p. 113,5

© Crown copyright. Contains public sector information licensed under the Open Government Licence v3.0

Similarly, a new IT policy, Personalised Health and Care 2020,80 proposed that all patient records would be real-time and interoperable by 2020.

Finally, the government published a clinical utilisation review in 2015. 81 This argued that NHS trusts should implement IT infrastructure for collecting and analysing real-time vital signs data. These data would, it was envisaged, be made available to NHS England in real time (i.e. presumably within a few hours). Few details were given about the ways in which data would be used, but the text implied that managers at NHS England would be able to monitor the work of nurses and doctors in the course of a shift.

Discussion

This brief account emphasises the long period over which the current infrastructures have developed. In the main, the infrastructures that have been implemented support centralisation: the flow of data sets from trusts to NHS Digital and other bodies. This has happened in spite of the fact that policy-making for data and policy-making for IT have been largely separate throughout the last 30 years. In later chapters we will explore the nature and extent of the overlap between the two, in practice, in trusts.

The nature of the third cluster, with its emphasis on the use of real-time data, is not yet clear: there is not enough policy or experience to date. It appears, however, to have some of the characteristics of Digital Era Governance, wherein IT systems are integrated and used to enable staff in different services – and different trusts – to co-ordinate their work. 82 This contrasts with the centralising tendencies of developments in earlier periods. We will also explore the extent of the moves towards the use of real-time data in trusts.

Key points

• This phase of the study comprised a telephone survey of 15 acute NHS trusts and a survey of the content of board papers of all trusts in England in 2014.

• The telephone survey revealed that two sites in our region were already using electronic ward whiteboards.

• The board paper survey showed that all acute NHS trusts used dashboards to present data on the quality and safety of services.

• The surveys, taken together, indicated that national bodies had a substantial influence on the data used by boards: the boards submitted the data collected to national bodies.

Introduction

This chapter presents the findings of a telephone survey of senior nurses, undertaken between September and November 2014, in 15 NHS acute hospital trusts in northern England. We also surveyed the routine quality and safety data presented in papers tabled for board meetings in all 156 acute trusts across England in January 2015.

Telephone survey

Dashboards were used by all 15 of the trusts interviewed and were described as a graphical means of displaying data on the performance of wards and departments, and, more broadly, as key components of trust management information systems. Most participants told us that dashboard information was collated manually from a number of operational systems, including PAS and Datix, the NHS system for reporting patient safety incidents. Reports were typically assembled manually by central performance management teams, and retrospective reports were circulated monthly to wards, directorates and trust boards. The assembly of board reports was easier in two trusts that had real-time data capture and reporting systems.

Board paper survey

These comments, and the comments about regulators, prompted us to review the papers for board meetings at all 156 acute trusts in England for January 2015, in the week beginning 16 March 2015. One hundred and fifty-two trusts published full sets of board papers, consistent with post-Francis reporting requirements. We requested papers from the other four trusts, two of which sent them by return and two of which did not send them within 4 weeks of our request. Almost all trusts (152 out of 154 published sets of papers) had papers on waiting times, the NHS Safety Thermometer, serious incidents, complaints and patient experience measures. Almost all (153 out of 154) had dashboard-based workforce reports. A total of 115 out of 154 trusts reported risk assessment framework data, and 58 out of 154 presented balanced scorecards. Eight out of 156 trusts reported data items that were not requested or required by any national body.

Discussion

We found that board-level dashboards are integral parts of acute NHS trusts’ management information systems. All acute NHS trusts used dashboards to summarise information for board-level meetings. The evidence suggests that trusts are working within a centralised NHS reporting system, using data for retrospective review.

In most of the 15 trusts, in contrast, the overall picture was of ward managers receiving limited, retrospective management reports. They were not in a position to manage risks in real time in the manner envisaged by Berwick and others. That said, two trusts told us that they had deployed real-time ward management systems and were able to manage quality and safety risks proactively in this way. Four further trusts told us that they had plans to implement such systems. The fact that ward systems had already been implemented influenced our thinking that it would be most valuable to observe, in the main, fieldwork at four acute hospital trusts. We described the effect of this decision on our sampling strategy in Chapter 2.

The surveys provided initial indications of the developing governance arrangements and shed some light on our aims and objectives. They suggested that authority was exercised in two ways. First, there was authority to determine what data were collected and reported, and the findings indicate that national bodies determine the majority of data items reported at trust board level. Viewed in historical context, this is not surprising. Ever since they were introduced in the late 1980s, performance indicators have been designed and managed by national bodies. The authority to define the performance framework still resides with national bodies. This point is consistent with the account of national data and technology infrastructures in Chapter 3.

Second, we found that board members receive a great deal of detailed data on wards and departments, presented in a wide range of graphical formats. There was general agreement that dashboards were valuable, both for trust boards in monitoring performance and for non-executives and governors holding boards to account. Some weaknesses of dashboard-based reporting were also noted, including the risk that aggregation of data masked important information about outliers.

Finally, the importance of IT infrastructure in capturing and making quality and safety data available to managers and ward teams, is not stressed by either Francis or Berwick, but is highlighted by these findings. This point also influenced our thinking about the fieldwork that we were to undertake in four trusts in phase 2; we felt that we should investigate the nature and role of IT infrastructures for handling quality and safety data in trusts.

Key points

• We identify five distinct technology ‘development paths’, concerning data used on wards, national data submissions, data processing systems, real-time ward management systems and infrastructures.

• The four trusts were at different points on a technology development trajectory, progressively integrating previously separate technologies.

• The overall performance of all four trusts, judged on a range of quality and safety metrics, improved in the course of the study.

Introduction

The four mini-biographies in Chapters 6–10 describe the artefacts and practices in different areas of four acute NHS trusts, and at national and local agencies. In drafting the mini-biographies, we came to realise that, although the Biography of Artefacts approach has analytical advantages, the artefacts – the technologies – as a whole can be difficult to ‘see’. It is easy to get lost in the detail. Accordingly, this chapter provides an overview of developments at the trusts, designed to make it easier to make sense of the accounts in the following chapters. We present material that would usually be found in the final chapter of a report, but we feel that it is more helpful to present it here. The overview comes in two parts. The first sets out five developments in data and in IT that we observed across the mini-biographies. The trusts were at different stages on a development path, progressively integrating previously discrete technologies. The second provides comments on the performance of the four trusts between 2014 and 2016: these data provide useful context for the observations and comments in the following chapters.

Four data and information technology development paths

The main aim of the study, set out in Chapter 1, was to establish whether or not ward teams in acute NHS trusts have the information systems they need to manage their own work and to report on that work to trust boards and other stakeholders. To address the aim, we needed to know what information systems they were using and how they were using them. This section outlines four distinct but related technologies that we observed. As we will see in later chapters, these have gradually been drawn together to create the information infrastructures that are the focus of this study.

The four trusts: performance

This section presents summary data on the performance of the four trusts. The data were obtained from a number of trust and national sources. It is worth noting that some data were difficult to locate – even for us – and difficult to interpret. For example, data on the same topic, such as serious incidents, could be reported in slightly different ways in board papers and Quality Accounts, due in part to the use of different cut-off dates for counting them. This point aside, however, the data show that, overall, the four trusts showed improvements in key national indicators during the three financial years of 2013–14, 2014–15 and 2015–16.

Key points

• Two wards were observed in each of the four trusts.

• There were observations of the use of whiteboards during shifts, of handovers, safety huddles and of ward meetings, and interviews with a range of staff.

• The four trusts were on a ‘technological trajectory’, with Quartet the least automated and Trio and Solo the most.

• The introduction of electronic whiteboards and mobile devices went smoothly on some wards, and far less so on others.

• Broadly, the move to digital technologies was viewed positively, although there were some dissenting voices.

Introduction

This chapter presents the ward mini-biographies, the first of four mini-biographies of developments at the four trusts. Developments at each of the trusts are presented separately, starting with the trust that had not introduced real-time systems at the start of the study, and ending with the trusts that had. The mini-biography presents evidence from our observations of the use of whiteboards, of handovers, of observations and interviews concerning observations of ward meetings, and comments on information culture on the wards more broadly.

Quartet

Fieldwork was undertaken on two wards between July 2015 and July 2016. Ward A had the same ward sister throughout, while Ward B had a new ward sister half way through our fieldwork. Staffing establishments on both day and night shifts deteriorated on both wards between June 2015 and May 2016. In September 2016, the proportion of registered nursing staff filled as planned during the day was around 80%. The results of the Friends and Family Test showed that > 99% of patients would recommend the two wards’ services. During the observation period, in 2016, the trust introduced a new application, the Electronic Health Record.

Duo

Fieldwork was undertaken on two wards between July 2015 and July 2016. Both wards had the same ward sisters throughout our observations. In September 2016, the percentage of registered nursing staff filled as planned during the day was around 90%. Ward C consistently performed well on its nursing metrics (100%), whereas Ward D was seeking to improve its scores throughout the period. Results from the Friends and Family Test showed that > 91% of patients would recommend the wards’ services.

Duo was one of the three sites that was developing real-time patient management systems, largely in-house. Our interest was in the use of these systems for managing wards. Both wards introduced electronic whiteboards before our fieldwork began in 2015. The whiteboards presented summary patient data in rows for each patient on the ward, including their name, their consultant, estimated discharge date, job lists and risk assessment data (covering falls, infections and pressure ulcers). The latter indicated both a patient’s risks and whether or not their latest timetabled assessment had been completed. Patient data could be input from any terminal or mobile device on the ward and were available almost immediately on the whiteboards. Equally, mobile technologies were deployed on the two wards during the period of observation, providing the best opportunity to observe experiences on the wards. Different functions were introduced on each ward, and technological developments on each one are presented separately below.

Trio

Fieldwork was undertaken on two wards between July 2015 and July 2016. Ward E had the same ward sister during our fieldwork, while Ward F had an acting deputy ward sister and a new ward sister in post near the end of our study. In September 2016, the percentage of registered nursing staff filled as planned during the day was between 87% and 98%. The results of the Friends and Family Test showed that > 97% of patients would recommend the two wards’ services.

Solo

Fieldwork was undertaken in two wards during a 6-month period, from February 2016 to July 2016. The sister on Ward G was relatively new to post, whereas the sister in Ward H had been in post for several years. In September 2016, the proportion of registered nursing staff filled as planned during the day was 96–98%. Ward G is classed as an enhanced supervision ward. The results of the Friends and Family Test showed that 93–97% of patients would recommend the two wards’ services.

Commentary

The four site accounts between them present a ‘direction of travel’ in the introduction of electronic whiteboards and mobile devices on wards. The account of experiences at Duo emphasises that the deployment of new technologies, although often anything but straightforward, can be achieved. Indeed, the systems at Solo and Trio, and to an extent at Duo, were successfully embedded in wards’ working practices.

The technologies have, in general, been viewed positively in relation to the paper and dry-erase technologies that they are replacing. We have found evidence that, in general, nursing staff are willing and able to engage with new technologies and value them. They are willing and able to capture data electronically and use them in the course of their work.

We were struck by the stability of working practices on the wards, notably in handovers and, where we were able to observe them, patient safety huddles. Across the trusts, similar data were used in handovers and huddles throughout the period of observation. Put another way, it appears that the new technologies did not disrupt these practices when deployment went smoothly. Conversely, when deployment went less well, notably on one ward at Solo, there were suggestions that there might be risks to patients’ care.

The site accounts suggest that the move to a paperless NHS – a current NHS England IT objective – is not a straightforward matter. It does not simply involve the substitution of paper with digital media. Rather, the move to digital media has been gradual. It has involved the incremental development of information infrastructures, with different components of the infrastructures often partially replacing paper.

Finally, the site accounts provide some evidence on the impact of technologies on the quality and safety of services. We found a range of views about the systems, with a number of strongly positive statements but also misgivings, the latter notably on the part of some junior doctors. Thus, ‘successfully embedded’ is not quite the same as ‘universal support’ for the new technologies.

Key points

• The trusts devoted considerable resources to capturing data and preparing data sets for national and local agencies.

• They used data required by national agencies as the basis for internal reporting of quality and safety.

• Other types of data were increasingly used in reports; notably, ‘raw’ mortality data were used instead of national indicators.

• Informatics teams at trusts that had successfully deployed electronic whiteboards and mobile devices on wards were able to give detailed accounts of the processes from their perspectives.

• The IT developments on wards were part of much broader developments in trust IT infrastructures.

• There were distinct development paths for data and IT systems, and for real-time and management IT systems.

Introduction

This chapter focuses on the work of the information and informatics teams in the four trusts, and thus on the work of staff who rarely appear in policy documents or in accounts of developments in the applied health services research literature. Theirs are the ‘secret histories’ referred to in Chapter 1. As we explained in Chapters 3 and 5, a number of distinct development paths have come together to create information infrastructures in acute NHS trusts. In this chapter we explore the infrastructures in more detail.

Information teams and data warehouses describes the development of board reports and data sets for submission to external bodies. Development of board reports and data sets sets out the processes involved in managing the four types of quality and safety data that we listed in Chapter 2: incidents, complaints, the NHS Safety Thermometer and mortality data. The reporting arrangements were substantially determined by national bodies for reasons set out in Chapter 3 and, as a result, the experiences of the sites were broadly similar. Finally, we describe the development of the IT infrastructures used to support ‘live’ ward systems and ‘offline’ systems for managing data sets for board reports and national submissions.

Information teams and data warehouses

This section and the following one (Development of board reports and data sets) describe the work of trust information teams and the technologies that they used to generate data sets for board meetings and for submission to external agencies. We found that information teams were organised by function – indeed, effectively by data set - so that there were individuals or small teams responsible for mortality data, for generating reports from Datix and so on. They managed data sets once they had been captured in wards and departments. This involved three main activities. The first was to validate data, a task undertaken in collaboration with nursing and other staff who were responsible for recording them. The second was to use data to prepare graphs and tables for board and other meetings. The data were typically presented as trends, so that each new month’s data were added as they became available. The third was to transfer – sometimes re-enter – data from trusts into national systems. Information teams were, then, the ‘secret’ mechanisms by which data captured in wards and departments were transformed for use by boards and external agencies.

To undertake these activities, information teams relied on data warehouses where the data sets were held. These, as the name implies, were computer servers where data sets that were used for management purposes were held. Historically, data sets were held in separate computer servers and were often managed by their own dedicated teams. This arrangement had increasingly made less sense in the period before the study, when trusts were expected to produce management reports each month using a wide range of data. Throughout the study, period subsets of data were transferred from ‘live’ systems into the warehouses:

So what changed? Some of it was perhaps the creation of a data warehouse. I think we have a really robust data warehouse where transactional data is fed in to the data warehouse every 15 minutes . . .

Solo, senior informatics lead

Development of board reports and data sets

We outlined the development of national data and IT infrastructures in Chapter 3. We noted that a number of national policies and official reports, including the second Francis report, advocated more effective board oversight of the quality and safety of services. All four trusts began to develop reports on the quality and safety of services about 4 years ago, between 2012 and 2013. Initially, monthly reports were modest, comprising key indicators set out in a few sides of A4 paper.

Developments were influenced by historical experiences. One of these concerned the tendency of clinical staff to be sceptical about the quality of data. Solo and Duo both stressed the importance, to them, of developing a ‘single source of truth’, authoritative statements about the performance of the trusts each month, presented in a consolidated set of graphs, charts and tables. At Solo:

I would say also, really important, was the data quality, because for quite a long time . . . you’d end up showing a clinician some data and they would end up saying that’s wrong . . . I can remember having a discussion with a gynaecologist and looking at her data and it said that she’d extracted a wisdom tooth.

Solo, senior informatics lead

Board reports, and the dashboards they included, increased substantially in length between 2013 and 2016; the trends in the contents of reports are discussed in Chapter 8 and details are provided in Appendix 7. Viewed from the perspective of the teams that prepared reports, there have been a number of influences on the development paths at the trusts. Several of our interviewees noted that early reports had presented data that were being collected for national submission, for example NHS Safety Thermometer data. Trio told us that it initially reported Patient At Risk scores, and then in 2013 replaced them with NEWS, when this was introduced as a standard national indicator. There had been experimentation with presentation of data along the way:

When we started [collating data], a few years ago now, we SPC’d everything [used a Statistical Process Control method], and it almost seemed like overkill, so we’ve tried to move back away from that . . . I think it’s just getting the balance.

Solo, informatics lead

Once trusts were producing longer reports, the number of data led some of our interviewees to think about their purposes. For example:

You had to plough through a report which was quite lengthy to find out what are the real issues, where now we, our prime aim is this is what we need to tell you about, these are areas where we feel vulnerable or are concerned about. So we pinpoint [issues for] the reader.

Quartet, non-executive director

More positively:

. . . they don’t read all 92 pages, and that is not the intention. The intention is to say actually I hope that 90 plus percent of what you might need to know – and might being the important word – is here. So . . . you then have got an easy place to access that information.

Solo, senior informatics lead

Another view was that reports should set out narratives that executive directors and non-executive directors (NEDs) could follow easily in meetings and use to highlight issues that needed to be addressed:

I think it’s about a story. So the board could receive a report about 18-week performance . . . saying this is how we’re performing for 18 weeks . . . it’s about typing of letters efficiently, about how the admin [administration] teams are booking things effectively. You know what I mean? The whole shebang in terms of how we deliver on that performance indicator.

Trio, senior nurse manager

We then walk through the business of that unit in terms of patient experience, so what have you shown in your audits of Friends and Family Test, your hygiene audits, hand washing, whatever that may be. Any complaints, what have we learned, what are the issues?

Quartet, NED

Interviewees also emphasised the growth, over the last 3 years, in the number of reports that they produced on a weekly or monthly basis. The details varied between sites, but a typical list included monthly board reports, monthly reports to directorates and reports for weekly performance meetings. There were also reports on specific topics, for example mortality (see Mortality).

Publication and external relationships

All of the trusts produced a Quality Account each year (mandated by NHS England). They also published board papers, and a range of other documents, for example in ‘open and honest’ sections on their web sites. The majority of the data that left trusts, however, were sent to NHS Digital and other bodies. Our interviewees stressed the number and diversity of submissions, and the extent of external scrutiny, from NHS England, CQC, NHS Improvement, CCGs and Commissioning Support Units. This created work for trust teams:

They [the Commissioning Support Unit] have a large team, whereas at the moment we’ve got three analysts doing all the SUS [Secondary Uses Service data sets], all the national returns, all the waiting lists, and then we get these new data sets put on us as well . . . they’ll challenge that, we’ll go into it and we’ll have to go back to them and give a reason as to why. I mean some of them . . . there’ll be multiple patient spells on the same day. But the problem might be that the other in-patient spell is at another trust. Which is fine, they can challenge it, but we can’t see their data.

Trio, information team

Similarly, they were required to submit almost identical data sets separately to NHS Digital (formerly HSCIC) and Public Health England:

Whenever we put that into an e-mail [to Public Health England] and say, ‘Can you not go to HSCIC . . .’ they say ‘No’. It has to be in a specific format which they need.

Trio, informatics lead

Trusts also send data sets to CCGs, to monitor local contractual agreements [Commissioning for Quality and Innovation (CQUIN)] and involve payments for achievement of agreed quality targets. Trusts cannot legally end personally identifiable patient data to CCGs. Thus, the reports contain aggregated data, with sensitive – personally identifying – data, such as details of very rare conditions, removed. More information on the data and relationships is set out in Chapter 10.

The situation today: pros and cons

Our evidence suggests that the arrangements for producing routine reports are well established in the four trusts. The advantages are captured in Box 1, which sets out an account of the annual cycle of indicator selection at Trio. It conveys the point that trusts are able to review, and change, the indicators that they use to manage the quality and safety of services. They are able to use a wider range of measures than those submitted to NHS Digital, and there was a widespread view that the management and use of data were improving. Thus, one interviewee observed that:

. . . we’re at the point that the data is meaningful, and it’s trusted, and I think that just allows you to go to a different level. So you could have the technical infrastructure and be able to do all the whizzy reporting that you wanted to do, but actually if your data and your information isn’t believed and owned then you can’t move on.

Solo, senior informatics lead

Less positively, however, there was a feeling that hard-working staff were trapped in a poorly designed system:

Once everyone has met that target and proved that processes are done . . . you can move onto the next one, but what tends to happen is they just grow and grow and grow.

Duo, informatics lead

There will be a national working group that’s decided . . . that we must have a three side form of VTE requirement assessments done, and that will be signed off and become national policy . . . And everybody will say ‘It’s just a minute’ . . . but actually there’s 17 of those and17 of those becomes 17 minutes.

Duo, informatics lead

There were also a number of comments about the time staff had to spend on verifying data and on producing reports. For example:

. . . out of the 20 days in a month which a person works . . . 18 of those days at the moment are about data verification . . . now that’s coming down, and gradually we’ve got to get that down to 3 or 4 days.

Duo, informatics lead

Incident reporting

We turn now to the first of four indicators that we monitored during the study. All four trusts used Datix software to record incidents and the actions taken following them, and had used it for different lengths of time. For example, Trio had used it since the early 2000s, while Quartet had used it since 2013. They have all moved from paper to PC-based recording of incidents:

We used to use a paper incident form for recording of incidents so everything was on paper . . . we made the decision, a number of years ago now, to scrap paper . . . papers were sat on desks for weeks and weeks and weeks.

Trio, senior nurse manager

Quartet had used another IT system before purchasing Datix:

Before that we had our first electronic incident reporting system called [brand name], and it was such a detriment to staff, it was slow, it was time consuming. Since we got rid of [brand name] and replaced it with Datix our incidents have gone up 50%, if not a little bit more . . . when we were on [brand name] people, staff, would phone us and say ‘it’s crashed and I’m not going to do it’ . . . I think that they thought it was going into a black hole, and nothing was done with the information.

Quartet, information team

Datix offers a range of modules. The trusts used different combinations of these, but all used the incidents and complaints modules. The members of information teams who used Datix were positive about it.

Learning from incidents

The details of arrangements for reporting back to the team involved in an incident varied between trusts. In Duo and Quartet, once a case was closed, the person who reported the incident received an e-mail containing details about the investigation and an agreed action plan. Relevant managers also received feedback in the form of a report generated via Datix. At Solo and Duo, this function was added in 2015 which allowed the risk management team to automatically compile a report that described the investigation and the action plan. Prior to December 2015, the risk management team had to e-mail ward managers to provide feedback.

When asked if they believed that they were still under-reporting, a respondent at Quartet said that she felt that they may be missing ‘a handful’ a year, against a total of several thousand recorded:

. . . I think that governance is quite high on our agenda anyway. And we have trigger lists in certain areas as well just to encourage reporting, so one area was underreporting . . . and CQC came in and did say that [a clinical service] was under-reporting, so we actually made a trigger list and said that . . . these kind of incidents, you should be reporting these, harm or not.

Quartet, information team

Interviewees believed that their trusts were committed to learning from incidents. For example:

We used to wait until a claim was settled and then do an action plan . . . It’s settled the consultant left and retired and is now living in Australia . . . so what we do now is that we learn from the very beginning.

Trio, quality lead

Complaints

Across the trusts, the arrangements for the initial handling of complaints did not change during the course of the study. Patients submitted comments, concerns, compliments and complaints, typically by letter, to the chief executive or another member of staff. All complaints were dealt with initially by a trust complaints team. The teams are typically colocated with, or part of, teams that also manage other patient-facing activities, for example volunteering and the administration of the Friends and Family Test.

Once the team received a complaint, it assessed its seriousness. It was rated as high or low risk, or high, medium or low risk (to the trust), depending on the trust (these ratings refer to the same thing; trusts can use either). The seriousness determined how it was handled. Sometimes the view was taken that the issues raised could be dealt with straightforwardly and a response was developed with the relevant clinical team. On other occasions, in contrast:

Where there are clinical issues that are raised where you would be worried about the implications, and really feel that we need to get on top of it as an organisation, and make sure additional harm isn’t going to be delivered to other patients, this kind of thing, it goes upstairs to a quality meeting. They then do an overview and they will advise [the directorate] in terms of does an investigation need to be undertaken, this needs to be an SUI [a Serious and Untoward Incident], this needs to be a level 2, can we make sure that duty of candour has been followed.

Duo, quality lead

Throughout this process, a complaints handler maintains a detailed record in Datix. A case file tracked the progress to date, including any e-mail correspondence between the complaints team and the directorate, and the names of the people the complaints handler had spoken to. There had been a move away from paper:

It wasn’t that they weren’t computer literate. I think they just all liked the safety of a file but they had mounds of files everywhere.

Trio, informatics lead

If a complaint was deemed to be serious, the complainant would be phoned (at Trio) or would receive a letter acknowledging that the complaint had been received and giving details about the formal complaints process and the expected time scales. The complaint handler would liaise with senior nurse managers to identify a manager, often a matron, to manage the complaint. The manager would then identify an appropriate person, often another matron, to undertake the investigation. After completing the investigation, the investigator drafted a letter to the complainant. The letter was reviewed by a senior trust manager and the manager of the complaints team before it was sent.

Although the administrative arrangements have been stable, the perceived importance of complaints data changed over time:

Complaints is a big one for us . . . before we didn’t really measure how quickly the complaints were turned around and whether it was a formal response required or a telephone conversation . . . [now] we have the turnround time reported, and the themes . . . have we responded in the right way and if not why not, and the themes are they reducing or are they the same themes month on month, and what does that relate back to then? So that’s been a huge turnround for us in terms of the complaint reduction and how our teams are managing complaint responses.

Quartet, information team

One of the things [the chief nurse] was really keen to do right from the get go when she came into post was to improve our complaints responses, and it was all about trying to work on the defensive nature of some of the responses that we were delivering, and trying to change the culture around that. So a whole heap of work has gone on . . . to do things like develop very close relationships with the [directorates], so that we are seen as a helpful supportive function here, to enable them to develop a different style of writing. We do it through complaints master classes, through targeted training . . .

Duo, quality lead

Trust managers monitored the management of complaints closely. The complaints manager would meet regularly – often weekly – with senior managers, typically with the chief nurse, the medical director and a claims manager. Trusts used internal performance measures, such as the numbers of complaints received, the time to resolve complaints and the numbers of complaints reopened after ‘response’ letter was sent. Complaints were always reported on in board papers. At Quartet:

We produce a quarterly learning from experience report, which is a narrative report which pulls the themes from [directorate monthly reports] and it’s about 16 pages but that goes to the commissioners, that goes to [the board] it goes to anywhere anybody wants to access it.

Quartet, information team

NHS Safety Thermometer

The details of data collection varied between the trusts but did not change during the course of the study. For any one ward, a matron or ward manager collects data for the Safety Thermometer survey on one morning in the second week of every month. Some trusts collected data from the case notes of 10 patients, following national guidelines. Others collected it from every patient on the ward, and hence the whole trust, at the time of the survey. Some of the trusts recorded NEWS scores, or the responses to patient experience questions, or both at the same time. In other trusts with electronic whiteboards on all wards, ‘live’ NEWS scores were available to all staff.

At Quartet, for example, the survey data were recorded on paper and then entered into a Microsoft Excel^® (Microsoft Corporation, Redmond, WA, USA) spreadsheet by a member of the information team, who then uploaded the data to SharePoint (Microsoft Corporation, Redmond, WA, USA). A matron checked the spreadsheet for any mistakes or missing data:

We’ve seen a reduction in falls and that’s only because we see them month on month, and we’re questioning about what other actions do you need to put in place. We developed a falls group [with] our director of nursing to make sure it was an area we focussed on specifically. So again we’ve seen a reduction in falls and the way they are managed.

Quartet, information team

A staff member from the informatics team collated all the spreadsheets uploaded to SharePoint. If missing data were found, the spreadsheets were sent back to wards. Data from the spreadsheets were then manually transferred to the UNIFY 2 system (Department of Health and Social Care; NHS Digital), and submitted to NHS Digital. The design of the UNIFY 2 forms meant that data had to be typed in; they could not be copied and pasted. Reports were also sent to board committees and directorates. It was possible, with some effort, to monitor a trust’s performance against other trusts:

. . . we submit that to UNIFY. [A colleague] gets the information from websites, national websites where they publish . . . what has been submitted nationally and then [she] does reports which compare us, as a trust, how we are doing against other trusts for our safety thermometer.

Trio, information team

Mortality

When a patient died, ward staff recorded details on the PAS or the trust electronic patient record system. Data on all episodes, including those that included a death, were then extracted to the trust’s data warehouse. More detailed information was recorded on a form, either on paper or on a tablet.

Once data were available to an information team, extensive checking was undertaken. Corrections were made, for example to rectify an ‘out of range’ number. Clinical coding was also checked, using Healthcare Resource Group grouper software. If episodes were identified as ‘ungrouped’, they were sent to the trust coding team for review and editing, and checked again the following day. Solo told us that it checked patient demographic data against demographic data held on the NHS Spine (NHS Digital) so that it was possible to assign patients to CCGs for contracting purposes. Recorded comorbidities were ‘pulled through’ from operational system to management system and all codes were sent to the discharging consultant to check. Solo had designed a system that presented likely comorbidities associated with the main diagnosis to coders and consultants (Solo, informatics lead).

Mortality rates are a major national NHS indicator and are closely scrutinised by national bodies. All of the trusts told us that they devoted considerable efforts to ensuring that their data were correct and to interpreting them.

Box 2 sets out developments in the ways in which Trio reviews its mortality data. Some of the concerns expressed at Trio were confirmed at other trusts. For example:

I was in a meeting with some clinicians earlier this week where . . . we were talking about the national attribution of activity and if we use Dr Foster’s [data] what shows under each consultant . . . if a patient has been under three consultants nationally it can only show as having been under one . . . that may or may not have been the consultant who did the operation on them . . . clinicians might say, ‘well I do 100 of those a year! . . . and it’s showing me as doing 10!’ Or, ‘I don’t do any of those ever! I don’t know how to do them. And I’ve done five of them according to the data’.

Duo, informatics lead

Ward whiteboards and mobiles: design and deployment

The four trusts were at different stages of development of their systems at the start of the fieldwork, as described in Chapter 5. Solo was able to provide information about developments that had taken place in the 2 years before the start of the fieldwork. Quartet deployed mobile technology in 2014, but this was used primarily to access an EHR system: it did not have electronic whiteboards and tablets during the study period. The text in this section is therefore weighted more to Duo and Trio than to Solo and Quartet.

Solo had long had its own development team:

We have our own in-house development team, so the application is developed in-house and therefore we have massive control over the way that we do data.

Solo, senior informatics lead

As a result, when they decided to develop real-time ward systems, they were able to undertake the work in-house. Trio had an in-house team with a background in pathology systems. At Trio in 2013:

Ours was a portal to start with, it was a front end that accessed a load of other systems, notably to start with pathology systems and diagnostic systems, but then we moved into the ward management areas, and clinical management areas.

Trio, informatics lead

As we took that into the web scenario, we went from a thick client to a web front end, we realised that there were a lot more things we could do, and that all the information we were gathering had a lot more uses.

Trio, informatics lead

At Duo, there was a legacy of a number of separate systems, including management and audit as well as data processing systems, and a pressing need to integrate the data held in them. They also had some in-house development expertise, supported by individual contractors. The need for mobile solutions was recognised, as most clinical staff were mostly ‘on their feet’. It was also felt that too many data were being captured for ‘corporate’ rather than clinical purposes (Duo, IT05). There was also a plethora of forms:

We’re able now to articulate exactly what going paperless means, there’s 30, 40, 50, 60, 70 different forms . . . we’ve got a programme of work around that, that’s going to show the progress as we work through, it feels like we have a model for transformation.

Duo, informatics lead

Once the whiteboards and mobile devices were in use on some wards, informatics teams began to see their potential. At Trio:

It didn’t take long to realise that if we made a system that topographically represented a ward, and you actually put patients into bed, you would revolutionise the way the hospital could work. We didn’t know at the time that we were doing it, but we kind of accidentally did it, we did revolutionise the way the hospital worked, because suddenly, we knew where all the patients were, we knew which consultant they were under, we knew how many beds were free in each ward, and a whole variety of other bits of information . . .

Trio, informatics lead

At Duo:

We’ve always had the concept of a single patient and a multipatient view . . . Mrs Smith was on ward 96 that day, who else was on ward 96 that day? . . . that data was in PAS in a separate system that no clinician, no doctor, no nurse ever logged on to, it was just the secretaries . . . The second you create a common bed number and you say to the nurses, ‘if you just put 1 to 24 in, in that column’, and sort it by bed number, you’ve now got a list of patients sorted by bed number, that will happen automatically in the background for you.

Duo, informatics lead

The sites also highlighted the range of responses staff had to new systems:

. . . we held up [tablets] and said, ‘if you all had one of these, would you be happy? Hands up who would use this’, and actually not many did.

Duo, informatics lead

The first ward was first generation, where we spotted all the flaws, to develop the version which would be more useful to people . . . it was staged, people knew it was coming, so if they were interested they could go to a ward that already had it. When it came they were trained, within a week they all hated it because it was electronic, but then after they actually engaged with it and used it. it was a novelty, so for about 2 weeks they really wanted it, really enjoyed it, and then after that it was the norm, it was what they used.

Trio, informatics lead

Broadly, younger nurses were enthusiastic about systems and older nurses were more wary. Sites used a number of strategies to encourage use, not least in making the systems as simple to use as possible and targeting tasks that were time-consuming and prone to errors for nurses, for example designing systems to calculate NEWS scores automatically. Thus:

All the staff could get involved with it, it would be live information all the time and like you say it wasn’t information that wouldn’t be required by any external agencies or whatever else, but it was pertinent to the trust, or to the ward individually.

Trio, informatics lead

Sites emphasised the importance of senior management and clinical support:

If he had . . . slagged us off, and said why is the trust spending money on our salaries and everything else, we would have sunk, but the fact that he did spend time with us made it work. It comes back to relationships.

Trio, informatics lead

More generally, informatics team support to clinical teams was viewed as crucial:

Normally with IT you ring up, get a ticket, and they get back to you in a few days and it gets fixed. So we tried to do that in a more timely way, if they rang us up with a problem [and] it was a bit more urgent than a screen not working, we would normally go straight out to them, rather than saying we’ll call you back in 3 days or whatever.

Trio, informatics lead

I think it’s great from a clinical user perspective, its brilliant that I as a clinical user can have a conversation with a developer directly, and we can have that kind of one-on-one conversation about what’s going to work for me, what’s going to be useable for me, what do I want out of the system and how I want it to work.

Duo, senior clinician

Development teams worked iteratively with clinical teams on the design of the systems:

I think once that mentality got out there, about the [white]boards, people would come to us saying, ‘could you do this?’, and we would give them a solution, they would take that back to their [management] group, they would agree on taking forward a development, and we would get a request from the board to start that development . . .

Trio, informatics lead

You have to make some quick decisions to change things on the basis of user feedback within a very fast timeframe, which would be challenging within a department where you didn’t have the very specific clinical involvement that we’ve got . . . we have a clinical team who are embedded within the informatics department and that’s been crucial to this happening.

Duo, quality lead

Dual systems: pros and cons

The trusts were also at different stages of integrating their IT systems. From a user perspective, the trusts with electronic whiteboards had two systems, one for real-time management and the other for reporting. In 2016, they were at a point where they could see the advantages of more integrated systems, with screens available to clinical staff fed by the ‘live’ systems, including the ward systems:

. . . we are actually turning up with proper quality metrics around things, and that’s showing the investment that we are putting into the data warehouse . . . we are articulating quality and risk far better than [we were] before . . . we are trying to get to the point where we are providing insight and intelligence, not just data.

Duo, informatics lead

There were also frustrations:

So how can we get the data right at the point of entry . . . what happens at the moment is that we go through this hideous verification exercises where the data comes in, our team tracks it, says it’s wrong, goes back to the clinical teams and says can you fix your data. It comes back and it still isn’t right so it goes through this hideous cycle. The warehouse is now picking up on some of these issues.

Duo, informatics lead

. . . I think there is almost like an information overload now, the information that we have got is incredible now . . . but what you actually do with it is a different thing. You can relay the information, you can produce it on graphs or whatever, but most of the information is designed to be used in real time, it is meant to be ‘now’.

Trio, informatics lead

At Duo, which was piloting tablets on some of its wards, the real-time systems needed high capacity infrastructure:

We underestimated the volume of transactions that would be going on. We, our technical team, had underestimated the volume of transactions because obviously [at our trust] you can have eight to 10 people per ward using it at the same time potentially, and the number of transactions going through it can be significant.

Duo, informatics lead

Looking ahead, interviewees believed that the new systems had great potential:

[An online management report shows that] 60% of your NEWS were done within routine time. Whilst that’s useful to have and useful for us to observe and understand if there’s areas of concern, what I feel that we should be doing more of is looking at the live information and saying actually . . . why has Ward [X] not done their assessments today? Because it’s at that time that you can change your practice to deal with patient safety.

Solo, informatics lead

In a perfect world we’d have statistical modelling going on in the background . . . and going, ‘guys, your SPC [Statistical Process Control] chart’s just wandering off, you haven’t broken your system yet, but you’re about to break the system’. And this place has a long-established track record of not doing that . . .

Duo, informatics lead

ADL [Activities of Daily Living] is not the . . . ideal way to do nursing so [our senior nursing team is developing a] modern way of nursing which is more diagnosis led . . . that in turn [means that you develop] your care plans differently.

Trio, senior nurse manager

Commentary

In this chapter we have traced the developments, over time, of three distinct technologies and their associated practices:

1. data concerning the quality and safety of services, which are captured and managed on a number of different IT systems

2. IT infrastructure for managing data for board and other meetings and for national data submissions

3. IT infrastructure for real-time use in wards, in the forms of electronic whiteboard and mobile devices.

We are struck by the fragmented nature of the systems used to capture and manage quality and safety data for national submissions. There is no obvious rationale for this, beyond national organisations’ requirements that trusts report data separately, using different national systems (SUS, STEIS, Unify). The fragmentation leads, among other things, to multiple recording; for example, a severe pressure ulcer will be recorded in patient records in the Safety Thermometer and as an incident in Datix.

We are also struck by the time costs of trust staff in collecting and managing data for external organisations. As we discuss in Chapter 10, it is not clear why so many data analytics staff are based outside trusts rather than in trust information teams.

More positively, the four trusts had data warehouses that linked the three technologies; most of the data we have discussed were stored in them, including a subset of data from the ward systems. The warehouses are the sources of the data for internal and external reporting. They were beginning to use the data in the warehouses locally, most strikingly to review mortality data 2–3 weeks, rather than 4–5 months, after patients had died. The challenge ahead is to release staff to work on data analysis rather than validation and submission to national bodies.

This mini-biography also reveals two networks of relationships. The first comprised the relationships between information teams and clinical staff. Considerable effort goes into assuring the quality of the data reported internally and externally, and this requires ongoing liaison. The second comprises the relationships between informatics teams and clinical staff during the design and deployment of whiteboards and mobiles. Both of these have been integral to the development we have described.

Key points

• This mini-biography explores change over time of board-level uses, and the value, of dashboards for oversight of the quality and safety of services.

• The overall direction of travel was towards more, and more informative, information systems.

• Dashboards were perceived to be valuable in providing effective oversight and, in particular, in proactively managing organisational risks.

• Conversely, there were also comments in some trusts about the number of data that had to be reviewed.

• Trusts perceived themselves to be more open and transparent in 2016 than in 2013.

Introduction

This chapter presents a mini-biography of board-level practices across the four case study sites. It aims to provide insight into changes over time in the use of two main artefacts: data and dashboards. It focuses on the first three objectives set out in Chapter 1, concerning the design, use and value of information infrastructures.

The mini-biography draws on the following sets of data in the study sites: observations of the board and board-level quality committees; documentary evidence from dashboards, papers and minutes presented at these meetings (June 2015–October 2016) and selected indicators contained in the trust-level dashboards and reports, over the time period from April 2013 to July 2016; and interviews with chief nurses and NEDs from each site at two time points (July 2015, June 2016), and chief medical officers at one time point (July 2016) from two of the sites.

This is a long chapter, and in order to orient readers to the material to follow, we make some introductory remarks here. In the views of the people who we observed and interviewed, all four sites were aiming to enhance oversight of the safety and quality of care, provide assurance on performance (externally and internally) and pursue openness, transparency and the legal duty of candour. They emphasised that they were promoting a cultural shift in reporting. This would be achieved through good governance, the pursuit of harm-free care reliably delivered and the management of risk. Two sites (Solo and Quartet) also emphasised enhancing internal and external organisational accountability, and Duo interviewees placed stress on enhancing organisational performance. The overarching theme in Trio was somewhat different, emphasising promoting harm-free care at the bedside.

Dashboard data content changes over time

Looking across the four sites, the overall direction of travel is towards the provision of more and more detailed data, produced by the information infrastructures described in Chapter 7. Two contrasting narratives are presented first, for Trio and Quartet, which, as we have seen, pursued distinctive technology development paths but had features in common in reporting to board-level meetings.

Fitness for purpose of dashboards for governance change over time

Practices: use and perceived value of dashboard data changes over time

Review of findings for Chapters 6–8

The mini-biography in this chapter set out to provide insight into changes over time in the board-level practices across the study sites. The heart of the study concerns the generation and use of data ‘from ward to board’, and we are now in a position to review the main flows of data within the trusts. Key findings are listed by research question.

Commentary

Across the four sites, there was evidence of perceptions of enhanced governance and board oversight over quality and safety, and productive use of the dashboard and dashboard-derived reports. There was also common agreement that the trusts were, overall, more ‘open and transparent’ than in 2013, and also were making progress in meeting the legal duty of candour. However, informants pointed to the need for more timely feedback from the board to wards, with two sites experimenting with a more devolved management approach in the course of the fieldwork.

The observational and interview data support the interpretation that developments in data available to boards, and their use, form at least part of an explanation for the enhanced governance and oversight capability. Other factors were also important, including the following: in two sites, staff enthusiasm and engagement with IT developments; emphasis on systems to support ward care and its management; impetus from the Francis inquiry for greater openness, transparency and the duty of candour; board members’ safety ward walkarounds; and the development of systems and supporting processes in reporting and responding to incidents and complaints, to ensure the legal duty of candour.

The plot that underlies the narratives of these changes over time provides evidence of a shift from classic performance management to a more supportive approach. Over time, there was increasing emphasis on managing clinical and organisational risks84,85 and an orientation emphasising the quality and safety of services. 86,87

Key points

• This mini-biography explores matrons’ use of data.

• Matrons used information systems for assurance about the quality and safety of services.

• More broadly, matrons used a range of informal processes to manage their services; routine data were just one source of information.

Introduction

This mini-biography explores the use of dashboards and data recorded at ward level in the work of matrons at directorate level in the four sites. It aims to provide insight into changes over time in the use of two main artefacts – data and dashboards – and the issues of openness, transparency and the legal duty of candour. The findings are based on data drawn from 12 observations of matron meetings with their ward sisters (2015 and 2016) and interviews with six matrons (2016). As the data set and time perspective are quite limited, the findings are indicative and should be read and interpreted in the context of, in particular, the ward mini-biography (see Chapter 6). The following sections explore the matrons’ approach and ways of working; the matrons’ use and perceived value of dashboards and other information in their work; and the matrons’ perspectives of changes over time in relation to the issues of openness, transparency and the legal duty of candour.

Overall, the findings point towards an inferred, directorate-level, cross-site aim centred on ‘the delivery of good/high quality and safe patient care’, with a current additional emphasis, explicitly mentioned in one site (Trio) and implicitly in another (Duo): ‘and, now, in terms of money’. This aim was amplified by the informant in Duo: ‘efficient [patient] care with all that encompasses . . . [about] standards’. The indicative cross-site plot being pursued at directorate level is concerned with ‘using the data, gaining assurance and enabling and supporting’. It can be summarised as using information to enable the matron to gain assurance about, and to support ward sisters in, the delivery of safe and high quality patient care.

Matrons’ approach and ways of working

Across sites, the matron responsible for a set of wards or services held monthly matron meetings with their ward sisters. The matron also participated in matron meetings (building on ward reports prepared for her by the ward sisters); one-to-ones with her line manager and/or head of nursing; monthly directorate meetings; and quality assurance, performance and risk meetings with senior managers, during which areas of concern and/or potential risk might be escalated. The matron also validated at least some of the data that ward sisters entered, for example regarding complaints, incidents or NEWS. This aspect of the matron’s work (for further insight, see Chapter 7) was only explicitly mentioned by informants in two sites (Solo and Duo) and only implicitly in another (Quartet). As one matron stated, ‘we have to validate it (NEWS) every month . . . as matrons’ (Solo matron). This was reiterated for Datix and the compassion audit by one of the ward sisters in Duo: ’the compassion audit is done by matrons’ (Duo ward sister).

The matron aimed to hold monthly meetings with the ward sisters, but across sites several meetings were cancelled and/or had variable attendance. Minutes of the meetings were taken and distributed a week or so before the next meeting. The meetings primarily took the form of ward sisters passing on information to the matron; joint discussion of problems and concerns; providing an opportunity to share learning; the matron giving feedback and support to ward sisters; and the matron passing on information from directorate and other meetings, including the board. Data collected throughout the observation period exemplified inter alia:

• ward sisters feeling able to discuss problems or incidents with colleagues and receiving supportive advice (Solo and Quartet)

• using patient stories to promote discussion and encourage reflection on potential learning (observed 13 January 2016, Solo; August 2015, Duo; observed 2 September 2015, Quartet)

• sisters giving feedback and updates and sharing areas of concern for their ward (e.g. ‘outliers’, bed pressures, problems with e-whiteboards) (observed, 13 January 2016, Solo), leading them to resort to paper recording in general (observed 25 February 2016, 9 September 2015, Quartet), to communication problems occurring during patient handover from another ward, and the use of agency staff (observed, 4 August 2015, Duo)

• sharing learning (e.g. serious incident) (observed 4 May 2016, 13 January 2016, Solo; observed 4 August 2015, Duo) or passing on learning from other teams (e.g. the use of electronic rostering systems)

• reflection on ways that e-information was used (observed 4 August 2015, Duo) and tutelage to sisters to encourage, ‘the habit of accessing it’ (Solo matron)

• matron providing positive feedback and ‘praise’ when key performance indicators had improved in the previous month

• matron reminding sisters what they could do ‘better’, for example regarding pressure ulcers, falls and complaints (observed 20 July 2016, Solo); sisters to use e-whiteboards within handovers (observed 4 May 2016, Solo) and in the daily safety briefing (observed 17 February 2016, Solo)

• matron passing on information on new initiatives [e.g. matron visibility ‘Blue Thursday’ (the matron to do a half shift on the ward)] and information passed on via external presentations to the meeting, for example on infection control (observed 8 June 2016, Solo) and deep cleaning (observed 4 August 2015, Duo).

Use and perceived value of dashboards and other information

Openness, transparency and the legal duty of candour

Across sites, compared with 2013–14, there was a general perception that key recommendations made by Sir Robert Francis, concerning openness, transparency and candour, had been taken seriously. In part, this was linked to the greater availability and publication of data, and in part to the increased visibility of members of the executive team in the organisation. Informants reported a changing culture, reinforced by matrons’ professional responsibilities to patients. As informants from Solo, Duo and Trio commented:

I try to be a realist, and just say. This is what it is. So, I don’t like to hear where I’m being told that, ‘well actually, your areas are alright because you’ve got this this this and this.’ I want to reflect accurately . . . I’ve got a duty to not only my patients but my staff to reflect things accurately, that’s how I see it.

Solo, matron

I think personally I have always been very open and transparent I will always say sorry for things because I think that is really important for people to hear that.

Duo, matron

That’s [openness, candour] [are] embedded in our systems and processes now.

Trio, matron

At the same time, an informant from Quartet observed:

We scrutinise ourselves too much . . . But I think we have got to, you know . . . I think we have taken that to gold standard . . . [But] with duty of candour . . . you can’t say your ever too open . . . We’ve got to be honest with patients and family . . . I think the culture is less than desirable sometimes . . . But then there is a bit of blame.

Quartet, matron

Concluding comments

Matrons used real-time systems – in the sites where they were available – and routine reports. These were just one source of information, and matrons also regularly visited wards, there were meetings with ward sisters and/or there were one-to-ones with them. Matrons worked collaboratively with ward staff to address and resolve problems, reflect on lessons learned and effect changes in care delivery. That is, while performance data were available and used, the overall management style was more consensual as well as performance driven. The publication of performance data across wards, in the form of RAG ratings, had the unforeseen (to us) consequence of encouraging competition between wards and spurring staff to improve their metrics.

If we take a step back, and consider this chapter in a wider context, we can make two further points. The first is that, combined with our observations about senior managers ‘managing by walking about’ in Chapter 8, data played a significant role in management but did not replace ‘informal’ sources of information gleaned from face-to-face contacts. The second is that directorates were – for nurses, at least – the places in the trusts where the information systems came together. Matrons used both the real-time and management systems, they passed information up and down their organisations and they integrated formally and informally acquired information in the course of their work.

Key points

• This mini-biography explores the use of trust data by local and national agencies.

• Trust data sets were submitted primarily to NHS Digital, which, in turn, provided data sets to other agencies.

• There was extensive oversight of the performance of trusts, which relied on data sets generated initially by the trusts themselves.

• CCGs and Healthwatch made relatively limited use of these data sets, relying more on information that they gained through local relationships with trusts.

Introduction

This chapter explores the value and use of trust dashboards and data by local service commissioners and Healthwatch, and the ways in which local and national agencies use the data provided to them by NHS Digital (previously the HSCIC). The chapter is in four parts. The first, building on Chapters 3 and 7, provides an overview of the data flows from trusts to national agencies. The second explores the data needs of local agencies, that is, CCGs and Healthwatch, and their use of these data for quality and safety oversight and assurance. The third considers national agencies’ use of data to monitor quality and safety in trusts. The fourth and final section looks at indicative evidence on national agencies’ collaborative working on quality and safety oversight and assurance of trusts.

The findings are based on eight interviews, conducted in July 2016, in five organisations: NHS Improvement (the merged Monitor and TDA, n = 1); CQC (CGC, n = 1); two regional offices of NHS England (NHS England, n = 2); CCG (n = 1); and Healthwatch (n = 3). As the local data set is limited, findings at that level are indicative only.

Data flows from the trusts

Figure 3 is a high-level schematic representation of data flows from trusts. For the set of key indicators we focused on in this study (see Chapter 7), data were submitted by trusts to NHS Digital. Some National Clinical Audit data sets were submitted to Public Health England. NHS Digital, after validating the data sets, provided data sets to other agencies: NHS England, CQC and NHS Improvement. The first two were able, legally, to request specific data sets from NHS Digital, but NHS Improvement could not, and relied on HES data sets. Trusts did not provide patient-level data to CCGs or Healthwatches: this is not permissible under the Health and Social Care Act 2012. As the interviews explored in this chapter show, data do not flow back to trusts as feedback from NHS Improvement or from CQC inspections.

FIGURE 3.

Main data set flows from trusts.

Local agencies and quality and safety data

In pursuit of their commissioning roles, CCGs need data to monitor and assure the quality and safety of the services. Our CCG informant depicted this as:

. . . Understanding the quality impact on service users and to any changes in clinical pathways.

CCG

The CCG was able to use data published by the trust, and the trust also provided workforce dashboards, a nurse ward dashboard-derived report (reporting a number of indicators) and improvement plans for health-care-acquired infections. The CCG then:

. . . Concentrated [its] efforts on [identifying and working with] the right indicators.

CCG

This CCG thus did not ask the trust to provide bespoke reports, but rather:

[It] worked and continue to work on accepting the information that’s been presented to their board . . . We receive a suite of documents . . . based on our forward plan . . . our most useful data is what the trust is delivering to their own board.

CCG

Healthwatches have broad remits, spanning health, social and voluntary, and care homes, and are financed by the local authority using non-ring-fenced centrally provided monies. Their emphasis was on representing the patient’s voice:

[We] basically try and put people at the heart of health and social care.

Healthwatch

Healthwatch was, in essence:

Doing . . . an assurance validation . . . and [we] always publish our report.

Healthwatch

Our informants argued that they added value:

Deliberately bringing more of the services of patients . . . of people’s feedback to what, traditionally, quality and performance committees tended to not put/give a lot of time on their agendas . . . They all looked at their KPIs [key performance indicators] . . .

Healthwatch

. . . The data that people can give us can be so powerful that people [service providers] want to do the change that you’re suggesting. You know, the work that we’ve done with hospitals, with CCG, the doctors – we bring them an issue and we tell them how it’s affecting people and they want to change.

Healthwatch

Patient stories were a key resource. Healthwatch in Duo had a database of individual patient feedback, and Trio conducted an:

Ongoing survey . . . people can go on the website and tell their story.

Healthwatch

As the Solo informant commented, Healthwatch also has:

. . . A strong idea about what are the issues that are frequent and . . . the issues that are severe . . . We [try] to balance the two. So, one person’s story probably isn’t going to be enough . . . But if it is more than that . . .

Healthwatch

In addition, Healthwatch conduct edsurveys on specific issues and also reviewed data on the NHS ‘patient portal’, My NHS, thus triangulating data about local trusts from different sources. Another informant pointed to the need to have strong links with advocacy providers:

So we are picking up the data that they’re finding out about and again it helps with that triangulation.

Healthwatch

Healthwatches also received limited specific data, on request, from their local trusts. Generally, our Healthwatches relied on public board reports and the annual quality accounts, on which they had to comment, and papers for committees they attended, for example quality committees at Solo and Trio. A Solo informant, reflecting on the meetings, identified a key challenge:

How do you sift through the data to actually have some intelligence about what’s happening?

Healthwatch

In addition, in Healthwatch’s preparation for an ‘Enter and View’ visit:

[We] ask [the trust] for every bit of information from the whole directorate or CSU [Commissioning Support Unit], and pick a sample.

Healthwatch

National agency use of data

Although the three national bodies, NHS England, CQC and NHS Improvement, hold slightly different remits, each draws on trust data submitted to NHS Digital. To make sense of their multiple data sources (e.g. NHS England draws on over 30 data sources – NHS England Regional Team), both NHS England and CQC have developed analytical tools to aid their work. NHS England has developed three national dashboards (Acute and Specialist trust quality dashboard, comprising 44 indicators; Mental Health dashboard; and Primary Care dashboard) and a ‘quality risk profile’, while CQC, refining the approach of intelligent monitoring, relies on a comprehensive surveillance model. In part the latter model was perceived as an aide to the provider.

It’s for the provider to understand what it is that we expect from them, and they structure their thinking about quality assessment around it, and so the shared view of quality is in part intended to prompt providers.

CQC

Moreover, the CQC informant continued:

A lot of organisations had constructed their own governance and assurance mechanisms around our indicators . . . I was quite surprised . . . I think that in itself reflects a shift in tone and expectations.

CQC

National agencies’ collaborative working

The core issue was summarised by the CCG informant:

Nobody seems to be agreed nationally, and this is a big issue across all quality surveillance groups. Nobody can define the definition of assurance . . . We’re all using different measures, aren’t we?

CCG

This echoes issues raised in Chapter 2. The informant continued more positively:

[We are] starting to get a collaborative [view].

CCG

Some indications of movement to greater collaboration were provided in the interviews with the national agencies. For example, the NHS Improvement interviewee told us:

We integrating, we’re going in to an integrating phase, and certainly the focus currently is on technology integration.

NHS England regional team

Not only was there movement towards technology integration, but, a NHS England informant indicated that there was also work towards having one data source for quality and a common definition and set of indicators across national agencies:

. . . The other key players around quality . . . the Care Quality Commission and NHS Improvement, we may well try and work towards having just one data source that we will tap into with a common set of indicators . . . There’s a National Quality Board that’s working as a working group around measuring quality . . . On that group is everybody on the system, so HSCIC, NHS Improvement, Department of Health [and Social Care] everybody that’s working around measuring quality in some way . . . They’ll be working towards having a common definition of quality . . . a common set of quality measures. But it’s a long process.

NHS England regional team

Concluding comments

As we saw in Chapter 3, national policies have substantially determined the data sets that acute NHS trusts are required to submit to NHS Digital. Chapter 7 described the considerable effort involved in providing those data sets. In this chapter, we have seen that NHS Digital, in turn, provides data sets to other national agencies and to CCGs. We need to stress that we conducted a relatively small programme of interviews, and were able to interview only one CCG representative.

However, if we put the evidence from the three chapters together, we can make three points. The first is that the data sets are extensively curated after they leave trusts: it seems reasonable to say that NHS information infrastructures extend far beyond them. The second is that the information infrastructures are designed for ‘one way data flows’. Data sets flow out of trusts, but we found only limited examples of data flowing back in, for example in reports after CQC inspections. In principle, national agencies might have produced comparative data sets that would allow trusts to compare themselves with one another, but, as we saw in Chapter 7, these are not produced for the data sets underpinning the NHS Outcomes Framework. National Comparative Audits produce comparative data for a range of clinical conditions, but are managed separately from the data sets of interest here. Third, and finally, the evidence of Chapter 3 and the last five empirical chapters indicates that the four trusts were being extensively performance managed. Healthwatch organisations might demur from this conclusion, but the number of external organisations, and the number of data they review, emphasise the fact that substantial resources are devoted to surveillance rather than, say, to data-driven quality improvement.

Key points

• Ward teams have the data they need to manage their work – but the combinations of IT that they use to manage it vary from trust to trust.

• Trusts believed that they had always been open, or were more open in 2016 than they had been in previous years. They published far more data about the quality and safety of services in 2016 than in 2013.

• Trust boards have the data they need for effective oversight of the quality and safety of services.

• The current data and IT infrastructures are amalgams of technologies.

• The infrastructures support the centralisation of control over the quality and safety of services.

• We have also found evidence for an emerging alternative model, involving the decentralisation of control to ward teams.

• Implications for research and for health care are identified.

Introduction

In this chapter, we discuss our findings. In the next section we address our aims and objectives, and then outline a Biography of Artefacts, pulling together the findings of our surveys and mini-biographies. We then comment on a broader question raised in our proposal, concerning the centralisation or decentralisation of control over decisions about the quality and safety of services. Finally, we set out implications for practice and recommendations for research.

Review of aims and objectives

The main aim of this study was to establish whether or not ward teams in acute NHS trusts have the information systems they need to manage their own work, and to report on that work to trust boards and other stakeholders.

We conclude that ward teams at the four trusts in the main field study had the information systems they needed to manage their own work effectively. Each of the trusts captured and used similar data, but used different combinations of IT to do so. Solo and Trio had already introduced real-time ward management systems at the start of the main fieldwork, and staff had integrated them into their working practices. Duo offered us the opportunity to observe the introduction of mobile devices, some aspects of which went smoothly, while others did not. Quartet was introducing components of recognisable real-time ward management technology towards the end of our observation period.

We assumed, at the start of the study, that data captured in ward systems, and used locally to manage wards, would have a sort of secondary life. They would be used to prepare quantitative reports – presented on dashboards – for board and other trust meetings, and data sets for submission to other organisations. This was not the case. There was a distinction between two broad types of data. One was data used in the routine monitoring of a patient’s status, including vital signs, risk assessments and care plans. These data were attached to patients throughout their stays, and little ‘escaped’ into management reports. They were data that only really made sense in the context of the care of a patient, or the management of a ward on a particular shift. The other types of data were part of the parallel systems that we described in Chapters 5 and 7, including data on mortality, incidents and the NHS Safety Thermometer. These data are counts of adverse events, or of possible adverse events in the case of deaths in hospital, and are data that national bodies believe to be indicators of quality and safety.

Taking the two types of data together, along with the IT systems used to capture and manage them, we can say that wards report comprehensively on their work to boards and to external agencies. The way in which they do this, however – across hybrid infrastructures and involving considerable staff time – raises questions, which we discuss below.

Study design: strengths and weaknesses

We used normalisation process theory to guide the design of the main phase of the fieldwork. 35,36 It proved to be very helpful, particularly in articulating the questions that we needed to ask in the course of our fieldwork. Our view is that using the theory was a strength of the early phase of the field study: it gave us confidence that we were asking an appropriate set of questions about the different phases of technology design and deployment. Equally, as noted in Chapter 2, we found that the theory was not suitable for the technologies that we were studying. This was not a weakness of the theory, but rather a mismatch between the theory and the technologies (information infrastructures) that we were observing.

We used the Biography of Artefacts approach to analyse the data presented in Chapters 6–10. 26 The main strength of the approach is that it provides an effective means of studying large-scale organisational phenomena, where there are important developments in more than one part of an organisation. We found that it was possible to study a complex phenomenon in several different places, across four acute trusts, and arrive at meaningful conclusions. We also found that the narrative, or biographical, approach generated valuable insights about the development of information infrastructures.

To set against this, we found that we had to develop some of the technical features of the approach. We made three main additions to published accounts of the approach, concerning the role of direct observation of working practices (many published biographies rely on a combination of documents and interviews), combining evidence from different sources (observation, interview, documents) and integrating accounts from multiple sites. The net effect of these observations is that we have found the Biography of Artefacts approach very helpful. However, our findings should be interpreted in light of the fact that we were developing aspects of the approach during the course of the study.

More broadly, our use of the Biography of Artefacts approach involves trade offs with other approaches. The approach emphasises the importance of change, over time, in large-scale infrastructures. As a result it de-emphasises detailed features of technologies and working practices that would be highlighted in other studies. For example, a human–computer interaction study would highlight the detailed design of screens in ward systems, or the fine details of the presentation of data in board papers – we have relatively little to say about these important features of systems. Similarly, we might have used practice theory,89 and focused on the ways in which working practices produce particular cultures on wards or in board level meetings. Indeed, a study of this kind could be valuable in a post-Francis NHS context. We would, however, have had to give up our focus on information infrastructures in order to pursue a practice theory-informed study.

Turning to our field methods, our reflections on the interviews is that most of our interviewees were giving us straightforward accounts of developments as they saw them. Interviewees were typically open about difficulties that they were facing. Similarly, we did not encounter any difficulties observing board quality committees or working practices on wards. To set against this, it is clear that the mini-biographies paint a relatively positive picture, particularly in the sites that made substantive progress with their real-time ward management systems. This jars with published accounts of deep-seated problems with quality and safety in acute NHS trusts. We do not have a good explanation for this gap, but we speculate that there might be three reasons. The first is that the study focused principally, if not entirely, on nurses, who may have benefited more from the new technologies than other staff groups. If we had focused more on consultants, or possibly allied health professionals, the tone of the report might have been different.

Second, the three sites with developed information infrastructures were genuinely proud of their achievements. As we comment later, the ward-level developments in particular mark a departure from the IT experiences of the 2000s, described in Chapter 3. Put another way, some credit is due to trusts that have begun to overturn the NHS’ reputation as a very difficult place to deploy IT systems. Third, as the data in Chapter 5 indicate, the sites really did make progress in improving the quality and safety of services during the study period. The evidence of our observations and interviews is consistent with the quantitative indicators. Although the quality of our data, and the routine data, can both be questioned, they do paint a consistent picture. Thus, the perspective of a trust manager is, quite reasonably, that services have been improving. This is not inherently in conflict with the observation that there is clearly a long way to go, on many fronts, before we can say that acute trusts are safe places to be a patient.

A Biography of Artefacts

The mini-biographies set out the direction of developments at the four trusts between 2013 and 2016. Taken together, they portray the development of infrastructures for capturing and using data about the work of acute hospital trusts. Our evidence shows that data and technology infrastructures have developed over many years to support the movement of data from ward to board, and beyond to external agencies. Moreover, the data are of a particular type, about transactions, or what might be termed data for ‘counting and accounting’. Viewed from this perspective, trust data warehouses and board dashboards are developments of established arrangements.

Taking the long view encouraged by the Biography of Artefacts approach, however, real-time ward management systems mark a departure. The departure can be characterised as a step change from data processing to real-time clinical systems, and from transactions – data for ‘counting and accounting’ – to data to actively manage clinical risks. We appear to have been observing ‘interim technologies’, that have a useful shelf life but are eventually superceded. 90 The most interesting example is electronic whiteboards. It seems possible that wards will eventually move to a situation where all staff have a mobile device for capturing and accessing data, and will no longer need whiteboards.

Trust data and IT infrastructures, and the national infrastructures that surround them, embody two distinct assumptions about the nature and purposes of quality and safety data. The first assumption is that top-down performance management is the most effective strategy for improving the quality and safety of services. This was one of the key assumptions made in the second Francis report on Mid Staffordshire NHS Foundation Trust. We conclude that top-down performance management dominates other approaches, such as clinical risk management and quality improvement, in the relationships between external agencies and trusts. However, we have identified a number of reasons to doubt that routine data on quality and safety are being used directly for performance management within trusts.

The second assumption is that the infrastructures support the proactive management of the quality and safety of services, as outlined in Chapters 6 and 7. Two developments suggest that the infrastructures are being used in new ways: they are being harnessed to support local, real-time monitoring of quality and safety. The first, and most marked development is that whiteboards and mobile technologies on wards make it easier to manage clinical risks proactively, that is, to manage patients’ risks rather than manage performance using transaction data. The focus on nurses’ information requirements can be contrasted with the historical focus on systems, including electronic records systems, to support doctors’ decision-making. The real-time systems have been developed largely or entirely by in-house informatics teams, working closely with ward nurses and other clinical staff, supported by trust boards. The relevant data – vital signs, risk assessments and so on – are integral to patient treatment and care.

Second, the four trusts have data warehouses, home to data from a number of ward and departmental systems (or from the integrated clinical system at Solo). These warehouses are environments where teams can undertake analyses on data sets that are ‘almost live’. They can run analyses on data from the last 24 hours, or last week – recent enough for staff to be able to recall the events the data relate to and act on it. The most striking example we came across was the analysis of ‘raw’ mortality data at Trio: the trust used to wait months for national performance indicators, but now analyses its own data much earlier, starting within days of the end of each month. This example hints at the potential currently locked up in trust data warehouses.

The second assumption, and the evidence supporting it, chimes with arguments in substantial academic and practitioner literatures on clinical risk management, quality management and patient safety initiatives. They all emphasise the importance of clinical teams taking responsibility for their services. We can summarise here by saying that trusts have been implementing a version of democratic experimentalism. 91 The term was coined to draw attention to two features of decentralised governance arrangements. They are democratic because they involve negotiation within frontline teams, and between those teams and senior managers. They are experimental in the sense that teams have the freedom to innovate – to improve services. Teams have to provide evidence that they are providing safe and high quality services, and are improving them over time: this can be achieved, in part, by reporting agreed metrics to senior managers. In the ideal version of democratic experimentalism, teams are autonomous and can negotiate the ways in which they will demonstrate their effectiveness. In the actual version we have observed here, in the four trusts, local authority sits within a set of centrally determined reporting arrangements. They have developed in spite of, rather than because of, the central data straitjacket.

It is tempting to believe that the tension embodied in the data and IT infrastructures can be resolved. The development of real-time ward management systems can be encouraged and perhaps increasingly used as sources of data for clinical audit and for management reports. The management of data sets for reporting could be streamlined over time, and the costs associated with managing them reduced. If this is to happen, we think that strategic choices will have to be made.

Logics of control

This line of argument brings us to our final points, concerning the strategic choices about the management of the quality and safety of services. There are three broad options. The first is to continue with the current arrangements. This study provides clear evidence that national bodies have a major influence on the data that trusts capture, use themselves in board reports and submit to NHS Digital and other national bodies. They have also had a major influence on the NHS-wide technology infrastructure, which, intentionally or not, strongly favours upwards reporting rather than the horizontal sharing of data between organisations. It is difficult to imagine why any of the interested parties would wish to continue with the current technological arrangements.

The second option is democratic experimentalism, whereby authority, and responsibility, lies principally with clinical teams. Trusts, as publicly funded bodies, will always be required to account for their work to national bodies, and indeed upwards to parliament. However, they would do so within a much-revised set of governance arrangements, whereby IT investments would be focused on supporting the work of frontline staff, and data for reporting to boards and beyond would be a by-product of frontline data capture (as envisaged in the 1992 Information Management and Technology Strategy, but not implemented before now). This option would recognise the importance of relationships between clinicians and informatics teams, and the fact that information requirements change over time, necessitating ongoing changes to the data captured and reported.

The third option is the information panopticon. A panopticon, a concept sometimes attributed to the nineteenth century philosopher Jeremy Bentham, is a prison with cells arranged around a central well, from which prisoners can be observed at all times. Zuboff92 extended this concept, arguing that IT could be used as information panopticons to gaze down on workers – clinical staff, in our case. The NHS policies Personalised Health and Care 2020,93 and the clinical utilisation review,94 reflect this line of thought. They envisage a NHS in which real-time data, presumably data about today’s or, at worst, yesterday’s services, continually feeds a central ‘brain’, which is able to monitor services. There have been a number of such centralised initiatives in the past, including attempts to manage wars from a continent away,95 to manage nuclear missile silos across the USA from a central command96 and the CyberSyn experiment in Chile in the early 1970s,97 where all industrial production was to be managed centrally. These experiences were not, it has to be said, encouraging. The fact that data were made available to central command in real-time – perhaps a few hours later, or the next day – disguises the point that the data were ‘stripped down’, removed from the context in which they were captured and hence much more difficult to interpret than they would be at ground level. In the NHS, the evidence of a long series of official inquiries, including the second Francis report, does not suggest that national agencies respond rapidly, or sometimes at all, even when there is clear evidence of unsafe practices.

It is not for us to say which of these options should be pursued. We feel that we can, however, point to the considerable progress made by the four trusts. The second and third options would build on the important move to real-time management of wards, but in very different ways, involving different technology infrastructures and radically different governance arrangements.

Implications for health care

1. Real-time ward management systems have been developed largely in-house, using agile methods and with ward nurses closely involved. They mark a significant departure in thinking and practice from the NHS’s historical reliance on commercially available data processing systems.

2. The trusts are acutely aware of the potential locked in the data sets in their data warehouses, and in their informatics and information teams. Their capacity to exploit the potential is currently very limited: most of their time is committed to preparing national data submissions. Trusts need to be able to free up staff time if they are to achieve data-driven quality improvements.

3. The development of real-time management systems presents the NHS with a strategic choice. Will sustained, substantive quality and safety improvements be achieved by centralising authority, and hence the flow of data, to boards and to external agencies? Or will they be achieved by the clinical teams caring for patients, supported by real-time management systems? The decision will have a major effect on the future development of these critical infrastructures.

Recommendations for research

1. The growth in the use of mobile technologies on wards for the management of clinical risks, as well as for recording patients’ status and treatment, may have an effect on the quality and safety of services. These effectiveness need to be established.

2. Similarly, there has been a significant growth in the use of electronic whiteboards on acute wards in the last 3 years. This study raises the question: are these interim technologies that will disappear when mobile devices are ubiquitous, or do they have an important role to play in monitoring the quality and safety of services?

3. National data submissions have developed in piecemeal fashion during the last 2 decades. A number of reports have drawn attention to the time that clinicians spend recording data or searching for it, but this is the first study that has highlighted the opportunity costs of managing data for national submissions. The overall design of national data submissions, and their costs and value, merit review.

4. Acute trusts now have data warehouses, which appear to have considerable potential to support analyses of current performance and modelling options for service improvements. The scope for greater local exploitation of these data sets needs to be evaluated.

Contributions of authors

Justin Keen (Professor of Health Politics, University of Leeds) led the project, was the first author of the final report and participated in all aspects of the research.

Emma Nicklin (Research Assistant, University of Leeds) contributed to the data collection, analysis and the writing of the final report.

Andrew Long (Professor in Health Systems Research, University of Leeds) contributed to the study design, data collection, analysis and the writing of the final report.

Rebecca Randell (Senior Research Fellow in Health Informatics, University of Leeds) contributed to the study design, overall conduct of the study and drafting of the final report.

Nyantara Wickramasekera (Research Assistant, University of Sheffield) contributed to data collection, analysis and report writing.

Cara Gates (Research Assistant, Leeds Beckett University) contributed to data collection, analysis and report writing.

Claire Ginn (Senior Manager, Analytical Services, NHS England) was a contributing member of the project team, which met bimonthly to review and advise on the study design, report writing and study dissemination outputs. She also helped to run the PPI meetings.

Elizabeth McGinnis (Clinical Co-ordinator, University of Leeds) was a contributing member of the project team, which met bimonthly to review and advise on the study design, report writing and study dissemination outputs. She also to helped run the PPI meetings.

Sean Willis (Senior Nurse, Leeds Teaching Hospital Trust) was a contributing member of the project team, which met bimonthly to review and advise on the study design, report writing and study outputs.

Jackie Whittle (Head of Nursing, Leeds Teaching Hospitals Trust) was a contributing member of the project team, which met bimonthly to review and advise on the study design, report writing and study outputs.

Data sharing statement

We are prepared to make pseudonymised interview transcripts available to researchers, subject to them obtaining the appropriate ethics approvals. Researchers should contact Justin Keen in the first instance.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health and Social Care.