A mixed-methods evaluation of transformational change in NHS North East

Article history

The research reported in this issue of the journal was funded by the HS&DR programme or one of its proceeding programmes as project number 08/1809/255. The contractual start date was in December 2009. The final report began editorial review in July 2013 and was accepted for publication in February 2014. The authors have been wholly responsible for all data collection, analysis and interpretation, and for writing up their work. The HS&DR editors and production house have tried to ensure the accuracy of the authors’ report and would like to thank the reviewers for their constructive comments on the final report document. However, they do not accept liability for damages or losses arising from material published in this report.

Declared competing interests of authors

Paula Whitty has been employed as Director of Research, Innovation and Clinical Effectiveness at one of the research study’s mental health trust study sites since April 2011 (and by the trust’s predecessors as Consultant in Medical Care Epidemiology since 1998). David Hunter is an appointed governor of one of the acute foundation trust hospital study sites involved in this research project and was a member of the commissioning board for the National Institute for Health Research (NIHR) Service Delivery and Organisation programme between 2009 and 2012, and the NIHR Health Services and Delivery Research programme between 2012 and 2014. Jonathan Erskine was a non-executive director of one of the primary care trust study sites until October 2011. Martin Eccles received a salary one day a month as a senior mentor for the National Institute for Health and Care Excellence Fellows and Scholars programme.

Copyright statement

© Queen’s Printer and Controller of HMSO 2014. This work was produced by Hunter et al. under the terms of a commissioning contract issued by the Secretary of State for Health. This issue may be freely reproduced for the purposes of private research and study and extracts (or indeed, the full report) may be included in professional journals provided that suitable acknowledgement is made and the reproduction is not associated with any form of advertising. Applications for commercial reproduction should be addressed to: NIHR Journals Library, National Institute for Health Research, Evaluation, Trials and Studies Coordinating Centre, Alpha House, University of Southampton Science Park, Southampton SO16 7NS, UK.

The North East Transformation System: key features, research aims and objectives

The NETS is of national and international importance because NHS North East (NHS NE), until its demise in March 2013, was the first Strategic Health Authority (SHA) to adopt a region-wide strategy that aimed to transform an entire health-care system. The initiative was bold and ambitious because the SHA served a population of 2.4 million people and the NHS in the region employed 77,000 staff. Up until that time, the implementation of lean and similar tools and methods in the NHS had involved relatively small-scale interventions confined to particular hospital departments and support services. 15 However, NHS NE was committed to addressing more complex system-wide issues, including addressing transformational change, culture and the relationship between clinicians and managers. Many aspects of the NETS were aligned with Darzi’s NHS next stage review conclusions about how best to ensure service redesign. 16

The NETS started with seven pathfinders that represented a wide range of NHS organisations, including the SHA, primary care trusts (PCTs), acute trusts and mental health trusts. These pathfinders constituted wave 1 of the NETS initiative. Subsequent waves included other NHS NE organisations that were programmed to undertake NETS training at regular intervals of around 12 months. The NETS took much of its inspiration and guidance from the Virginia Mason Production System (VMPS) whose lean method was derived from the Toyota Production System (TPS).

Although NHS NE was committed to supporting a full evaluation of the NETS, it was anxious to capture the learning from its early experiences. To this end, a 6-month scoping study of the background and initial steps was commissioned from Durham and Newcastle Universities. 11 The scoping study took place between January and July 2008 and investigated the NETS, its aims and objectives and the initial developments that had occurred in the seven first-wave pathfinder organisations. It prepared the ground for the main study, whose aims and objectives are described below.

Management–profession interface

The tension between managerial and professional values is well documented in the literature and underpins the rationale for a Compact. A BMJ editorial in 2001 posed the question: why are doctors unhappy?17 It suggested that the causes were multiple, although one in particular was highlighted: ‘the mismatch between what doctors were trained for and what they are required to do’. 17 Trained in some specialty or field of medicine, ‘doctors find themselves spending more time thinking about issues like management, improvement, finance, law, ethics, and communication’. 17 In an article in the BMJ the following year, Edwards et al. suggested that the cause of doctors’ unhappiness ‘is a breakdown in the implicit compact between doctors and society: the individual orientation that doctors were trained for does not fit with the demands of current healthcare systems’. 18 They described the old compact and why it is no longer regarded as legitimate, and outlined what a new compact might look like. The old compact comprised two aspects: what doctors give and what they get in return (see lists below). The mismatch between these has been the cause of dissonance over what doctors might have reasonably expected the job to be and how it now is. Some commentators have suggested that the psychological contract – or compact – is a useful concept to explain this problem. 18–20

What doctors give:

• sacrifice early earnings and study hard

• see patients

• provide ‘good’ care as the doctor defines it.

What doctors get in return:

• reasonable remuneration

• reasonable work/life balance later

• autonomy

• job security

• deference and respect.

A new and more sustainable Compact is required because the old promise to doctors is either no longer valid or can act as a barrier to modernisation. Among the new imperatives to be addressed in a revised Compact are the following:18

• greater accountability (e.g. guidelines)

• patient-centred care

• becoming more available to patients, providing a personalised service

• working collectively with other doctors and staff to improve quality

• evaluation by non-technical criteria and patients’ perceptions

• a growing blame culture.

Edwards et al. asserted that it was not possible to return to the old compact and that clinical leaders and managers must create ‘a new compact that improves care for patients, improves the effectiveness of the healthcare organisation, and helps create a happier workforce’. 18 The following year, once again in the BMJ, Davies and Harrison returned to the theme of the discontented doctor and argued that a principal reason for the dissatisfaction doctors experienced was their relationship with managers. 21 This manifests itself in a perceived sense of diminished autonomy and reduced dominance. The authors argued for ‘better alignment between doctors and the organisations in which they provide services’ while noting that the extent of ‘cultural divergence between managers, doctors, and other professional groups suggests that such a realignment will be far from easy’. 21 They concluded by insisting that there is no practicable alternative to doctors engaging with management. Yet, despite such calls, the unease felt by many doctors and their lack of being valued has persisted. 22 This was a major reason for inviting a surgeon, Ari Darzi, to lead the next stage review of the NHS, which had clinicians and other front-line staff at the heart of the change process – change that is ‘locally-led, patient-centred and clinically driven’16 (High Quality Care For All: NHS Next Stage Review Final Report, © Crown copyright 2008).

For their part, managers are also unhappy with their lot. They can appear beleaguered functionaries in a system that is more politicised than ever and whose political heads regard themselves as its leaders. 23 A major exponent of lean in the UK, Seddon, considered that distortions in the health system ensure that the patient is not put first. 24 The result is an elaborate set of managerial ploys which are, in effect, forms of cheating or gaming to arrive at the results desired by their political masters. But it is a further contributor to the unhappiness felt on both sides of the management–medicine divide.

The awkward and often dysfunctional relationship between managers and professions is far from being a new phenomenon. In their study of hospital organisation in 1973, Rowbottom et al. 25 noted that

the position of doctors . . . presents a fascinating, and possibly unique, situation to any student of organisation. Never have so many highly influential figures been found in such an equivocal position – neither wholly of, nor wholly divorced from, the organisation which they effectively dominate.

The work of Degeling et al. 26,27 demonstrated the importance of getting professionals and managers to:

• recognise interconnections between the clinical and financial dimensions of care

• participate in processes that will increase the systematisation and integration of clinical work and bring it within the ambit of work process control

• accept the multidisciplinary and team-based nature of clinical service provision and accept the need to establish structures and practices capable of supporting this

• adopt a perspective which balances clinical autonomy with transparent accountability. 27,28

The findings from Degeling’s work pointed to significant profession-based differences on each of the four elements of the reform agenda. 26–28 They also demonstrated the barriers that face those seeking to introduce changes in the delivery of health care. For those changes to happen there needs to be a common sense of purpose and a set of core values shared by the key stakeholders. These prior conditions do not exist. Degeling’s work showed that all attempts to impose managerial controls on clinical work are doomed to failure unless a different approach to managing change and engaging with clinicians and other health-care staff is adopted. 26,27

Organisational culture and leadership styles in health care

Culture is something of a weasel word that may simply be empty rhetoric. It is often invoked too readily and simplistically in a health-care context, the belief being that if culture change can occur then issues of organisational performance will be resolved. Despite this, culture does matter. Many commentators such as Schein29 and Mannion et al. 28 have emphasised the importance of culture in shaping organisational behaviour and hence achieving improved performance. However, change can be stifled by culture. As Mannion et al. stated, culture constitutes the informal social aspects of an organisation that influence how people think, what they regard as important, and how they behave and interact at work. 28 Organisational culture has been defined by Schein29 as:

the pattern of shared basic assumptions – invented, discovered or developed by a given group as it learns to cope with its problems of external adaptation and internal integration – that has worked well enough to be considered valid and, therefore, to be taught to new members as the correct way to perceive, think, and feel in relation to those problems.

Culture is therefore not merely that which is observable in social life but also the shared cognitive and symbolic context within which a society or institution can be understood. 28 But Mannion et al. 28 resisted the temptation of searching for a ‘magic bullet or simple cultural prescription for the ills of the NHS’ (pp. 214–15). In their view, ‘what works’ is contingent upon context ‘and on how and by whom efforts targeted at culture reform are evaluated and assessed’ (pp. 214–15). They counselled against the adoption of a ‘one size fits all’ approach to culture management in the NHS and ‘encourage[d] the adoption of more nuanced strategies which seek to deploy a judicious mix of instruments and supporting tactics depending on setting and application’ (pp. 214–15). One of the principal components of effective culture management relates to leadership styles. 30 Much has been written about leadership and hundreds of definitions offered but, as Goodwin observed, ‘it is principally local context that largely determines the leadership approach to be adopted, meaning local challenges, the history and relative strength of local relationships, local resource issues and local ways of doing things’ (p. 330). 31

Some writers on leadership subscribe to a trait or competency approach, i.e. one size fits all, which ignores context. The NHS competency framework is an example of this. Competencies have been criticised for being overly reductionist, overly universalistic or generic, focusing on past or current performance, focusing on measurable behaviours and outcomes, and resulting in a limited and mechanistic approach to development. 32 Critics also believe that ‘such frameworks reinforce the underlying assumption that leadership resides in the individual’ (pp. 32–3). 33 They are regarded as too generic and ignore ‘the context of a situation and the complexity of running very challenging and diverse workplaces’ (pp. 32–3). As Western argued, ‘the experience on the ground may be that there is little room for seizing the future and empowering others when the context feels disempowering due to a production-line atmosphere where success is measured against meeting targets and deadlines’ (pp. 32–3). 33 Situational leadership is therefore regarded as more appropriate in the context of complex adaptive systems.

In their study of the impact of leadership on successful change in the NHS, Alban-Metcalfe and Alimo-Metcalfe found that competencies did not predict effectiveness but that ‘engaging’ leadership styles significantly predicted motivation, satisfaction, commitment, reduced stress and emotional exhaustion, and team effectiveness/productivity. 34 For them, leadership was viewed as a shared or distributed process and one that was embedded in the culture.

Like other writers on leadership, Goodwin30,31 also noted that leadership is not a characteristic of one person but rather is a process ‘played out between leaders and followers, without whom leadership cannot exist’ (p. 330). 31 Not all commentators believe that leadership and management are entirely separate entities, but those who do consider that leaders are different from managers because they view people from an emotional perspective, seeing them as individuals. 33 But managers can demonstrate leadership and a leader can have managerial skills. Bennis defined leaders as those who ‘master the context’ whereas managers ‘surrender to it’ (p. 45). 35 Leadership is about passion, vision, inspiration, creativity and co-operation, rather than control, which is the hallmark of management. 33 A variant on this view is that a leader creates change whereas a manager creates stability. Running through all these definitions is the derogatory assumption that management is the ‘other’ to leadership; a view of the manager as an outdated mechanistic functionalist. Leadership is very clearly in vogue and ‘sexy’; whereas managers are regarded as transactional in their approach, leaders are seen as transformational.

In keeping with this view of leadership as being about emotions and meaning rather than control, Goodwin claimed that leadership is a dynamic, relationship-based process that uses a twofold approach:

• creating an agenda for change using a strong vision; and

• building a strong implementation network to get things done through other people. 30

In their Leadership for Health Improvement framework, Hannaway et al. 36 applied improvement science thinking, which borrowed many of its ideas and principles from lean. The approach has been applied in the NHS as a result of the work of the former NHS Modernisation Agency and its successor, the NHS Institute for Innovation and Improvement, which has been superseded by the NHS Leadership Academy. The 10 High Impact Changes for Service Improvement Delivery includes the optimisation of flow and the reduction of bottlenecks, the application of systematic approaches, improved access and role redesign. 37

Complexity and health

It is generally accepted that leading and managing a health system is a complex business where there are few certainties and where ambiguity and paradox are often present. These need to be managed rather than denied or obscured by an inappropriate managerial model. Failure in public policy and public services occurs, according to Chapman, because ‘assumptions of separability, linearity, simple causation and predictability are no longer valid’ (p. 65). 38 Under such conditions of growing complexity, it is essential that those responsible for managing and governing take on a wider, more holistic perspective, ‘one that includes complexity, uncertainty and ambiguity’ (p. 65). 38 Systems thinking marks a shift away from regarding the entities being managed as if they were linear, mechanical systems. As Plsek (Paul E. Plsek & Associates, Inc., 2003) commented in materials distributed at a Leadership for Health Improvement workshop held in York in 2006, ‘existing principles of management and leadership are based on old metaphors that fail to describe adequately or accurately complex situations’ (© 2003 Paul E. Plsek & Associates, Inc.; reproduced with permission from Plsek P. An Organisation is not a Machine! Principles for Managing Complex Adaptive Systems. Materials prepared for Leadership for Health Improvement programme. York: Unpublished; 2006). In a complex system, the complex adaptive manager and/or leader:

• manages context and relationships

• creates conditions that favour emergence and self-organisation

• lets go of ‘figuring it all out’

• relies on ‘good enough’ analysis of the problem and its solution

• requires minimum specifications to act rather than prescribing actions in advance.

Plsek, in the workshop materials referred to above, defined a complex adaptive system as ‘[a] collection of individual agents with freedom to act in ways that are not always totally predictable, and whose actions are interconnected so that one agent’s actions changes the context for other agents’ (© 2003 Paul E. Plsek & Associates, Inc.; reproduced with permission from Plsek P. An Organisation is not a Machine! Principles for Managing Complex Adaptive Systems. Materials prepared for Leadership for Health Improvement programme. York: Unpublished; 2006).

The remainder of the literature review presented below focuses on lean thinking and tools, as these have been central to the NETS initiative and, in particular, in demonstrating that successful change is possible and motivational for staff. The sections The evolution of lean, Fordism and Toyota Production System focus on the origins and evolution of lean in the manufacturing sector, including its impact in the North East region of England. Lean in the UK examines the recent take-up of lean thinking in the UK NHS as well as the public services sector more widely.

The evolution of lean

Lean production evolved from established production management approaches. These include the concept of interchangeable parts devised by Eli Whitney, scientific management developed by Taylor, Henry Ford’s mass production and the TPS. 39–41

Taylor found that in craft systems, skills and knowledge were transferred through the apprentice model. 39 Braverman considered that management was buying knowledge which was the workers’ capital. 42 There were variations in the volume and quality of work performed by different individuals as they had different ways of performing tasks. Taylor used the term ‘soldiering’ to describe workers who deliberately did work slowly. 40 He noted that there was conflict between management and workers fighting over the control of work and pay. There was a lack of standardised working methods and training. The end result was that there was waste, which was to the detriment of both the workforce and management. 43

The scientific management movement was very influential in the development of modern institutions which carry out labour processes. 42 Taylor stated that: ‘the principal object of management should be to secure the maximum prosperity for the employer, coupled with the maximum prosperity for each employee’ (p. 9). 39 Taylor believed that this required each individual to maximise efficiency by producing the greatest possible daily output.

Taylorism is based upon four principles:

1. It is necessary to systematically analyse work through time and motion studies to develop standardised methods.

2. Organisations should train employees in best practice approaches rather than leaving them to train themselves.

3. Workers should be provided with detailed instructions that prescribe how to undertake standardised tasks.

4. There should be a separation between ‘planning’, undertaken by managers using scientific management principles, and ‘doing’, performed by workers. 44,45

The core elements of Taylorism are (1) that operations should be standardised and optimised scientifically using time and motion studies and (2) the division of labour between managers and workers. 46 Taylor highlighted the importance of training as a mechanism for ensuring that individuals were able to work at maximum efficiency. 39

Taylorism has several limitations. 45 It creates non-value-adding supervisors and other indirect workers, which increases costs. Flexibility is reduced by the reliance on semiskilled workers and high levels of demarcation. However, new workers can be integrated quickly into the production process, which increases numerical flexibility. It becomes unattractive to work on the shop floor. Taylorism is widely considered to be anti-worker. 42

Fordism

Ford developed mass production at the Highland Park plant in 1913. His approach incorporated many aspects of Taylorism. He automated the production of standard parts using repetitive processes and introduced a continuously moving assembly line. 40 The pace of work and the volume of production were determined by the line speed. The combination of Taylorist approaches and assembly lines, together with a rigorously controlled and well-paid workforce, became known as ‘Fordism’,43 which achieved economies of scale through the division of labour. The objective was to minimise the unit cost through high-volume manufacturing. Production workers were not responsible for quality; there were specialist inspectors and repair staff. There was also a strict separation between the planning and execution of work, and a high division of labour. 46

Toyota Production System

The TPS was developed by Taiichi Ohno. 41 He identified several barriers to implementing mass production approaches in Japan. After the Second World War there was limited domestic demand. Furthermore, customers required a large variety of vehicles, so there was a requirement for low volume and high variety. Workers were reluctant to work in a system that considered them to be a variable cost, owing to legislation that strengthened workers’ rights during the period of American occupation. Japanese companies were starved of capital and foreign exchange; therefore, companies were unable to purchase expensive Western equipment. There was intense competition from overseas manufacturers that were keen to enter the Japanese market while defending their established markets. 47 Toyota could not afford to fund the working capital required to maintain the buffers that would be needed to maintain the high utilisation of production lines that were subject to line imbalances, quality problems and other sources of variability. Toyota therefore developed an alternative solution, which was to operate with minimum inventory while simultaneously maintaining high resource utilization. 48 The TPS is based upon two concepts. First, costs are reduced through the elimination of all forms of waste (those things that do not add value to the product). Second, there is a need to fully utilise workers’ capabilities. 49

The TPS may be viewed as a set of tools for reducing waste or as the set of principles that led to the development of the tools. 50 Liker51 identified 14 principles associated with four themes: (1) long-term philosophy; (2) the right process will produce the right results; (3) add value to the organisation by developing people; and (4) continuously solving route problems drives organisational learning.

Lean in the UK

In the 1980s and 90s inward investment by Japanese automotive companies exposed the uncompetitiveness of many UK automotive components suppliers. 52,53 The Society of Motor Manufacturers and Traders collaborated with the Department of Trade and Industry to form the Industry Forum in 1996. 54,55 This initiative was supported by major automotive companies. They seconded staff with expertise in improving manufacturing processes, who became ‘master engineers’. 56,57 Their role was to train UK engineers in the use of ‘best practice’ manufacturing tools and techniques in order to increase the competitiveness of the UK suppliers. The Industry Forum developed training programmes that used a ‘Common Approach Toolkit’. The ‘building blocks’ included clear out, configure, clean and check, conformity, custom and practice (5C)/sorting, set in order, systematic cleaning, standardising and sustaining (5S),58 the seven wastes,59 standardised work and visual management. The workshops also included training in data analysis, problem solving, set-up improvement and line balance. 56 Skills, knowledge and delivery techniques were disseminated through ‘Master Classes’,60,61 which included training and practical, shop floor-based process improvement activities. The aim of the Master Class is to introduce staff to best practice approaches and to improve performance in terms of quality, cost and delivery. 56

The superior performance of Japanese exemplars encouraged the dissemination of lean principles and tools to other contexts, including service industries and health care. However, Western organisations have often been able to adopt specific lean tools but have found it difficult to change the organisational culture and mindset. The impact of lean interventions is often localised. Organisations often fail to achieve the desired improvements to the overall system. 62

Lean philosophy and strategies

The TPS was developed in Japan by Ohno and Shingo and forms the basis of lean manufacturing. 63 Vollmann et al. 64 considered the goal of lean to be zero inventories, zero defects, zero disturbances, zero set-up time, zero lead time, zero transactions and routine operations that operate consistently day to day. Transactions consist of (1) logistical transactions (ordering, execution and confirmation of material movement); (2) balancing transactions, associated with planning that generates logistical transactions (production control, purchasing, scheduling); (3) quality transactions (specification and certification); and (4) change transactions (engineering changes, etc.).

In lean manufacturing, waste may be viewed as any activity that creates cost without producing value. 65,66 Ohno outlined seven forms of waste:67

1. Overproduction, i.e. making too many items or making items before they are required. The result is extended lead times and increased inventory, which incurs carrying costs.

2. The waste of waiting, i.e. time when materials or components are not having value added to them.

3. The waste of transportation, i.e. the movement of materials within the factory, which adds cost but not value.

4. The waste of inappropriate processing describes the use of a large, expensive machine instead of several small ones leads to pressure to run the machine as much as possible rather than only when needed.

5. The waste of unnecessary inventory, which increases lead times, reduces flexibility and makes it difficult to identify problems. This form of waste increases carrying costs and may cause waste through obsolescence.

6. The waste of unnecessary motions relates to ergonomics. If operators become physically tired it is likely to lead to quality and productivity problems.

7. The cost of defects is caused by internal failures within the factory, including scrap, rework and delay, as well as external failures that occur outside the manufacturing system (repairs, warranty cost and lost custom).

Bicheno59 identified additional ‘new’ wastes: untapped human potential; inappropriate systems that add cost without adding value; wasted energy and water; wasted materials; wasted customer time; and the waste of defecting customers – it may cost many more times to acquire a customer than it does to retain one.

Lean comprises a philosophy, a way of thinking that focuses upon value. Often this is considered in terms of cost reduction:62

This migration from a mere waste reduction focus to a customer value focus opens essentially a second avenue of value creation . . . Value is created if internal waste is reduced, as the wasteful activities and the associated costs are reduced, increasing the overall value proposition for the customer . . .

The other main emphasis is on continuous improvement that is usually based upon teamwork undertaken by empowered employees.

Lean initiatives in the North East of England

In 2000, the level of productivity (measured in terms of gross value added, i.e. the value of outputs minus the value of inputs) in the North East was 25% lower than the national average. 68 This situation had made the support of manufacturing companies a major policy objective of One North East (ONE), the regional development agency (RDA) (RDAs were abolished in 2012 as part of the government’s deficit reduction programme). In 2002, ONE funded the North East Productivity Alliance (NEPA), which aimed to improve the productivity of regional companies by training employees in lean tools using the Master Class approach.

The NEPA Master Classes selectively trained employees in the following tools: (1) 5S/5C; (2) standard operations; (3) skill control; (4) Kaizen; (5) visual management; (6) process flow; (7) problem solving; (8) Single-Minute Exchange of Dies (SMED); (9) production-led maintenance; (10) work measurement; (11) failure mode effect analysis; (12) poka-yoke (mistake-proofing); (13) value stream mapping (VSM); and (14) advanced problem solving. 69

The NEPA approach was further developed in 2008 through the European Regions for Innovative Productivity (ERIP) project. Research led by Newcastle University developed an improved lean implementation approach for small business that aimed to standardise processes and reduce costs while improving quality and delivery performance. 70 This approach was applied in 25 small- and medium-sized enterprises (SMEs) across the North Sea Region of Europe. Policies for improving the productivity and efficiency of SMEs had not been fully resolved by any of the European member states. Therefore, a transnational approach was advocated to develop a transferable solution. Building on the research highlighted above, a grant was awarded by the European Regional Development Fund (Interreg funding) to support the transfer of lean into SMEs across the North Sea Region. The objective of the ERIP project was to develop a methodology, using the knowledge developed in the North East of England (building on the NEPA work), which could be used by SMEs across the North Sea Region of Europe to implement lean to become more competitive. A key challenge identified through the research was that SMEs found it difficult to allocate the necessary time and resource to undertake the improvement activities. To address this, a new approach was developed through the research, called bite-sized lean. 70 This demonstrated that while lean could be applied in various contexts, it needed to be tailored. The next sections review research relating to lean in the public and health-care sectors respectively.

Lean in the public sector

McNulty and Ferlie71 argued that the UK’s new public management (NPM) reforms, which began in the early 1980s, eventually evolved, by the mid-1990s, into what they termed ‘NPM 4’. NPM 4 is characterised by a melange of private and public sector management ideas, emphasising a value-driven approach, concerned with the quality of service, and a continuing commitment to a distinctive public sector ethos of collective provision. Although McNulty and Ferlie had reservations about the application of private sector models such as business process re-engineering (BPR) to public sector settings (within the context of NPM 4), they concluded that the shift to a NPM model has made the public sector more receptive to ideas of process redesign. Lean is focused on process rather than product, and to this extent McNulty and Ferlie’s observations are highly relevant. For example, they found that a number of public sector organisations experience a problem over organising their work with regards to process and functional principles. 71 This is precisely the tension described by Seddon72 when he criticised local government for setting up ‘front office/back office’ call centres to process the various requests and demands from local citizens.

Although BPR and total quality management (TQM) have certainly attracted considerable attention in the public sector – including health care – there is evidence from recent literature that lean management and the TPS are currently more in vogue in a variety of public service settings. 73 For example, Hines and Lethbridge74 reviewed a project that implemented a lean value system in a university. The authors found a number of case studies relating to lean initiatives in academic settings, and they engaged in a 3-year initiative to embed lean methods and thinking in a client university. 74 Radnor and Bucci75 investigated the use of lean in UK business schools and universities. They found that the application of lean in UK universities was still relatively new and primarily applied in support and administrative functions.

Hines et al. 76 explored the use of lean in the Portuguese and Welsh legal public sectors, particularly in court services. McQuade77 discussed the organisational transformation brought about by lean thinking in a UK social housing group. Erridge and Murray78 reported on the application of lean principles to local government procurement processes, using the example of procurement contracts drawn up by Belfast City Council. The authors concluded that ‘lean supply’ was compatible with the best value approach to procurement, as long as characteristics of lean that are most closely aligned to manufacturing are adapted to fit the culture of local government. Scorsone79 considered the implementation of lean by the city government of Grand Rapids, MI, USA, in the face of fiscal restrictions and a dwindling workforce. Radnor and Bucci80 evaluated a lean programme undertaken in Her Majesty’s Court Services. The programme sought to focus on administrative functions as part of the operational aspects of court services. The underlying challenge of the project was to improve the delivery of service through a waste elimination programme, as well as simplify processes and free up capacity to be able to do more work. The outcome of the evaluation found that the programme had created impact within the court services. Leadership played a key role along with the dedicated project team. Other findings identified the understanding of why there is a need to change, changing and updating practices which had not been revisited, and engaging with the workforce in a manner that promotes a desire to change. Radnor and Bucci provided a framework to support the implementation of lean in public services. This framework requires an understanding of an organisation’s processes, customer requirements and types of demand. These factors are identified as key to ensuring that any lean programme can be successful. 80

One of the most wide-ranging evaluations of lean in the public sector – in terms of the scope of the research undertaken – is the report titled Evaluation of the Lean Approach to Business Management and Its Use in the Public Sector. 81 This document aimed to provide a comprehensive assessment of the success of the lean philosophy and tools in transforming a number of public sector organisations in Scotland. The report, commissioned by the Scottish Executive, covered eight case studies and three pilot sites, including local authorities, health agencies and a government (Royal Air Force) agency. It described a range of levels of engagement with lean, from full implementation (acceptance of lean thinking across all levels of an organisation, and use of lean tools and techniques, together with some likelihood of sustainable transformation) to light-touch lean [adoption of a ‘quick win’, toolkit approach, usually based on rapid improvement events (RIEs)]. These case studies collectively warn against a lean implementation approach that relies too heavily on the lean/TPS ‘toolbox’ without a complementary commitment to lean thinking at all levels of the organisation. The implication is that the tools of lean/TPS (RIEs, 5S, Kaizen blitz, etc.) are less likely to become embedded and have sustainable value unless they are part of a wider package of organisational reform. Furthermore, Radnor and Walley82 suggested that many public sector organisations lack an understanding of process management. Organisations that only gain quick wins, usually via RIEs [or rapid process improvement workshops (RPIWs) in VMPS parlance], find it difficult to sustain improvement in the long term. In a number of cases, the authors found a lack of alignment between the lean/TPS implementation and the organisations’ strategic objectives.

An over-reliance on the lean/TPS toolbox can make it difficult to embed process-oriented thinking. Radnor et al. 81 found that a common public sector response has been to avoid specific, transformational, quick win tools, believing these to be unwelcome imports from a manufacturing environment and inappropriate for use in public service. The message here is that balance is required. The case studies showed that the success of lean/TPS implementation was context dependent, and relied to a large degree on a number of organisational and cultural factors. When lean is not fully aligned with the strategy of a public sector organisation, there is a risk that it will not be sustainable in the long term. Having a critical mass of people who are trained in lean and accepting of it as a transformational agent is also essential.

As a brief summary, Radnor et al. 81 identified the following critical success factors for implementing lean in public service organisations:

• organisational culture and development

• organisational readiness

• management commitment and capability

• external support from consultants (at least initially)

• having a strategic approach to service improvement

• teamwork and whole-systems thinking

• timing – setting realistic timescales and making effective use of staff commitment and enthusiasm

• effective communication channels across the whole organisation.

These points were reinforced by a range of papers in a special issue of Public Money & Management (February 2008). These considered the relevance of lean in improving public sector services; aspects of lean thinking that ‘fit’ public service organisations; the transfer of lean experience from other sectors; and the extent to which lean is a distraction or a panacea.

Lean in health care

Most of the research on lean in health care has focused on hospitals. Spear83 highlighted a series of avoidable medical errors and patient safety issues in the US hospital sector. He advocated the use of the TPS to remove ambiguities in processes and to empower health-care workers to solve problems as they arise, rather than opting for work-around solutions. Spear pointed out that ‘No organisation has fully institutionalised to Toyota’s level the ability to design work as experiments, improve work through experiments, share the resulting knowledge through collaborative experimentation, and develop people as experimentalists’. 83

Radnor et al. 84 investigated the introduction of lean in four UK NHS hospital trusts. They found a widespread use of lean tools that led to small-scale and localised productivity gains and highlighted significant contextual differences between health care and manufacturing that made it difficult to move towards a more system-wide approach. In particular, some of the principles proposed by Womack et al. 47 do not apply. ‘Customer value’ in health care is different to manufacturing because the patient is normally a recipient of treatment and does not commission or pay for the service. The provision of health care is often subject to budgetary constraints that make it capacity led; there is limited ability to influence demand or reallocate resources saved by improvement measures.

Fillingham15 described the use of the TPS for improving patient care at the Royal Bolton Hospital in the UK, which has been widely considered to be an exemplar case. He reported a 42% reduction in paperwork, better multidisciplinary teamworking, a reduction in length of stay by 33% and a 36% reduction in mortality. Ballé and Régnier85 reported on the use of lean to reduce medication distribution errors, nosocomial infection rates and catheter infections in a French hospital. Although the initiative was deemed successful, the authors identified resistance to the standardisation of clinical and nursing practices. Gowen et al. 86 investigated the application of continuous QI, Six Sigma and lean in US hospitals. They concluded that lean was significant in reducing the sources of errors, but that it did not improve organisational effectiveness. Chiarini87 researched an improvement project utilising lean and Six Sigma tools to reduce safety and health risks to nurses and physicians who managed cancer drugs in an Italian hospital. The author identified that the tools helped improve health and safety and reduced pharmaceutical inventory. Yeh et al. 88 looked at the application of lean thinking and Six Sigma and how they could be used to improve processes in treating an acute myocardial infarction. The outcome was that the medical quality improved, as did market competitiveness. Esain and Rich89 focused on improving patient flow through hospitals to reduce waiting times.

Outside the hospital sector, Boaden and Zolkiewski90 conducted a process study of the non-clinical aspects of a UK general practice, with particular attention to the relationship between the patient and the managerial and administrative aspects of the organisation. Endsley et al. 91 considered process and flow issues in family medical practice in the USA. They focused upon understanding patient needs and administrative procedures, which led to reduced waste. Endsley et al. made a good case that, from the patient’s perspective, many of the frustrations involved in accessing general practitioner (GP) services arise not from direct contact with the physician, but from missing paperwork, unacceptably long waiting times, and poorly managed hand-offs between doctor, practice nurse and receptionist. 91

One of the most frequently cited exemplars of lean in health care is the Virginia Mason Medical Center (VMMC). It adopted the TPS to create the VMPS. The research on VMMC has included work by Weber,92 who investigated how improved logistics and productivity reduced costs and defects; Furman and Caplan,93 who outlined the patient safety alert system; Nelson-Peterson and Leppa,94 who described the elimination of waste in nursing procedures; McCarthy95 on the application of the TPS; Bush96 on eliminating waste; Kowalski et al. 97 on nurse retention and leadership development; and Pham et al. 98 on the redesign of care processes.

The next chapter considers the origins and evolution of the NETS, which was supported by consultancy from VMMC and Amicus.

Why the North East Transformation System?

In the North East of England the NHS performs well in terms of meeting targets and performance measures, but the population has poor health due to the region’s industrial heritage and socioeconomic factors. 99 Although this might seem to be a paradox at first glance, it is not really, as good health is determined by factors that lie outside the health-care system. 100 The social and economic circumstances in which people live and work can have a significant impact on their state of health, as Marmot’s work on the social determinants of health and the life course amply demonstrates. 101,102 Nevertheless, while in existence, NHS NE believed that the NHS could do much better by focusing on quality and patient safety and adopting a whole-systems approach to an individual’s and community’s health state. 11

The NETS was instigated as a result of the SHA board’s conviction that a new approach to the way in which it conducted its business was both required and essential. At a meeting held to share information about lean activities across trusts in the North East of England in February 2011, the SHA’s medical director, one of the chief champions of the NETS, commented that the solution was no longer simply doing ‘the same thing but harder’, but doing it smarter. Merely doing what had always been done would deliver exactly the same result. To shift the paradigm or do something genuinely different required changing the rules of the game and transforming the culture of the system. This is something Don Berwick and his colleagues at the Boston-based Institute for Healthcare Improvement in the USA had known for a long time. 103 It was a central theme in his report to the coalition government on the lessons to be learned from the failures at Mid Staffordshire hospitals. 104 NHS NE sought to achieve system-wide rather than localised transformation through a major change programme that would engage all parts of the health system in the North East, including commissioners as well as providers of services. There was an early intention to encourage GPs to undertake training in the principles of the NETS, but it was recognised that GP practices were unlikely to form part of the vanguard, as they lacked the required resources to do so.

Throughout the NHS there was considerable interest in organisational change and the tools available to embed and sustain it. These were reviewed in a National Institute for Health Research (NIHR)-funded study by Isles and Sutherland,105 intended for health-care managers, professionals and researchers. The study concluded that change in the NHS will not be straightforward. The NHS was an example of a ‘complex adaptive system’, which Plsek and Greenhalgh defined as ‘a collection of individual agents with freedom to act in ways that are not always predictable, and whose actions are interconnected so that one agent’s actions changes [sic] the context for other agents’. 12 Complex adaptive systems invariably have fuzzy boundaries, with changing membership and members who simultaneously belong to several other systems or subsystems. In such contexts, tension, paradox, uncertainty and ambiguity are natural phenomena and cannot necessarily or always be resolved or avoided. Instead, they need to be embraced by the various stakeholders and harnessed in such a way that they result in sustainable solutions to complex problems. Arising from such concerns, the term ‘whole-systems thinking’ is now routinely used by managers and clinicians to capture the particular features of a complex health-care system and reflect the following features:

• an awareness of the multifactorial nature of health care and an acknowledgement that complex health problems – often termed ‘wicked problems’ because they have no simple or easy solutions – lie beyond the ability or capacity of any one practitioner, team or agency to fix106

• an interest in designing and managing organisations as dynamic interdependent systems committed to providing safe, integrated care for patients. 105

The NETS and its evaluation was informed by systems thinking. A system cannot be considered in isolation from its context and overall environment,107 nor do systems constitute neat chains of linear cause-and-effect relationships which can be isolated and understood in their own terms. In health-care systems, complex networks of inter-relationships are the norm. 108

Isles and Sutherland’s 2001 review noted that the problems and situations that occur cannot be resolved through the use of a single tool or strategy. 105 Consequently, NHS managers had to acquire the ability to diagnose different situations, as well as the skill to find the right tool to use in the particular circumstances that they face. NHS NE’s choice of the NETS and its three main components was certainly informed by such a diagnosis as well as exposure to the VMMC’s adoption of a change programme. The VMPS was derived from TPS principles and methods that were subsequently adapted to suit the North East context.

The NETS comprises three components, familiarly known as the ‘three-legged stool’: Vision, Compact and Method. These components are not individually pioneering. It is the combination that constitutes the NETS approach to transformational change and lends it particular novelty as far as the NHS is concerned.

Compact

The Compact arose from the enduring tension between managerial and professional values and the attempt to find some accommodation between the two groups with their differing cultures. It was influenced by the Physician Compact introduced by the VMMC at an early stage of its change journey. The concept of a compact was simple enough: an explicit deal between two parties which, in the case of the VMMC, comprised the clinicians and the VMMC organisation. 109 The intention was to move from an implicit compact to a new explicit one which reflected changes in health care and its management.

Such tensions were not confined to the VMMC or even the USA. In the UK, successive governments have sought to change the way the NHS operates and is managed. Central to these efforts has been shifting the frontier between medicine and management in favour of the latter. It was a development that began in earnest with the first major reorganisation of the NHS in 1974 and has been an enduring theme ever since. However, the effect of a growing managerial encroachment into medicine was not unanimously welcomed by clinicians. Many were suspicious of such developments and opposed to what they perceived to be an erosion of their clinical freedom. An editorial in the BMJ in 2001 posed the question: why are doctors unhappy?17 It suggested the causes were multiple, but highlighted one in particular, which concerned the training doctors received and the nature of the work they were subsequently required to perform. Despite their medical training in a particular specialty, the reality of life on the front line required doctors to think about other matters, including management, finance, ethics and communication. 17 Edwards et al. 18 suggested that the cause of doctors’ unhappiness lay in a breakdown between them and society at large. Doctors were trained to function as individuals with a fair degree of autonomy, but the demands of modern health-care systems required them to be accountable for their actions and to operate as members of a team. They described the old compact which underpinned the NHS and why it was no longer regarded as legitimate, and outlined what a new compact might look like. The old compact comprised two aspects: what doctors gave and what they got in return (Table 1). The mismatch between these was the cause of the dissonance over what doctors might have reasonably expected the job to be and what it now was. Some commentators have suggested that the psychological contract – or compact – is a useful concept to explain this problem. 18–20 Jack Silversin from Amicus, an international consultancy specialising in health-care system improvement, was an influential figure during this period on both sides of the Atlantic. His contribution to the debate was significant because he had worked on the Physician Compact at VMMC and subsequently provided an input into the NETS and its development of the Compact.

TABLE 1 - The old compact18

What doctors give	What doctors get in return
Sacrifice early evenings and study hard	Reasonable remuneration
See patients	Reasonable work/life balance later
Provide ‘good’ care as the doctor defines it	Autonomy
	Job security
	Deference and respect

A new and more sustainable compact was required because the old promise to doctors was either no longer valid or could act as a barrier to modernisation. Among the new imperatives to be addressed in a new compact were those listed in Box 1.

BOX 1 - The new compact18

New imperatives

• Greater accountability (e.g. guidelines).

• Patient-centred care.

• More available to patients, providing a personalised service.

• Work collectively with other doctors and staff to improve quality.

• Evaluation by non-technical criteria and patients’ perceptions.

• A growing blame culture.

Edwards et al. 18 were of a view that returning to the old compact was not possible and that doctors and managers should work together to devise a new compact that: was fit for modern health-care systems, sought to improve patient care and the effectiveness in which they worked, and would result in a happier workforce. Apart from the issues with patients and their care, a principal reason for the discontent among doctors was the dissatisfaction they experienced in their relationship with managers. It manifested itself in a perceived sense of diminished autonomy and reduced dominance. To address these concerns, Davies and Harrison21 argued in favour of a better alignment between doctors and the organisations in which they worked. However, given the tensions and cultural differences that existed between doctors, managers and their professional groups, they were under no illusions that such a task would be easy or straightforward. They concluded by insisting that there was no practicable alternative to doctors engaging with management. Despite such calls, there was still unease felt by many doctors who perceived that they were not valued. 22 This was a major reason for Ari Darzi being invited by the government to lead the next stage review of the NHS. As noted in Chapter 1, this was an attempt to re-engage clinicians in the reform effort together with other front-line staff so that they were at the heart of the change process – change that was ‘locally-led, patient-centred and clinically driven’. 16

However, the unhappiness felt was, and is, not confined to clinicians. Managers are also unhappy with their lot, a situation that has arguably, and for many, deteriorated further in recent years as a result of being subjected to continuous organisational change whose precise purpose is often unclear and whose impact often falls short of expectations. Combined with a culture of fear and ‘terror by targets’,110 the outcome is an environment which, as the Berwick report put it, ‘is toxic to both safety and improvement’. 104 The ‘harvest of fear’ evident at Mid Staffordshire resulted in

a vicious cycle of over-riding goals, misallocation of resources, distracted attention, consequent failures and hazards, reproach for goals not met . . . A symptom of this cycle is the gaming of data and goals; if the system is unable to be better, because its people lack the capacity or capability to improve, the aim becomes above all to look better, even when truth is the casualty.

Berwick 2013104 [A Promise to Learn – A Commitment to Act: Improving the Safety of Patients in England, © Crown copyright 2013, contains public sector information licensed under the Open Government Licence v2.0 (www.nationalarchives.gov.uk/doc/open-government-licence/version/2/)]

Managers in such a dysfunctional environment can appear beleaguered functionaries in a system that seems more politicised than ever and whose political heads regard themselves as its true leaders. 23 Indeed, critics of the waves of reform under the Labour government between 1997 and 2010 hold that the ‘terror by targets’ regime was largely responsible for distortions in the health system which, unintentionally, ensured that the patient was not put first. 110 The result, as Berwick noted, is an elaborate set of managerial ploys, often labelled ‘gaming’, to arrive at the results desired by their political masters and mistresses.

The term ‘culture’ is often invoked too readily and simplistically in a health-care context, especially in the aftermath of the Francis report into the events at Mid Staffordshire Hospital between 2005 and 2009. 111 It is assumed that culture change will address issues of organisational performance. Culture is important in terms of shaping organisational behaviour and improving performance. 28 Change can also be stifled by culture. Mannion et al. 28 commented that culture comprises the informal social aspects of an organisation that influence how people think, what they regard as important, and how they behave and interact at work. Organisational culture has been defined as29

the pattern of shared basic assumptions – invented, discovered or developed by a given group as it learns to cope with its problems of external adaptation and internal integration – that has worked well enough to be considered valid and, therefore, to be taught to new members as the correct way to perceive, think, and feel in relation to those problems.

Culture is therefore not merely that which is observable but also the shared cognitive and symbolic context within which a society or institution can be understood. 28 But Mannion et al. counselled against the adoption of a ‘one size fits all’ approach to culture management in the NHS and encouraged ‘the adoption of more nuanced strategies which seek to deploy a judicious mix of instruments and supporting tactics depending on setting and application’ (pp. 214–15). 28 These issues were very much at the heart of the NETS and how it sought to win ‘hearts and minds’ across the North East region. Those leading the initiative recognised the importance of establishing shared goals, cultural change and collaborative working.

As the literature review in Chapter 2 revealed, a principal component of effective culture management is leadership style. 30 Perhaps the key factor to note is that leadership entails much more than the actions or behaviour of an individual, as such a focus ignores the importance of both context and complexity. Situational leadership is therefore regarded as more appropriate in the context of complex adaptive systems. Alban-Metcalfe and Alimo-Metcalfe,34 in their study of leadership and successful change in the NHS, found that a culture of ‘engaging’ leadership significantly predicted motivation, satisfaction, commitment, reduced stress and emotional exhaustion, and team effectiveness/productivity. Leadership, then, is not about control but about co-operation and creating an agenda for change using a strong vision.

In their Leadership for Health Improvement framework (Figure 1), Hannaway et al. 36 employed a mix of improvement science concepts together with softer notions of emotional intelligence and political astuteness.

FIGURE 1.

The ‘Leadership for Health Improvement’ framework (reproduced with permission from Figure 7.2 in Hunter DJ, editor. Managing for Health. London: Routledge; 2007. p. 158). 36

Conclusions

In later chapters, we describe and assess the journey taken by the NETS from its inception to the present time. Large-scale transformational change is challenging and often problematic in any complex adaptive system. This is especially so when considering a large region’s entire health-care system. 71 There are multiple reasons for this. Constituent organisations have conflicting aims, different cultures and varying skill mix. These issues were evident in the NHS organisations participating directly in, or more loosely associated with, the NETS. External forces, often unforeseen and unexpected, can also have a decisive impact on what happens regionally and locally. This occurred to the NETS in mid-2010, when the coalition government elected in May 2010 announced a major restructuring of the NHS in England. The research reports on the impact of the NETS within the North East region and how it adapted to a changing environment.

Research design

Transformational change may be viewed in terms of content, context, process and outcomes. 117 Furthermore, there is a distinction between episodic and continuous change. 118 This led to different types of research questions that needed to be addressed through qualitative and/or quantitative methods. 119 Various typologies of mixed-methods research have been proposed that consider sequence, priority, purpose, etc. 120 The research methods included semistructured interviews, observation, documentary analysis, focus groups, attendance at meetings and presentations and interrupted time series (ITS) analysis.

The research approach is shown in Table 2, which identifies the sources of qualitative and quantitative data that were used to evaluate processes and outcomes. The priority and sequence of the data collection and methods varied according to the quadrant and also the research question being addressed. Initially, a deductive approach was adopted and employed to aid the analysis of qualitative data at the end of the first year of the study, and to assist in identifying key issues associated with transformational change. As the study progressed, emerging themes were analysed inductively to complement the deductive approach. 121,122 The research methods also took into account multiple levels of analysis; for example, at the level of specific interventions the stakeholders included a sponsor, a process owner, a workshop leader, a team leader, a subteam leader, an advisory group and participants. 123 The NETS included formal evaluations of the impact of the interventions after 30, 60 and 90 days.

Study sites

Stratified purposive sampling is an approach where certain cases are selected to ensure that they vary according to preselected parameters. 115 This approach was adopted so that the sample of cases included organisations that were representative of the scale, scope and geographic location of the participating trusts. A similar strategy was adopted within each case to ensure that the respondents selected for interview were representative of the various professional groups, skill mix and band level.

The plan outlined in the initial proposal was to conduct research in five pathfinders that had implemented the VMPS and two non-pathfinders as controls. The pathfinders were subdivided into two waves: the first wave started in 2008 and the second in 2009. This was intended to reflect organisations at different stages of the NETS journey. The actual design was modified to reflect the range of methods used to achieve transformational change (including study sites not using the VMPS as their Method), as well as the changes to NHS organisations across the North East that occurred following the 2010 general election and came to be enshrined in the Health and Social Care Act 2012. 124

The research design (see Figure 2) included 14 study sites (Table 3) which formed the basis of nine case studies (as some organisations operated on multiple sites). Four cases were wave 1 pathfinders, one was a wave 2 pathfinder and four were non-pathfinders. All stages of the research involved multiple investigators to allow for triangulation and to enable different perspectives to be obtained from multiple observers. 112

Longitudinal research design, including timetable

A representation of the research project is shown in Figure 2, which contains an overview of the work conducted during each year of the 3-year research study. Figure 2 is a schematic diagram intended to provide a visual reminder of the elements of the design; details are explained in the remainder of this section.

FIGURE 2.

Research design. KPO, Kaizen Promotion Office; OD, organisational devolvement.

Methods

This section outlines the key methods used to obtain, manage and analyse the data that resulted from our research work.

Qualitative data analysis

The Pettigrew et al. 125 framework of the ‘receptive contexts for change’ (Figure 3) was adopted as a theoretical construct to investigate and understand transformational change in NHS NE. It comprises eight factors: (1) quality and coherence of policy; (2) availability of key people leading change; (3) long-term environmental pressure to trigger change; (4) supportive organisational culture; (5) effective managerial–clinical relations; (6) co-operative interorganisational networks; (7) simplicity and clarity of goals and priorities; and (8) fit between the change agenda and its locale.

FIGURE 3.

The ‘receptive contexts for change’ framework [reprinted with permission of SAGE Publications, London, Los Angeles, New Delhi and Singapore, from Pettigrew AM, Ferlie E, McKee L. Shaping Strategic Change: Making Change in Large Organizations: The Case of the National Health Service. London: SAGE; 1992 (© SAGE, 1992)]. 125

From these eight factors a series of subcodes were derived through a detailed exploration of the explanation of the interaction of these factors offered by Pettigrew et al. 125 The coding framework is set out in Table 4.

TABLE 4 - The coding framework

Code	Subcode
Quality and coherence of policy	Blueprint
	Vision
	Coherent policy
	Fragmented policy
	Framing strategic issues
	National policy
	Regional policy
	Shared world view
	Trust policy
	Commitment building
Key people leading change	Continuity
	Leadership
	Leading change
	National level
	Personality
	Regional level
	Stability
	Team building
	Trust level
Environmental pressure	Delay
	Denial
	Energy drain
	Low morale
	Radical change
	Restructuring
	Financial
Supportive organisational culture	Achievement
	Challenging and changing beliefs
	Deep-seated assumptions
	Flexibility
	Hierarchies
	Openness
	Risk taking
	Role models
	Value base
Managerial–clinical relations	Blocking change
	Common ground
	Communication
	Honesty
	Identifying needs
	Trust
	Relationship building
Co-operative interorganisational networks	Boundary spanners
	Communication points
	Informal links
	Purposeful
	Sharing and learning activities
	Bargaining
Simplicity and clarity of goals and priorities	Conflict resolution
	Key priorities
	Organisational framework
	Patience
	Persistence
	Managing complexity
Change agenda and its locale	Pace of change
	Change in power balance
	Trust-level culture
	Regional-level culture
	National-level culture
	Workforce changes
	Opportunities

The researchers engaged in three iterations to arrive at the final deductive framework. The approach considered a pattern of association rather than direct causation between the independent and dependent variables. 128 The framework considers receptive conditions for change which are dynamic and reversible through changes in personnel or management action. The approach identifies patterns in processes and recognises emergence, possibility, precariousness and iteration. The framework is based on the following principles (1) change is studied over time in the context of interconnected levels of analysis; (2) change is considered in terms of the past, present and future; (3) the relationship between context and action is explored; and (4) change is considered to be neither linear nor singular. All qualitative data (e.g. interview data, focus group data, etc.) were coded against this framework.

During the next step of the analysis, the data were coded deductively against the NETS ‘three-legged stool’ (Vision, Compact, Method) to provide the NETS organisations with analysis that would correspond more directly to their transformational change ambitions. The framework was developed through an analysis of the SHA’s policy documents on NETS. The codes associated with the Vision were coded against the ‘seven no’s’ which were central to NHS NE’s approach to QI. The Compact codes were developed by looking at the policy documents that set out the mutual expectations between staff and the organisation. The psychological contract aimed to ensure that all staff and managers were clear about the expected approach and behaviours that should be demonstrated in the workplace. The codes developed for the Method focused on the lean tools that were adopted in the workplace and used in the RPIWs or equivalent improvement events for non-VMPS sites. An identical approach to developing the NETS ‘three-legged stool’ framework and the coding process as outlined above was adopted to ensure consistency and reliability. Again, all qualitative data (e.g. interview data, focus group data, etc.) were coded against this framework. The coding framework is shown in Table 5.

TABLE 5 - The NETS ‘three-legged stool’ coding framework

OEE, overall equipment effectiveness; QC, quality circle.

Code	Subcode
NETS Vision	No barriers to health and well-being
	No avoidable deaths, injury or illness
	No avoidable suffering or pain
	No helplessness
	No unnecessary waiting or delays
	No waste
	No inequality
	General NETS Vision
NETS Compact	Unhappiness
	Old Compact
	New imperatives
	New Compact
	No knowledge of Compact
NETS Method	Gemba Kanri (standardisation, 5S and visual management)
	Seven wastes
	Production and material control (Kanban Pull)
	Intercompany lean (single sourcing)
	Organisation for change (teamworking, continuous improvement and QC)
	TQM (focus on the customer)
	Cellular or line layout (flow)
	Ergonomics
	SMED
	OEE
	Andon
	Jidoka
	Smallest machine concept
	Fool proofing

Qualitative data analysis faces the challenge that other researchers may interpret the data in a different way, and this issue had to be addressed as part of the study. Miller and Crabtree129–131 argued that five iterative phases needed to be undertaken, namely describing, organising, connecting, corroborating/legitimating and representing the account. By following this approach, the crystallising, corroborating and legitimating aspects of the data analysis process were achieved.

In summary, the deductive frameworks were developed to initiate the investigation of transformational change in NHS NE. The first stage of analysis was to deductively code all qualitative data to these frameworks. Secondly, all qualitative data were then coded inductively, allowing themes and issues that the two deductive frameworks did not detect to be identified. This was an iterative process that occurred continuously throughout the 3 years of the study period.

The development of the Vision

The initiators of the NETS emphasised the development of a Vision to create a shared value system. It was an essential means to ensure that improvements were driven from the bottom up, as well as from the top down. In January 2011, one of the SHA’s most senior staff members made the point that although the NHS was adept at publishing and formally adopting standards for improved patient care, sound finances and better patient safety, it seemed to struggle to apply these effectively at the point of delivery. The Vision (regional and local) was seen as a central, enabling link between aspiration and action on the ground:

. . . it is crucial that the vision is shared, truly believed and firmly embedded in the minds of people and the very fabric of organisation thinking. Without this the purpose and goals to which people are striving become less clear and potentially misaligned.

NHS NE99

A 2010 NETS Coalition Board briefing on the history of the VMPS stressed the importance of the regional Vision (the ‘seven no’s’) as a psychological tool to strongly encourage a shift from a mindset that is tolerant of, and expects, errors and defects, to one that believes the perfect patient experience is possible. The Compact sought to shape the psychological contract between staff and the organisation in which they worked. Thus, both the Vision and Compact were mutually reinforcing and aimed to bring about transformational change through aligning stakeholders’ values in order to achieve a patient-centred organisation. The Method, the most visible component of the NETS though not the most important, provided a toolkit that facilitated the improvement of operations to achieve the Vision.

An organisation cannot usually adopt a complex management practice without shaping the Method to meet its specific context and requirements. Therefore, the way the NETS was adapted by trusts varied. Views varied concerning the study sites’ success in creating, sharing and embedding a vision that was more than just rhetoric or well-meaning words. However, nearly all of the sites had produced a written statement of the Vision. A number of sites incorporated these into highly visible graphics that were used on official documents and websites, displayed as posters in public areas, and even produced as staff badges.

Senior SHA leaders were convinced about the purpose of a clearly stated Vision as a unifying rallying point. However, there was variation between the study sites in terms of the widespread understanding of the Vision. Some interviewees said that, on becoming part of the NETS, their organisation made use of an existing Vision statement to avoid repeating an exercise that was considered to be already complete. In one case, this was a contributing factor in deciding not to adopt the VMPS version of the NETS, as explained by a senior director of a hospital trust: ‘. . . our plea was . . . we have just gone through a 3-year . . . assessment process and we have a vision . . . What we want to do is focus on implementing that . . . as opposed to we start again . . .’ (site 14, senior director).

However, some interviewees from the VMPS study sites commented that the development of the Vision, Compact and Method had to be considered as a holistic package. In VMPS study site 10, the alignment of the Vision with the Compact and Method was reinforced through the distribution of a prompt card that reminded staff of the Vision and its surrounding values, the behaviours linked to those values and a summary of key lean tools. A senior member of the KPO team saw the prompt cards as a way for junior staff to challenge upwards, to hold leaders to account: ‘If we’re working with junior staff I often challenge them to challenge their leaders, because if leaders are not following our trust Vision and Compact then I think staff have every right to question it’ (site 10, KPO team member).

Not all VMPS sites – even wave 1 pathfinders – made significant progress in embedding their NETS-inspired Vision throughout the organisation. The medical director of one site, which had focused on its drive to attain foundation trust (FT) status, commented that the Vision was understood by senior staff, but had not been shared sufficiently at all levels. This was because operational problems facing middle managers took priority over the dissemination of the Vision. In general, board members and senior managers appreciated the importance of linking the Vision and Compact and disseminating these throughout their organisations. This was the case even in a study site (02–05) that had decided not to pursue the VMPS version of the NETS after some initial interest. This was one of the NHS organisations disbanded at the end of March 2013. The acting chief executive officer (CEO) expressed the view that implementing the Vision and Compact was still important irrespective of the impending reorganisation.

The utility of the organisation-wide Vision, as well as the regional Vision, was recognised by many interviewees as a means to unite staff behind a ‘flag’. It acted as a memorable shorthand for a number of strategic objectives and communicated the organisation’s values to patients, public and other, external agencies. Some interviewees saw significant value in adapting the process of creating and communicating a Vision to organisational units or initiatives. This approach supported the development of business units within FTs with their own leadership teams, budgets and targets.

Compact development

It was recognised by NHS NE that a step change in its approach to change management was required to achieve transformational change. The Compact focused upon aligning people and organisations to the Vision. To quote NHS NE’s 2008 strategy Our Vision, Our Future, Our North East NHS: A Strategic Vision for Transforming Health and Healthcare Services Within the North East of England:99

Whilst necessary, developing vision is clearly not sufficient and the NETS approach insists upon the vision being complemented in equal measure with a compact that aligns culture and behaviour as well as consistent method for continuous improvement.

[The Compact] describes the unwritten rules, the behaviours and the signals that are sent by managers.

It explicitly sets out the ‘unwritten rules’ making them written and transparent, it sets out the expectations and behaviours that are required to be more effective in delivering change and it is mutually binding and enforced.

It is a reciprocal contract for the ‘give and the get’.

In defining the Compact in these terms, NHS NE explicitly recognised the critical nature of this particular leg of the ‘three-legged stool’, and therefore the role of people in delivering transformational change.

The following subsection examines interviewees’ perceptions of the role of the Compact within the NETS, how Compacts were developed and implemented, and how trusts sought to integrate the Compact with existing managerial practices, including the management of change.

The adoption and use of the Method

Although the SHA initiators of the NETS encouraged use of the VMPS as the Method of choice, it is noteworthy that early official documents often make mention of generic improvement methods:

What we require is change at every level, a focus on behaviours, culture and proven improvement methods and a wider recognition through the systematic use of the quality equation of the links between appropriateness, outcomes/effectiveness, quality of service/personalisation and elimination of unnecessary waste when assessing overall quality.

NHS NE99

This was reflected in the different QI tools adopted by the NETS organisations. These included the TPS-based VMPS, the Unipart Way, the NHS Institute for Innovation and Improvement’s ‘Productive Series’ and a variety of borrowings and adaptations from lean programmes as practised by a range of health-care organisations in the UK, Europe and further afield. All of these approaches were intended to promote and improve quality by

focusing on value-added activity, systematically removing waste (or non-value adding activity) and eliminating defects. It considers quality through the eyes of people and patients . . . to demonstrate the relationship between different features that impact on the value of care as experienced by our patients and people.

NHS NE99

A number of interviewees thought that, despite the promotion of all three elements of the NETS, the Method was generally overemphasised:

. . . I think we draw that NETS triangle as Vision, Compact, Method and Vision’s at the top. But we behave in a way that almost the Method is at the top and maybe that’s where we’ve got it slightly wrong . . . if your Vision and Compact’s in place you’ll find a Method.

Site 08, KPO

I think we’re very much focused on, you know . . . Method – I think we know a lot about the Method, we know about the tools and, you know, the RPIW’s not the be all and end all.

Site 11–13, service improvement manager

The reasons given for this varied. One phase 3 interviewee complained that the very success of one VMPS RPIW in reorganising and streamlining an important and highly visible care pathway had led to a rush of enthusiastic interest in repeating the process in related areas of the organisation. However, this meant that the staff who were involved in the original RPIW were unavailable to run the standard 30-, 60- and 90-day report-outs, and the share-and-spread activity was conducted piecemeal. This was borne out to some extent by the difficulty the research team encountered in many of the VMPS study sites in obtaining the 30-, 60- and 90-day documentation that was supposed to follow each RPIW.

A few interviewees, including some from non-VMPS sites, said that it was tempting to achieve ‘quick wins’ by using the range of available lean tools. This was not seen as necessarily bad practice – indeed, the VMMC recognised that this was a powerful way to achieve early acceptance of the value of lean thinking among staff – except in so far that it could divert attention away from the need also to pursue development of the Vision and the Compact.

Finally, the Method was clearly associated with recognition and reward. During the period of our study, several NETS organisations were nominated for, and won, Health Service Journal and Lean Academy awards. The SHA and the central NETS team regularly organised large-scale report-out sessions at which up to 100 or more staff were present to showcase process improvement. Research team members attended several of these events, and observed that Compact or Vision development appeared to play a peripheral role. The trust’s Vision might be a visible element, but it was not celebrated per se. Changes in staff behaviour and trust culture that had an impact on patients were sometimes highlighted, but the emphasis was mostly on process improvements and changes to the physical environment.

Summary

The initiators of the NETS emphasised the development of a Vision to create a shared value system. It was an essential means to ensure that improvements were driven from the bottom up, as well as from the top down. An organisation cannot usually adopt a complex management practice without shaping the method to meet its specific context and requirements. The way the NETS was adapted by trusts varied. Senior SHA leaders were clear about the purpose of an explicit Vision as a unifying rallying point. However, there was variation between the study sites in terms of the widespread understanding of the Vision. It was acknowledged, primarily by the VMPS study sites, that the development of the Vision, Compact and Method had to be considered as a holistic package. It was also acknowledged that a number of the study sites had made significant progress in embedding their NETS-inspired Vision throughout the organisation.

In some trusts there were large numbers of staff involved in developing Compacts. In these cases there was a low level of cynicism and little resistance to their development and dissemination. However, in most trusts the Compact was not given equal weight compared with the Vision and Method. This was despite widespread recognition of the significance of the Compact among senior staff. This research found that trusts see value in addressing the behavioural aspects of the employment relationship through the use of ‘gives and gets’ and explicit statements on standards of behaviour. Some trusts have taken a strategic decision to integrate work on the Compact with other aspects of the way they manage staff. The Compact has been used to develop a supportive organizational culture including managerial–clinical relations in order to facilitate transformational change that is at the heart of the NETS. 123

Despite the promotion of all three elements of the NETS in some sort of equilibrium, the Method was generally overemphasised. It was noted that it was tempting to achieve ‘quick wins’ by using the range of available lean tools which could inadvertently divert attention away from the need to also pursue development of the Vision and the Compact. The Method also became associated with recognition and reward. The VMPS RPIW method was highly standardised, whereas non-VMPS sites were more flexible in tailoring the implementation of their QI programme to suit their context. It was found that the Method adopted was less important than the clarity of purpose, i.e. moving towards achieving the Vision. In the early years of NETS, the VMPS sites focused more on running improvement events and training workshops for team leaders. It was clear that sustaining improvements was more important than the actual Method adopted. Some individuals saw the Method as a panacea to solve all problems and did not always appreciate its limitations.

Case study 1: purposeful inpatient admission rapid process improvement workshop

This case study concerned an early NETS RPIW undertaken by one of the wave 1 NETS pathfinder organisations. It took place in early 2008 – prior to the start of the research study – and thus provided a baseline comparison with later RPIW case studies. The RPIW was an attempt to apply the NETS VMPS methods to problems associated with inpatient wards throughout the trust; these are summarised in data obtained from the RPIW project form (Table 9).

Study site 10 was one of the first organisations in NHS NE that agreed to become a wave 1 pathfinder and train staff in VMPS methods of quality management. Senior leaders were therefore concerned to apply the NETS version of the VMPS to areas of the trust’s work that were likely to result in successful, demonstrable improvements to quality, patient safety and staff satisfaction. The process for inpatient admission and subsequent treatment was recognised to be in need of overhaul and improvement. Furthermore, it was acknowledged that several previous improvement attempts had failed, reportedly because previous projects had been under-resourced or not sufficiently supported by key staff; a successful RPIW with sustained evidence of a better service would therefore achieve significant recognition and have an impact throughout the organisation. As a senior clinician commented:

But I suppose the pressing issues for our area and why we got involved is there’d been a number of issues for a number of years, that there’d been several improvement events and techniques already tried and had failed . . . much slower processes to a certain degree, ones that just dragged on over months with no changes being made . . . that again just fizzled out so the changes weren’t made.

Site 10, clinical psychologist

Another main driver for tackling inpatient admissions was the 2-year timetable for moving some wards to new accommodation. This was seen as an opportunity to introduce changes to processes and to relationships between staff, patients and families, in advance of moving to a different environment. As the same interviewee noted:

We wanted for the first time to be able to look ahead and say right we’re moving into a new unit in 2 years’ time, do we want to move in doing the same old thing we’ve been doing here that’s not working or do we actually seriously want to plan ahead and be able to move into this. But there was also that awareness of what the future environment’s going to look like, there’s going to be a reduction in beds that we’re going to have to face. If we could improve something, maybe that wouldn’t be as painful, and that’s how it turned out to be. We could reduce the number of beds pretty painlessly compared to other places . . .

Site 10, clinical psychologist

Patient and staff safety was also a major concern, particularly the number of incidents of violence and aggression, and the consequent use of control and restraint techniques. Unsurprisingly in these circumstances, the trust received a number of complaints directly from patients and families, and via the Patient Advice and Liaison Service, in the months leading up to this RPIW. Staff sickness rates were felt to be unacceptably high over the year preceding the RPIW – on occasions reaching 10–15% of staff working time – which in turn caused a high number of overtime hours to be worked.

In general, it was clear to senior leaders and to staff working on the wards that the processes involved in inpatient admission, treatment and discharge were inefficient, contributed to a chaotic environment, caused a considerable waste of resources, left staff dissatisfied and were not focused on the best outcomes for patients.

Case study 2: community psychosis ‘superflow’ rapid process improvement workshop

This case study concerned a ‘superflow’ RPIW event in study site 10 that took place in early 2011, almost 3 years after case study 1. The term ‘superflow’ indicates that, in effect, a number of interlinked RPIWs were carried out concurrently; in this case, three RPIWs, corresponding to distinct elements (referral, assessment and discharge) of the patient pathway for psychosis in the adult mental health service.

The scope of the superflow RPIW was ambitious (Table 10). The week-long event required considerable organisational and logistical planning, together with the active involvement of a large number of staff. In itself, this demonstrates emerging confidence and trust in using the study site’s quality improvement system (QIS). The key problems addressed were multifactorial: a lack of standard work across the whole pathway; variation in practice in different geographical locations; poor communication with patients, carers, family members and other NHS staff; and a lack of reliable and useable data on quality and performance.

Rapid process improvement workshop outputs and outcomes

The scale and scope of the superflow RPIW were reflected in a long list of outputs, initially in the form of standard process description (SPD) and patient management documentation (Table 11).

TABLE 11 - Output documents from psychosis superflow RPIW

CPA, Care Programme Approach; DNA, did not attend.

‘5 P’ here refers to an integrative and multidisciplinary model that captures the factors that affect the patient’s diagnosis and treatment.

Referral	Assessment	Discharge
DNA letters × 2	12-week formulation letter	Absence recording sheet
SPD first contact	5 P’s formulation guidelinesa	Case note recording for activity (template and example)
SPD DNA management	5 P stress vulnerability diagram	CPA GP letter
SPD initial referral	Initial formulation flow chart	CPA review standard format
SPD referral allocation	Intervention plan template	Discharge checklist
SPD visual control board	Letter to GP after initial appointment	Pre-discharge letter to GP
SPD transfer of care	Pictorial team formulation	SPD CPA review
SPD transfer	Psychosis assessment and discharge	SPD formulation meeting 6-month review
Visual control board	Psychosis superflow formulation documentation	SPD loss of contact or disengagement
	SPD assessment appointment 1	SPD medication review appointments
	SPD assessment appointment 2	SPD planned discharge
	SPD formulation initial meeting	SPD service user declines service
	SPD formulation meeting (care co-ordinator)	SPD staff absence
	SPD formulation meeting (medical staff)	SPD treatment activity recording
	SPD psychological assessment
	Standard care plan template

All of the outputs listed in Table 11 were produced in support of a standardised care pathway, as shown in Figure 4.

FIGURE 4.

Psychosis superflow overview: the standardised post-RPIW care pathway (adapted with permission from a study site diagram). CPA, Care Programme Approach; DNA, did not attend. Information supplied and permission granted by a senior member of staff in study site 10’s QI team, 2011, personal communication.

The longer-term outcomes of the superflow RPIW were almost as varied as the immediate outputs summarised in Table 11. Four months after the superflow event had taken place, those staff closely associated with the implementation of the RPIW reported a number of positive changes that were directly attributable to having standardised processes supported by common documentation:

• Psychosis team care co-ordinators were better equipped to deal with their workload and staff sickness absence had reduced. An initial concern that staff would find the new and unfamiliar ways of working stressful proved unfounded.

• Acceptance of standard work had highlighted gaps in the structures of the multidisciplinary teams in different locations, but had also empowered staff to make requests for – and obtain – team members with the professional skills to fill those gaps.

• The superflow RPIW had demonstrated the need to make changes to the study site’s electronic record-keeping system, which had proved occasionally unwieldy and ill-suited to the new ways of working. In particular, the system had been shown to be poorly equipped to provide historical evidence of a patient’s clinical history and previous treatments, and this was being addressed as a priority.

• Staff were now often able to move from one team to another without having to learn new protocols.

• Patients and staff had a much clearer understanding of the purpose of admission and treatment within the psychosis pathway.

• There was positive feedback from GPs, who were reported to be pleased with the new forms of communication with patients.

• Patients assessed as needing treatment elsewhere, either in primary care or in mental health secondary care, were transferred in a timely way, with no waiting for handovers between different agencies.

Not every outcome of the superflow RPIW was reported as positive. Staff noted, for example, considerable variation in the implementation of the standard work across different locations. This was attributed to a combination of factors, including a historical lack of commitment to change in some of the psychosis teams and some failings in team leadership. The formulation of standard work had also highlighted gaps in the study site’s existing metrics and data, and some staff expressed doubts that these problems would be overcome quickly.

Case study 3: surgical pathways assessment area (abdominal pain) rapid process improvement workshop in an acute hospital

This case study was based on a RPIW event that took place in an acute hospital study site in October 2011 (Table 13). The study site was a wave 1 NETS pathfinder that had contributed to the NETS programme from its inception.

Case study 4: other North East Transformation System environments

Case studies 1, 2 and 3 concentrated on specific RPIW events in two wave 1 provider organisations that used the VMPS as their method for change and QI. This case study brings together a number of considerations that derive from a study of the NETS in a different set of environments: a wave 2 VMPS pathfinder; a commissioning organisation; and two provider trusts that employed a range of different, non-VMPS lean tools as their preferred method.

Summary

This chapter has presented four case studies chosen to illustrate the application of the NETS as a programme for QI activities. Case studies 1 to 3 were chosen to provide some background and context against which to view the results of the ITS analysis of five RPIWs. Case study 4 brought together some higher-level observations about a number of different types of NETS organisations, including a wave 2 VMPS pathfinder, NETS in a health-care commissioning environment, NETS in a hospital trust that initially used the ‘Productive Series’ and NETS in a hospital trust which made use of an eclectic suite of methodologies. From this analysis it was identified that the Method seemed to be less important than the ability to bring about improvements. These had to be driven by the people who had a key understanding of the challenges faced, namely the staff, who needed training in lean tools and the support of colleagues and senior managers. It appeared that the non-VMPS sites were able to pick and choose which improvement methods to adopt and when, implying a freer application to enable more flexible working in comparison with the structured RPIW format which the other sites were required to follow.

A key outcome of the case studies reviewed was an emphasis on removing inefficient processes and practices, standardisation, identifying if the ‘system’ was operating as planned (visibility) once any improvement had been made, and monitoring the outcomes. The QI activities often resulted in an improved understanding that measurement was an important part of any Method adopted. In addition, staff often also realised that suitable metrics were not available, or that the data were of poor quality.

Impact on performance

The range, scope, volume and complexity of the multiple NETS activities carried out in the study sites from 2009 to 2012 made it necessary to adopt a sampling approach. Interviews, focus groups, documentary materials and observational fieldwork were used to evaluate the impact of the NETS on patient safety; the quality of care; the patient experience; the efficiency of processes; productivity; and staff satisfaction. The interviewees in phases 1, 2 and 3, whether individual or in focus groups, were remarkably congruent in their assessments of the effects of the NETS on the performance of their respective organisations and on the NHS region as a whole. Most NHS staff had experienced many reorganisations (locally, regionally and nationally) and had been introduced to a variety of change programmes; they were therefore disinclined to be overly optimistic about the likely impact of the latest initiative.

North East Transformation System and the quality of care

From its inception, the NETS focused on making measurable improvements to clinical care. A number of interviewees said that this emphasis was adopted in large part because of a need to convince initiative-weary doctors and nurses of the value of the principles of the NETS, and to gain their commitment to the programme. One study site exemplifies this view particularly well:

I was particularly keen that in the early days of this we didn’t attach anything to do with efficiency or cost savings or anything like that because I think that it’s not going to hook clinicians in at all. They’re interested in improving the quality, and to be honest I think we should be as well.

Site 06, business development manager

. . . we tried to make it very clear it was about changing hearts and minds, and it was about quality, increasing quality for patients, which is obviously very important for our clinicians.

Study site 06, QI manager

Most interviewees saw quality and safety as very closely linked. Therefore, the views expressed above in respect of patient safety (see Embedded patient safety) were often echoed when staff reflected on the NETS and improvements to the quality of care. The lean programmes (VMPS, Unipart Way or the Productive Ward) taught staff that QI would arise from removing and reducing waste, focusing on adding value and concentrating on seeing processes from the patient’s point of view.

The NETS was effective in reinforcing and encouraging a whole-system view of quality. For example, back office staff could understand how their work could contribute to QIs on the front line. An administrator in study site 07, for example, explained that his first experience of NETS training had resulted in the establishment of a better process for managing insurance claims. He estimated that potential savings of several hundred thousand pounds could be achieved across the whole organisation. He explicitly linked this cost saving to an improved service for patients.

The message about the primacy of QI was not universally accepted. During one observation of a series of linked VMPS improvement activities, an attempt to make use of 5S techniques to create a better working environment was rebuffed by a number of staff. They were unwilling to accept the suggestions and pleas from their colleagues. QI staff were aware that the department had an embedded culture of resisting change. However, they regarded the failure of this project as a positive reason to review how they implemented the organisation-wide Compact.

Some interviewees expressed the view that the NETS insistence on QI, rather than, say, cost saving or raw productivity, acted as a useful means to ‘flush out’ colleagues who were not pulling their weight, or who were overly concerned with their place in the organisation’s hierarchy. Two staff gave the example of a reorganisation of practices in their department which was designed to more effectively meet the needs of a particular cohort of vulnerable patients. The reorganisation required more and better communications between different professional roles. This was difficult for two senior team members to accept, who were subsequently moved to a different part of the organisation. The planned reorganisation was then successfully implemented and team morale improved considerably. In effect, the focus on quality of care had positive consequences for both patients and staff.

The role and development of the North East Transformation System Coalition

The ‘NETS Coalition’ was the term used to describe the collective of NHS organisations that were involved in the NETS programme. It included organisations committed to involvement in wave 1, VMPS pathfinders and those likely to take part in waves 2 and 3, as well as non-VMPS organisations that had agreed to follow the Vision/Compact/Method approach to QI.

In practice, early NETS Coalition activities were directed largely by the representatives of the pathfinders (wave 1, and later wave 2) who were committed to VMPS training. These representatives attended meetings as the NETS Coalition Board, chaired by senior SHA staff. The name of this group – the co-ordinating committee of the NETS, in effect – changed over time, from the ‘North East Transformation Coalition Board’ to the ‘VMPS Coalition Board’, and eventually the ‘NETS Coalition Board’, to reflect the changing activities and interests of its membership. For simplicity, we have used the terms ‘NETS Coalition’ and ‘NETS Coalition Board’ throughout.

Administrative support for the NETS Coalition Board, as well as practical resources (funding, desk space, meeting rooms, and so on), were provided by the SHA from 2007 to 2010. The SHA also provided some resources for managing contract negotiations with VMMC and Amicus; co-ordinating VMMC training; arranging training visits to Seattle, WA, and Japan; and KSL activities. Funds were made available to support the region-wide report-out sessions that were an important feature of the first years of the NETS. They provided the opportunity for large numbers of NHS staff to see the outcomes of improvement activities carried out by many different organisations. Staff gained direct experience of speaking at a region-wide event, using a common language of improvement.

The NETS was a complex and large-scale transformational change initiative. Agendas and minutes of the NETS Coalition Board provided a high-level view of the impact of the NETS across the region and documented variations in the number of NETS Coalition members over time. There were significant changes following the announcement of the NHS restructuring in July 2010. The key points from these documents are summarised in Tables 17 and 18 (presented separately to assist legibility).

Communication via visual management in the North East Transformation System organisations

Visual management was an important component of interventions. One of the key outcomes was increased visibility of processes and performance. This was the case for a range of situations: front office or back office; ward or full pathway; and whether using RPIWs or other methods. Increased visibility allowed staff to be more informed, which helped them to fulfil their role effectively. For example, one interviewee highlights the visibility of a patient’s status in the following terms:

We have yeah, we do have actually the big wall thing that as I say we put all of the referrals on to, and it was decided that . . . the way we did it previously we didn’t, but this is just as you say a visual way of sort of being able to see straightaway how many patients there are, who they’ve been allocated to, how long it’s been since they were referred, and how long it took to get them through the system and things like that. So yes, we have a big wall board in our office, in the admin office, because we’re like all admin and it’s all in there.

Site 10, medical secretary

Another interviewee identified the advantage of communicating the plan of activities using visual management:

And so for example you can very clearly as you walk round the hospital, you can see the wards that have done it, because what you’ll see as before and after is just de-cluttering and the store cupboard is better and so on. And in some of them, for example the staff will decide to put the prices of things on the store cupboard shelves so that you’re making an instant choice knowing the price of it.

Site 01, senior board member

Visual management allowed staff to identify the key information that they required and needed to communicate to others. This in turn allowed staff to compare their performance (internally and externally):

Yes, we have, like I say I think we’ve got tiers of information that we collect. I mean obviously . . . internally each service has its own sort of barometer in terms of service requirements, and we use things like visibility walls et cetera to share that with the staff. And it can vary, I mean for example it can be things around uptake of the flu, you know, immunisation, sickness levels, we vary it as required. But again we’re very immature I think in terms of where we’re at with that, compared to maybe our colleagues in the foundation trust who have reams of information they can pull out at the press of a button.

Site 11–13, business manager

Improving the visibility of information and data has also helped improve the quality and accuracy of reports:

We didn’t know because unfortunately prior to doing the RPIW we didn’t have a fantastic reporting regime. The culture of the trust was quite poor where staff felt that we hadn’t got time to fill in a large A3 piece of paper for nothing to happen, for the feedback to be negative or non-existent. So the reporting culture was very poor. So as a result staff feel more empowered, just hearts and minds really. Reporting as I say has gone up approximately 75% now. So it means our workload has gone up 75% because we are, even though the forms are shorter, we’re getting 75% more reports, but we now reply to everyone, give feedback, acknowledge receipt of the incident report.

Site 07, finance administrator

At the opposite end of the continuum, visibility can highlight where data and information can take time to filter through to the people who need it. The following comment makes the point:

Trying to think of another example where we might have had to, I think that it’s, again it’s not a symptom of specifically health but it’s probably a reflection of community services in the NHS is that we find it difficult to provide performance information in as timely a way as we would like.

Site 02–05, senior board member

The sheer volume of reporting and associated data made the collation and management of information important:

Well it doesn’t always happen, but what should happen is in each organisation they should basically have the high-level, the very high-level value stream maps. Basically they should know what’s happening in the organisation, but we know that no it doesn’t happen, so you’ve got your high-level maps, and in terms of strategic goals for the organisations, it could be for yearly goals for example, they should be looking at their vision and what are they looking to achieve in the near future. And as a result of that in terms of scoping and planning you will get your topics so to speak from that. And in working with and if you do have a kaizen promotion office and you do have champions in various departments across the organisations, it’s a way of tapping in and then saying right we’re going to look at a particular process.

Site 09, NETS co-ordinator

Visual management helped to identify key performance indicators and the costs of providing a high-quality service:

Your performance element is very much around, why we did this was because we’ve got staff working in some of our services and some of the staff sort of higher up in those services as such might know all about the business side of that service, what the key performance indicators are, all that sort of thing, but actually sometimes in some services quite a few of those staff weren’t aware of what the key performance indicators were, how much things were costing. Things cost an awful lot of money in the NHS, and actually making that very visible and very visual kind of highlights to every member of staff what they’re striving to achieve with the view that if everybody understands that, then people are focused on what you need to do to meet the vision mission and aims of the organisation.

Site 06, QI manager

Standardisation of products and services is a core component of lean. It reduces variation so that a product or service is delivered exactly the same way every time. The use of visual management highlighted variations in the approaches developed by different RPIW teams. This suggests that a standardised approach would be beneficial:

There were some other areas like cardiothoracic ward and orthopaedic wards and surgical wards that also put their own patient status at-a-glance boards in. And then what we realised was actually we were creating a risk across the organisation because all of the symbols were different. They were all producing their own system which if you worked across the trust was just going to cause all sorts of problems. So we knew we needed to do something about that.

Site 01, nurse manager

This section has looked at how the Method applied visual management techniques. Through increasing visibility, staff were able to obtain information that helped them perform their roles more effectively. Both VMPS and non-VMPS sites achieved similar benefits.

The North East Transformation System and patient involvement

Baggott141 noted that patient and public involvement (PPI) had received considerable attention in the NHS since the late 1990s. PPI should inform decisions concerning the allocation of health and health-care resources, health policy (at local, regional and national levels) and system planning. However, despite the ambitions of successive administrations, meaningful and effective PPI has often proved elusive. Baggott identified five barriers to successful PPI:

1. poor understanding of the multidimensional nature of PPI

2. a confusing and complex mixture of different initiatives to encourage PPI

3. PPI structures that lack ‘independence and integrity’

4. a lack of evidence-based analysis of the weaknesses of patients and the public in the face of powerful forces within the health-care system

5. ‘. . . institutional changes may be insufficient to promote genuine empowerment of citizens . . . cultural change is probably more important . . . Small-scale institutional changes may achieve more than completely new structures, particularly if existing institutions can be modified to involve and engage with patients, users and carers more effectively’.

The fifth point is highly relevant to the findings in relation to the NETS and PPI. The NETS was conceived to encourage staff to see the world through the eyes of patients, their families and carers, and to focus on making changes to processes only where they were of direct benefit to these ‘customers’. It was a ‘bottom-up’ approach, with senior leadership commitment and resourcing supplied ‘top down’, and it was promoted as achieving transformational change through small-scale but continuous improvements to processes.

The NHS staff who had visited the VMMC in Seattle, WA, or who had had exposure to VMMC trainers, were generally impressed by the benefits accruing from active patient involvement in improvement activities. One interviewee summed up his experience of seeing PPI in action in Seattle thus:

[There was] a great emphasis on getting patient feedback in very simplistic ways. It could just be how did we do today, and it [could] just be putting a counter in a box saying very good, good, poor . . .

Site 10, service development manager

The same interviewee remarked that, over time, the NETS emphasised using metrics to inform and sustain change. It also encouraged the involvement of patients and the use of patient feedback to make judgements about clinical outcomes following the redesign of care pathways.

A VMPS KPO manager (site 08) also stated that the NETS had been a spur to take PPI seriously, to build networks and to include service users on project boards. Within the organisation, the profile of staff charged with promoting PPI had risen considerably: ‘But I work quite closely with our . . . patient involvement lead, whereas perhaps 18 months ago I knew her name but now we’re regularly in contact’ (site 08, KPO manager).

Some of the non-VMPS study sites also had strong messages about the importance and value of involving patients in the redesign of processes and working practices. However, in these cases the PPI appeared to focus more on ward-level activity, rather than policy setting or influencing strategic objectives. The emphasis here was on promoting a safety culture:

. . . we’re using these information centres now for each, each ward [shows visual control board to interviewer] . . . And this bit here is the real change, because that’s about patient and relative involvement and what patients and relatives can do to actually make [trust name] a better place to be in and a safer place to be in.

Site 01, nurse manager

The same interviewee saw the mobilisation of patients, carers and family members as a way to augment staffing levels and provide another perspective on safety concerns. A senior nursing director from another study site (site 14), which made use of an eclectic array of improvement methodologies, felt that PPI had been integrated into QI initiatives for ‘some time’, and gave an example of involving children in decisions about their health care:

In child health, we decided 18 months ago one of our priorities was doing something more locally with children, and that’s a raging success . . . [A child] got involved in this group . . . and as a result of that he’s coming to clinic, he’s talking about what they want in the clinic appointment and their own handling of their illness.

Site 14, senior nurse manager

These accounts of PPI seem to offer some evidence of the barriers identified by Baggott141 being overcome, at least partially. In some cases, the NETS appeared to have encouraged staff to think harder about PPI as a multidimensional activity. A single, patient-centred QI programme instead of a complex mix of initiatives may have helped to create a coherent focus that gave it the independence status and credibility that would otherwise have been missing. In relation to Baggott’s fifth barrier, the NETS concentrated on cultural change and small-scale, incremental improvements.

It was not possible to conduct interviews directly with patients. The evaluation of the impact of the NETS in terms of PPI was therefore based upon staff interviews. Some proxy measures found in documentary materials such as board reports and supporting documents for awards were also used. These secondary sources do not, on the whole, present as positive a picture as the interview data. In part, this may be because the learning curve involved in setting up PPI was steep. The study sites may not have felt confident in claiming success in the early years of the NETS. In addition, many early improvement activities were concerned with internal processes, where it would have been difficult (or impossible) to put PPI into practice. Patient stories feature frequently in annual reports and proposals for awards. However, without comprehensive and consistent reporting of how patients and the public were engaged in improving processes and pathways, it remains difficult to judge the effectiveness of the NETS in this regard.

Interrupted time series analysis of selected rapid process improvement workshops

As described in Chapter 4, Data and data sources, the data obtained for the ITS analysis were analysed using the repeated measures approach recommended for short time series. The detailed findings from each analysis are provided below [see Site 09 surgical pathway (abdominal pain), Site 10 purposeful inpatient admission rapid process improvement workshop and Site 10 community psychosis rapid process improvement workshops (referral, treatment, discharge)]. A summary of the key findings for each RPIW included in the ITS is provided in the next section.

Site 09 surgical pathway (abdominal pain)

The main data set examined for this analysis was for all attendees at A&E presenting with abdominal pain, for the period 1 September 2009 to 30 September 2012. The analysis of emergency surgery theatre start and finish times used a different data set from the theatre system and included all patients (see Table 6 for more details of data sets and data definitions). The RPIW week was in September 2011. Attendance at A&E by patients with abdominal pain increased over the period of follow-up (Figure 5). There was a corresponding increase in the number of patients admitted but this was not reflected in the number of patients being X-rayed.

FIGURE 5.

Attendees at A&E presenting with abdominal pain from 2009 to 2012. Dashed line indicates the date of the RPIW intervention.

Proportion of patients X-rayed in accident and emergency

The proportion of patients X-rayed in A&E was investigated using logistic regression analysis. The first stage of the analysis was to ascertain whether or not there was any trend in the proportion of patients X-rayed over the entire period of follow-up. This baseline model suggested a significant downwards trend over time. The reduction in odds of an X-ray corresponding to a change of 1 year was 0.85 [95% confidence interval (CI) 0.81 to 0.89]. The second step was to determine whether or not there was a shift in the general level of X-ray requests following the RPIW in September 2011. The estimated change in the odds of requesting an X-ray was 1.18 (95% CI 0.99 to 1.40), which was of borderline significance (p = 0.06). The final step was to investigate whether or not there was a different trend after the RPIW compared with before. This produced a significantly better model. Prior to the RPIW, the reduction in odds of requesting an X-ray corresponding to an increase in time of 1 year was 0.85 (95% CI 0.77 to 0.94); following the RPIW the corresponding reduction in odds was 0.47 (95% CI 0.37 to 0.61). The difference in slopes was highly significant (p < 0.001).

Time from arrival in accident and emergency to being X-rayed

The time taken for patients to be X-rayed was analysed using linear regression. On average patients spent about 2 hours in A&E before being X-rayed. Fitting a linear trend at step 1 suggested an annual increase in time to being X-rayed of 10.0 minutes (95% CI 7.0 to 13.0 minutes) over the period from September 2009 to September 2012. Adding a difference in mean corresponding to the introduction of the RPIW at step 2 suggested that the RPIW had the immediate impact of reducing the time to being X-rayed by 11.5 minutes (95% CI 2.0 to 21.1 minutes). The third step in the analysis was to fit different slopes before and after the RPIW. The difference in the gradients was significant (difference = 17.3 minutes per year, with 95% CI 1.1 to 31.6 minutes per year). This final model suggested that after an initial impact of the RPIW the change was not sustained; the rate of increase in time to being X-rayed after the RPIW was greater than that before.

Checking the fit of the regression model, it was noted that there were a number of suspiciously short and long times recorded in the data. To investigate the impact of these outliers, the analysis was repeated using a plausible range of times defined as being between 15 and 600 minutes of arrival in A&E. The difference in slopes before and after the intervention was no longer statistically significant. The revised model suggested that the impact of the intervention was a reduction of 10.3 minutes in the time spent before being X-rayed (95% CI 2.7 to 17.9 minutes).

Proportion of accident and emergency attendances with abdominal pain admitted to hospital

Whether or not a patient was admitted was investigated using logistic regression analysis. Fitting a linear trend (step 1) suggested that over time (September 2009 to September 2012) the proportion of patients admitted increased; each year the odds of a patient being admitted increased by a factor of 1.13 (95% CI 1.08 to 1.18). Fitting an impact of RPIW at step 2 suggested that the trend observed in step 1 could be attributed to a general increase in the likelihood of admission following the RPIW. The odds of admission increased by a factor of 1.30 (95% CI 1.12 to 1.51). Fitting a more complex model with a different trend before and after the intervention (step 3) did not generate a significant improvement in fit. The most parsimonious model is one that only includes an impact of the RPIW. This estimated impact of the RPIW based on this final model is an increase in the odds of admission by a factor of 1.26 (95% CI 1.16 to 1.36). In practice this corresponds to a change in the proportion of patients being admitted from 34.9% to 45.2%.

Proportion of admissions who receive a surgical procedure

Approximately 14% of patients who were admitted went on to have a surgical procedure during their stay (Figure 6).

FIGURE 6.

Proportion of patients admitted who went on to have a surgical procedure.

Whether or not a patient underwent a surgical procedure was analysed using logistic regression. Fitting a linear trend (step 1) provided some evidence of a downwards trend over time, with the odds of a patient having surgery changing annually by a factor of 0.91 (95% CI 0.82 to 1.00; p = 0.06). Adding an impact of the RPIW (step 2) suggested that the odds of surgery following the RPIW changed by a factor of 0.74 (95% CI 0.53 to 1.03; p = 0.08). Fitting a different trend before and after the RPIW (step 3) did not improve the fit of the model.

It was noted that, at step 2, fitting an impact of the RPIW appeared to explain the trend observed in step 1. With a term corresponding to the RPIW included in the model, the estimated background trend was no longer significant (p = 0.69). This suggested fitting a more parsimonious model with just a reduction in the likelihood of a surgical procedure following the RPIW. Removing the trend suggested that the impact of the RPIW was to reduce the odds of patients having surgery by a factor of 0.78 (95% CI 0.64 to 0.95).

Time from arrival in accident and emergency to having a surgical procedure

Time from arrival in A&E to having a surgical procedure was heavily skewed (Figure 7).

FIGURE 7.

Distribution of time to surgical procedure (n = 518). Mean 2413.07 minutes, standard deviation 3607.403 minutes.

Removing cases where time to surgery was more than 2 weeks and taking a log transformation provided a variable that had a more symmetric distribution (Figure 8). The monthly means of the log-transformed variable are plotted in Figure 9.

FIGURE 8.

Distribution of time to surgical procedure (n = 524). Mean 6.93 minutes, standard deviation 1.491 minutes.

FIGURE 9.

Log-time to procedure by month of follow-up.

The log-time to procedure was analysed using linear regression. At step 1 there was no suggestion of a linear trend; the annual change in log-time to procedure was 0.01, with 95% CI –0.14 to 0.15. Adding an impact of the RPIW (step 2) suggested that it may have produced a very slight change in log-time to procedure (change = –0.53, 95% CI –0.99 to –0.07). However, by fitting a model with an impact of the RPIW with no annual trend the estimated impact fell to –0.18 (95% CI –0.45 to 0.10). Finally, a more complex model with a different trend before and after the RPIW suggested that the slope after the RPIW (–0.48, 95% CI –1.17 to 0.21) was less than the slope before the RPIW (0.35, 95% CI 0.08 to 0.62) (difference in slopes = –0.83, 95% CI –1.57 to –0.84). There was very little difference in terms of goodness of fit between the alternative models. Any impact of the RPIW was fairly modest when considered alongside the natural variability in this variable.

Proportion of admissions who have an ultrasound scan

Thirty-one per cent of patients admitted had an ultrasound scan. The monthly data varied considerably over the period of follow-up (Figure 10).

FIGURE 10.

Proportion of admissions who had an ultrasound scan by calendar month.

The likelihood of a patient having an ultrasound scan was investigated using logistic regression. Fitting a linear trend (step 1) suggested that, annually, the odds of a patient having a scan increased by a factor of 1.06 (95% CI 0.98 to 1.15; p = 0.13). Adding an impact of the RPIW (step 2) suggested that after the RPIW the odds of a patient having a scan changed by a factor of 0.77 (95% CI 0.60 to 0.99; p = 0.04). Fitting a different trend before and after the intervention (step 3) suggested that the RPIW had an impact on the proportion of patients receiving an ultrasound scan. Prior to the RPIW, there was an increasing trend in the proportion of patients who had an ultrasound scan. After the RPIW the trend was reversed, and over time fewer patients had an ultrasound scan (the difference in slopes on the logistic scale was –0.508 with 95% CI –0.884 to –0.132; p = 0.008).

Inpatient ultrasound: times from admission to ultrasound

The time to ultrasound was skewed and typically between 1 hour and 1 week. A plot of the geometric means by calendar month is shown in Figure 11.

FIGURE 11.

Geometric mean time to ultrasound by calendar month.

The time to inpatient ultrasound was analysed using normal regression after undertaking a log transformation. The distribution of log-transformed data is shown in Figure 12.

FIGURE 12.

Distribution of log-transformed time to ultrasound scan (n = 1201). Mean 7.42 minutes, standard deviation 1.025 minutes.

Fitting a linear trend (step 1) across the entire period of interest suggested a slight annual increase in log-time to ultrasound of 0.06, with 95% CI 0.01 to 0.13 (p = 0.07). In step 2, when an impact of the RPIW was added, the estimated change in log-time to ultrasound was 0.05 with 95% CI –0.15 to 0.26 (p = 0.62). Finally, fitting a different slope before and after the intervention (step 3) did not produce a significant improvement in fit. There was no evidence that the RPIW had a clinically significant impact on time to ultrasound.

Emergency theatre start (between 08.00 and 10.00) and finish (before 20.30) times

One of the aims of the RPIW was that the period when routine procedures (non-‘life or limb’) in the emergency theatre are undertaken should commence prior to 10.00 and finish before 20.30. The data set included the start and finish times of all procedures but there was no variable to indicate which procedures were ‘routine’ and which were ‘life or limb’. A pragmatic solution to this problem was to regard any procedure that commenced either prior to 8.00 or after 20.30 as ‘life or limb’. These procedures were then excluded from the analysis. For each day, it was determined whether or not the first ‘routine’ procedure started before 10.00 and the last finished before 20.30.

The target of starting before 10.00 and finishing before 20.30 was achieved on 47.5% of the 1119 days considered. There was considerable variation between days of the week, as shown in Table 20. Differences between days were statistically significant (χ62 = 32.6, p < 0.001).

TABLE 20 - Proportion of occasions on which target was achieved, by day of the week

Day	Start before 10.00 and finish before 20.30?		Total
	No, n (%)	Yes, n (%)
Sunday	58 (36.5)	101 (63.5)	159
Monday	100 (63.3)	58 (36.7)	158
Tuesday	86 (53.8)	74 (46.3)	160
Wednesday	89 (55.3)	72 (44.7)	161
Thursday	70 (43.8)	90 (56.3)	160
Friday	89 (55.6)	71 (44.4)	160
Saturday	95 (59.0)	66 (41.0)	161
Total	587 (52.5)	532 (47.5)	1119

The target was achieved most often on Sundays and least often on Mondays.

In contrast, the differences between calendar months were not significant (χ112 = 15.7, p = 0.15). Plotting the monthly figures (Figure 13) suggests an upwards trend over the 3-year period commencing in September 2009.

FIGURE 13.

Proportion of days with procedures starting before 10.00 and finishing before 20.30, by month.

The data were analysed using logistic regression. Given the difference between days of the week, this categorical variable was included as the baseline model. At the next step a linear trend was included. This was highly significant. Each year the odds of achieving the target increased by a factor of 1.40 (95% CI 1.22 to 1.61, p < 0.001). The next step was to fit an impact of the RPIW. There was almost no change in the odds of achieving the target following the RPIW (odds ratio 0.98, 95% CI 0.62 to 1.49). Finally, fitting an interaction between the effect of the RPIW and the linear trend indicated that the RPIW had no impact on the rate of change. The difference in trends on the logistic scale before and after the RPIW was –0.30 with 95% CI –1.03 to 0.44. Prior to the RPIW, the target was achieved on only 43% of occasions; after the RPIW this figure rose to 55.7%. However, the above analysis suggests that it is likely that this increase would have occurred anyway, given the underlying trend in the 2 years from September 2009.

Site 10 purposeful inpatient admission rapid process improvement workshop

The data set used for this analysis was linked across several ward changes for males and females across the period from April 2005 to October 2012 (see Chapter 4, Table 6 for details). For the purposes of the analysis, the RPIW was assumed to be April 2008 for males and October 2008 (‘share and spread’) for females.

The key variables for this analysis related to the length of time the patient spent receiving care in hospital and, specifically, on the intervention wards. There were a number of potential dependent variables (see Chapter 4, Tables 6 and 7). Note that in the analysis, ‘length of stay in hospital’ is equal to the time between admission to hospital and admission to the ward, plus the length of spell on the ward and the time from discharge from the ward to discharge from hospital.

However, as can be seen in Figure 14, a number of the variables were highly correlated. Generally, the length of stay on the ward was highly correlated with the length of stay in the hospital. However, patients may have been admitted to the hospital some time before being transferred to the ward. Although most patients were discharged from the hospital when they left the ward, there were some patients who were transferred elsewhere within the hospital prior to being discharged. It was decided that the two most important variables to analyse were the time that the patient spent on the ward (which could be most influenced by staff behaviour) and the time from admission to the ward to discharge from hospital (to allow for apparently shortened stays which were actually due to transfers to other wards). Other variables of interest were the time between admission to hospital and start date on the ward, and the time between discharge from the ward and discharge from hospital. Before considering the impact of the RPIW on the time spent on the ward, a number of processes of care variables were examined so as to understand the concomitant changes that were happening during the period of interest.

FIGURE 14.

Length of stay in days in hospital and on the ward. (a) Total length of stay by time spent on ward; (b) time between admission to ward and discharge from hospital by time spent on ward; (c) time between admission to ward and discharge from hospital: frequency distribution (n = 4195; mean 13.01 days, standard deviation 81.338 days); and (d) total length of stay by time between admission to ward and discharge from hospital.

Number of patients admitted or transferred on to the wards per month

The number of patients admitted or transferred on to the wards is shown in Figure 15. The number of male patients admitted or transferred on to the ward fell from a mean of 27.6 per month prior to the RPIW to 22.2 after the RPIW (difference in means = –5.4, 95% CI –7.8 to –3.0). In contrast, the number of female patients admitted or transferred increased slightly from 18.0 per month to 20.9 per month, although this change was not statistically significant (difference in means = 2.9, 95% CI –0.6 to 5.3).

FIGURE 15.

Admissions and transfers to hospital wards by calendar month.

Proportion of patients admitted directly on to the ward

The proportion of patients admitted directly on to the ward was smaller after the RPIW (Table 21); a greater proportion of patients were being transferred to the ward. This was particularly true for men, with 18.3% of male patients after the RPIW having been admitted elsewhere.

TABLE 21 - Number of patients admitted directly on to the ward

Sex	RPIW	Patient admitted directly to ward?		Total
		No, n (%)	Yes, n (%)
Male	Pre RPIW	11 (1.1)	1031 (98.9)	1042
	Post RPIW	228 (18.3)	1016 (81.7)	1244
	Total	239 (10.5)	2047 (89.5)	2286
Female	Pre RPIW	6 (0.7)	828 (99.3)	834
	Post RPIW	95 (9.1)	952 (91.9)	1047
	Total	101 (5.4)	1780 (94.6)	1881
Total	Pre RPIW	17 (0.9)	1859 (99.1)	1876
	Post RPIW	323 (14.1)	1968 (85.9)	2291
	Total	340 (8.2)	3827 (91.8)	4167

Time between admission to hospital and admission to the ward

Most patients were admitted directly on to the ward. For those patients who were not directly admitted to the ward, the mean time between admittance to hospital and transfer to the ward was 38.1 (95% CI 11.4 to 65.0) days for men and 51.6 (95% CI 10.3 to 92.8) days for women.

Length of spell on the ward

For each patient, the time between admission or referral to the ward and leaving the ward was calculated. The data were normalised by taking a log transformation (Figure 16).

FIGURE 16.

Distribution of log-transformed length of stay with superimposed normal curve (n = 186). Mean 2.55 days, standard deviation 0.362 days.

For each calendar month the mean of the log-transformed length of stay was calculated and the antilog taken to give the geometric mean (Figure 17).

FIGURE 17.

Geometric mean length of spell on ward, in days, by month of admission by sex.

There appears to be a considerable variability in the length of stay, with some particularly large geometric mean values for women during 2007. The log-transformed length of stay was analysed using linear regression.

At step 1 a linear trend was fitted across the period April 2005 to November 2012. The length of stay on the ward fell by 3.9% (95% CI 2.3% to 5.6%).

Adding an impact of the RPIW at step 2 suggested that the RPIW led to a reduction in the length of time spent on the ward; the time spent on the ward following the RPIW fell by 18.0% (95% CI 4.3% to 29.7%). With a difference from pre to post RPIW, the effect of a linear trend was no longer significant and this term was removed from the model.

Given the systematic differences between wards, a difference in the impact of the RPIW was fitted for male and female patients. There was a significant interaction effect. There was a significant impact of the RPIW for female patients but not for male patients; length of spell on the ward was reduced post RPIW to

• 68.4% (95% CI 60.8% to 80.0%) of pre-RPIW values for women

• 93.4% (95% CI 83.9% to 103.9%) of pre-RPIW values for men.

A concern with these results was that they primarily reflected the spikes in length-of-stay times for women admitted in 2007. It is possible that length of stay was already falling for female patients by the beginning of 2008. The above results may reflect a regression to the mean. The other issue with this analysis was that admissions were categorised by whether they occurred before or after the RPIW. Some patients who were admitted prior to the RPIW were discharged following it. This issue was addressed by the following analysis.

Time between admission to the ward and discharge from hospital

The time to discharge from hospital (starting from the first admission to the ward) was analysed using a Cox proportional hazards model with time-varying covariates. All patients who were admitted to the ward after 1 April 2005 were included in the regression analysis.

In this analysis the intervention variable was a time-varying covariate. The length of spell for those patients who were on the ward at the time of the RPIW was split into two periods; in the first their management was undertaken pre RPIW and in the second it was undertaken post RPIW. The results from the modelling were calculated as hazard ratios. A higher hazard indicated a faster rate of discharge of patients from the ward (one of the aims of the RPIW).

Fitting a trend at step 1 indicated that there was an annual increase in the rate of discharge (hazard ratio 1.015, 95% CI 1.001 to 1.030). Adding an impact of the RPIW at step 2 suggested that if allowing for an annual increase in the rate at which patients were discharged the RPIW was to decrease the rate of discharge (hazard ratio 0.80, 95% CI 0.72 to 0.90).

The next steps were to investigate separate effects for men and women. Allowing for an annual trend, the estimated impacts of the RPIW were

• a reduction in the rate of discharge for men; hazard ratio 0.70 (95% CI 0.60 to 0.81)

• very little change in the rate of discharge for women; hazard ratio 0.93 (95% CI 0.77 to 1.11).

For comparison purposes, assuming no annual trend, the corresponding unadjusted estimates were

• a reduction in the rate of discharge for men; hazard ratio 0.89 (95% CI 0.82 to 0.97)

• an increase in the rate of discharge for women; hazard ratio 1.12 (95% CI 1.02 to 1.23).

The true impact of the RPIW was likely to be between the adjusted and unadjusted estimates. Any trend in discharge rates was in the hypothesised direction for women admitted to the study wards, but the analysis suggested that male patients were waiting longer to be discharged.

Summary

The number of men admitted to the ward each month fell after the RPIW. The proportion of patients admitted directly on to the ward (date of admission = ward start date) fell after the RPIW. This was true for both male and female patients but was particularly noticeable for men, with the proportion of male patients being transferred in from elsewhere increasing from 1% to 18%. The length of time that women spent on the ward decreased following the RPIW, whereas the impact on the length of time that men spent on the ward was ambiguous. When considering the total length of time between arrival on the ward and discharge from hospital, there was some evidence that men may have been discharged more slowly. There may have been a slight increase in the rate of discharge for women. Both of these results suggest that some patients were being transferred to other wards for extended treatment following the RPIW.

Site 10 community psychosis rapid process improvement workshops (referral, treatment, discharge)

The data set analysed for these RPIWs consisted of all referrals to eight adult community psychosis teams (three interventions, five controls) for the period 1 January 2010 to 31 July 2012 (see Table 5 for more details). There were 3036 episodes of care included in the analysis. The RPIW week was in January 2011.

Duration from referral received to first allocation

Time from ‘referral received’ to ‘first allocation’ was highly skewed (Figure 18).

FIGURE 18.

Time in days from referral received to first allocation.

The mean was 4.1 days and the standard deviation was 10.3 days. For 59% of patients (95% CI 56% to 61%) the date of first allocation was the same as the date that the referral was received. Initial exploration suggested that the most appropriate way to model the data was to fit a zero inflated negative binomial regression model. Using this procedure the likelihood of being allocated on the date that the referral was received, and for other patients, the length of time spent waiting (that is, those for whom the date of allocation was not the same as the date of referral) were modelled simultaneously. Summary data corresponding to these variables were plotted by calendar month as shown in Figures 19 and 20.

FIGURE 19.

Proportion of patients allocated on the day that the referral was received.

FIGURE 20.

Mean time to allocation for those patients who did not receive an allocation on the day that the referral was received.

Any effect of the RPIW appeared to be in the hypothesised direction. There appeared to be a higher proportion of patients allocated on the day that the referral was received in intervention sites following the RPIW. The mean time to allocation for the remaining patients appeared to be lower than in the control sites following the intervention. Of note is the apparent fall in the mean time to first allocation in the control sites in 2011–12 compared with 2010.

Using a zero inflated negative binomial regression model we simultaneously modelled the probability that a patient was allocated on the day that the referral was received and time to allocation for the remaining patients. The dependent variable was time in days to first allocation. The first step was to include a trend over time in each part of the model. Both trends were significant:

• annual trend in the proportion on a logistic scale = –0.29 (95% CI –0.56 to –0.02)

• annual trend in time to allocation on a log scale = –0.45 (95% CI –0.57 to –0.32).

The negative coefficients indicated an overall reduction in the proportion of patients allocated on the day of referral (which would be in the opposite direction to the hypothesised impact of the RPIW) and a reduction in the time to allocation for other patients (which would be consistent with the hypothesised impact of the RPIW).

The second step was to adjust for differences between mental health teams. There were eight teams. Adding differences between teams in both halves of the model improved the fit significantly (change in –2 log likelihood = 296.3 for the loss of 14 degrees of freedom). The adjusted estimates of trend were

• annual trend in the proportion on a logistic scale = –0.12 (95% CI –0.32 to 0.08)

• annual trend in time to allocation on a log scale = –0.45 (95% CI –0.58 to –0.32).

There was almost no change in the trend in time to allocation, but the estimated trend in the proportion of patients allocated on the day of referral was reduced substantially. The third step was to add a step change in all sites (both intervention and control) following the introduction of the RPIW. There were four parameters of interest:

• annual trend in the proportion on a logistic scale = –0.25 (95% CI –0.64 to 0.14)

• estimated step change in the proportion on a logistic scale = 0.22 (95% CI –0.36 to 0.81)

• annual trend in time to allocation on a log scale = –0.14 (95% CI –0.40 to 0.12)

• estimated step change in the time to allocation on a log scale = –0.58 (95% CI –0.98 to 0.18).

None of these coefficients differed significantly from zero but the CIs were fairly wide. The fourth step was to fit an impact of the RPIW by adding a binary indicator variable to the model that contrasted intervention sites after the RPIW with intervention sites pre intervention and control sites for the entire period from 2010 to 2012. The parameter estimates were

• annual trend in the proportion on a logistic scale = –0.25 (95% CI –0.63 to 0.13)

• estimated step change in the proportion on a logistic scale = 0.08 (95% CI –0.52 to 0.69)

• estimated impact of the RPIW on the proportion on a logistic scale = 0.45 (95% CI –0.16 to 1.06)

• annual trend in the time to allocation on a log scale = –0.15 (95% CI –0.40 to 0.11)

• estimated step change in the time to allocation on a log scale = –0.72 (95% CI –1.13 to –0.30)

• estimated impact of the RPIW on the time to allocation on a log scale = 0.54 (95% CI 0.08 to 1.00).

This model suggested that the impact of the RPIW was to increase the proportion of patients who were allocated on the day that the referral was received, but that the time to allocation for other patients increased (although neither effect was statistically significant).

Given that neither of the two trend terms differed statistically from zero, they were removed and the model re-estimated as follows:

• estimated step change in the proportion on a logistic scale = –0.24 (95% CI –0.61 to 0.12)

• estimated impact of the RPIW on the proportion on a logistic scale = 0.45 (95% CI –0.17 to 1.07)

• estimated step change in the time to allocation on a log scale = –0.91 (95% CI –1.16 to –0.67)

• estimated impact of the RPIW on the time to allocation on a log scale = 0.53 (95% CI 0.07 to 0.99).

The estimated impacts of the RPIW generated by this model were almost identical to those obtained from the previous model, but the model itself enabled a much clearer interpretation of the results. The estimated step changes in the model corresponded to the estimated change for control teams following the introduction of the RPIW. Considering the proportion of patients allocated on the day of referral, this fell for control teams (change = –0.24 with 95% CI –0.61 to 0.12) and was therefore not statistically significant. The corresponding change for intervention teams was an increase of 0.21 with 95% CI –0.29 to 0.70. The difference between these figures was the estimated impact of the RPIW, which was 0.45 (95% CI –0.17 to 1.07). These figures were consistent with the data shown in Figure 18. The estimated impact of the RPIW corresponded to an odds ratio of 1.57 (95% CI 0.84 to 2.90). None of these effects were statistically significant. Considering the time to allocation for the remaining patients, the estimated change for control teams on the log scale was –0.91 (95% CI –1.16 to –0.67). There was also a reduction in the time to allocation in intervention sites; the estimated change on the log scale was –0.38 (95% CI –0.77 to 0.01) (although this was smaller and only of borderline significance; p = 0.06). The difference between these changes did, however, differ significantly from zero and was the estimated impact of the RPIW (0.53, 95% CI 0.07 to 0.99). These figures were consistent with the data shown in Figure 20. The time to allocation fell more in control sites, but this was primarily attributed to the fact that the time to allocation had already fallen considerably in the intervention sites 6 months prior to the intervention. There was much less room for improvement in the intervention sites, which may explain the modest reduction in time to allocation observed in those sites. It was very difficult to explain the very large reduction observed in the control sites (and this was not explored as part of the study).

Duration from referral received to first successful face-to-face contact

Time to first contact was highly skewed (Figure 21).

FIGURE 21.

Frequency distribution of time in days to first contact.

The mean time to first face-to-face contact was 18 days, with a standard deviation of 25 days. Sixteen per cent (95% CI 14% to 18%) of patients had their first face-to-face contact the day that the referral was received (Figure 22).

FIGURE 22.

Proportion of patients with face-to-face contact on day of receipt of the referral, by calendar month.

There appeared to be very little difference between control and intervention sites over the period 2010–12.

The mean time to first face-to-face contact for other patients is shown in Figure 23.

FIGURE 23.

Mean time to first face-to-face contact for patients whose contact was not on the day that the referral was received, by calendar month.

The time to the first face-to-face contact for other patients appeared to decrease steadily over the period 2010–12 but there was considerable monthly variability. The time in days to first contact was analysed using a zero inflated negative binomial regression model. A trend over time was fitted for both parts of the model and was adjusted for differences between teams (fitted as fixed effects). The estimated annual trends were

• annual trend in the proportion of patients with face-to-face contact on the day that the referral was received (on a logistic scale) = –0.12 (95% CI –0.40 to 0.16)

• annual trend in the time to face-to-face contact for other patients (on a log scale) = –0.29 (95% CI –0.37 to –0.21).

Consistent with Figures 21 and 22, there was a downwards trend in the time to first face-to-face contact for other patients, but the trend in the proportion of patients seen on the day of receipt of referral did not differ significantly from zero. The trend was removed from the inflation part of the model and a step change that corresponded to the introduction of the RPIW was included in both parts.

• Step change in the proportion of patients with a face-to-face contact on the day that the referral was received (on a logistic scale) = 0.18 (95% CI –0.26 to 0.63).

• Annual trend in the time to face-to-face contact for other patients (on a log scale) = –0.17 (95% CI –0.30 to –0.03).

• Step change in the time to face-to-face contact for other patients (on a log scale) following the RPIW = –0.22 (95% CI –0.43 to –0.00).

The step change in the proportion of patients with a time to contact of zero did not differ significantly from zero. For other patients, both the trend over time and a step change coincident with the RPIW were statistically significant. Modelling the impact of the intervention in both halves of the model:

• step change in the proportion of patients with a face-to-face contact on the day that the referral was received (on a logistic scale) = 0.40 (95% CI –0.28 to 1.07)

• impact of the RPIW on the proportion of patients with a face-to-face contact on the day that the referral was received (on a logistic scale) = –0.46 (95% CI –1.41 to 0.50)

• annual trend in the time to face-to-face contact for other patients (on a log scale) = –0.16 (95% CI –0.30 to –0.03)

• step change in the time to face-to-face contact for other patients (on a log scale) following the RPIW = –0.16 (95% CI –0.39 to 0.06)

• impact of the RPIW on the time to face-to-face contact for other patients (on a log scale) = –0.19 (95% CI –0.44 to 0.07).

There was no evidence that the RPIW had a marked impact on the time to first contact. However, the time to first contact decreased steadily over the period of follow-up for all sites. Again, the improvement in the control group militated against being able to conclude a positive effect of the intervention.

‘Did not attends’ at first appointment

One hundred and eleven patients out of 1924 did not attend the first scheduled face-to-face contact (proportion = 5.8%, 95% CI 4.7% to 6.9%). There was considerable variability in this figure from month to month (Figure 24).

FIGURE 24.

Proportion of DNAs at first contact by calendar month.

Whether or not a first contact resulted in a DNA was analysed using logistic regression. At step 1, after including differences between community teams as fixed effects, a linear trend was fitted. Each year the odds of a DNA changed by a factor of 0.82 (95% CI 0.62 to 1.06), which does not differ significantly from 1 (corresponding to no change). At step 2 the trend was removed and separate changes following the intervention were fitted for control and intervention sites:

• change in likelihood of a DNA on a logistic scale in the control sites = –0.19 (95% CI –0.69 to 0.32)

• change in likelihood of a DNA on a logistic scale in the intervention sites = –0.62 (95% CI –1.50 to 0.00).

There was a slight reduction in the likelihood of a DNA in both control and intervention sites (although in neither case was the reduction statistically significant). The estimated impact of the RPIW was the difference between these two figures (–0.43, 95% CI –1.24 to 0.37), which corresponded to an odds ratio of 0.65 with 95% CI 0.29 to 1.45. Although the CI spanned 1 (corresponding to no impact of the intervention), it was very wide. This suggested that there was insufficient power to detect an impact of the intervention against such very large background variation.

Duration from referral received to first assessment

The time to assessment was highly skewed (Figure 25) with a mean of 24.1 days and standard deviation of 54.8 days.

FIGURE 25.

Distribution of time in days from referral received to assessment.

An assessment was recorded the day that the referral was received in 18.3% of cases (95% CI 16.5% to 20.2%). This proportion was similar for control and intervention localities (Figure 26), although there appears to be a spike corresponding to November 2011.

FIGURE 26.

Proportion of cases where assessment was recorded on day of receipt of referral by calendar month.

The distribution of mean time to assessment for the other cases is shown in Figure 27. A notable feature of this graph is the downwards trend in time to assessment in control sites.

FIGURE 27.

Mean time in days to assessment for patients not assessed on day referral was received by calendar month.

Logistic regression indicated no impact of the intervention on the proportion of patients assessed on the day of referral (odds ratio 0.96, 95% CI 0.59 to 1.56). Time to assessment was analysed using a negative binomial regression model. Fitting a trend over time yielded the following estimate:

• annual trend in time to assessment (on a log scale) = –0.10 (95% CI –0.17 to –0.03).

Adding a difference pre/post January 2011 yielded the following estimates:

• annual trend in time to assessment (on a log scale) = 0.10 (95% CI –0.01 to 0.22)

• change post January 2011 in time to assessment (on a log scale) = –0.37 (95% CI –0.55 to –0.19).

This suggests that most of the change over time can be explained by a drop after January 2011. Removing the trend over time gives

• change post January 2011 in time to assessment (on a log scale) = –0.24 (95% CI –0.34 to –0.14).

Fitting different changes for intervention and control localities:

• change post January 2011 in time to assessment in control localities (on a log scale) = –0.20 (95% CI –0.31 to –0.09)

• change post January 2011 in time to assessment in intervention localities (on a log scale) = –0.31 (95% CI –0.45 to –0.17).

There was a larger reduction in intervention localities. The estimated impact of the intervention (the difference between these figures on a log scale) was –0.11 (95% CI –0.26 to 0.04). The direction of effect was as predicted but the effect did not differ significantly from zero. This estimate on the log scale corresponded to a reduction in time to assessment of 10% (with the 95% CI indicating a change that was between a reduction of 23% and an increase of 4%).

Duration from referral received to diagnostic formulation

For each referral there were multiple records corresponding to different tasks undertaken by the community team. For each referral the first task with the descriptor ‘diagnosing/formulation’ or ‘assessment’ was identified and the duration between referral and this task was calculated. The mean time to diagnostic formulation across all referrals received in a calendar month is shown in Figure 28.

FIGURE 28.

Mean time to diagnostic formulation by calendar month.

The time between the referral being received and the diagnostic formulation was then log transformed and analysed using a multilevel normal regression model. At step 1 a trend over time was included in the model. There was a significant annual decrease in the time to diagnostic formulation:

• annual change in log (time to diagnostic formulation) = –0.46 (95% CI –0.60 to –0.32).

At step 2, a step change post January 2010, corresponding to the implementation of the RPIW, was introduced to the model:

• annual change in log (time to diagnostic formulation) = –0.17 (95% CI –0.41 to 0.08)

• estimated step change in log (time to diagnostic formulation) = –0.52 (95% CI –0.88 to –0.16).

Separate changes were fitted for the control and intervention teams:

• annual change in the log (time to diagnostic formulation) = –0.16 (95% CI –0.40 to 0.09)

• estimated step change in the log (time to diagnostic formulation) in the control sites = –0.47 (95% CI –0.85 to –0.09)

• estimated step change in log (time to diagnostic formulation) in the intervention sites = –0.64 (95% CI –1.08 to –0.19).

There was a reduction in the time to diagnostic formulation in both control and intervention sites. The estimated impact of the intervention was the difference between these two figures (–0.16, 95% CI –0.40 to 0.09) which, back transformed, corresponds to a reduction in time to diagnostic formulation of 15%, although this was not statistically significant (95% CI corresponds to between a reduction of 41% and an increase of 23%).

In practice, there was a reduction in the time to diagnosis formulation in the intervention localities which may or may not have been a result of the RPIW. However, there was a similar change in non-intervention (control) localities. Pooling this information suggested that the effect of the RPIW was not statistically significant. It is difficult to explain the large reduction in time to diagnostic formulation in the control sites (and, as noted earlier, what was happening in the control sites was not explored as part of this study). Additionally, the very wide CI reflects the inherent variability in the data and suggests that we may not have had sufficient power to detect clinically important differences.

Discharge rates

One of the aims of the RPIW was to reduce the time from referral to discharge. Not all referrals had a recorded date of discharge. The first investigation was to look at the impact of the RPIW on the likelihood of a date of discharge being recorded. The number of referrals with a recorded date of discharge decreased over time (Figure 29).

FIGURE 29.

Proportion of referrals with a recorded date of discharge by calendar month.

This was true for both groups. Fitting a logistic regression model with a common trend over time but a different change in each group following the introduction of the RPIW yielded the following estimates:

• overall annual trend on a logistic scale = –0.90 (95% CI –1.16 to –0.64)

• change after January 2011 in the control group = 0.06 (95% CI –0.36 to 0.48)

• change after January 2011 in the intervention group = 0.03 (95% CI –0.43 to 0.49).

This model suggests that the variability in the data can be explained by a trend over time that is the same for both groups.

The estimated impact of the RPIW was the difference between the step changes (–0.03, 95% CI –0.46 to 0.40). This suggested that the RPIW had very little impact on the proportion of patients who had a recorded date of discharge. Taking the exponent, this impact corresponded to an odds ratio of 0.97 with 95% CI 0.63 to 1.50.

Duration from referral received to discharge

There were 1169 referrals in the database with a recorded time to discharge. The summary statistics are shown in Table 22.

TABLE 22 - Time in days from referral to discharge

Locality/time	Mean (days)	n	Standard deviation (days)
Control 2010	279.8	418	286.9
Control 2011–12	193.7	389	162.5
Intervention 2010	241.0	187	241.5
Intervention post RPIW	184.6	175	156.7
Total	230.7	1169	229.7

The time to discharge in days was highly skewed, as can be seen in the box plots in Figure 30. The horizontal line within the box corresponds to the median time to discharge. This appears to have fallen in the control localities but increased in the intervention localities post January 2011. The times were log transformed, log_e(1 + time in days) (Figures 31 and 32).

FIGURE 30.

Time to discharge in days.

FIGURE 31.

Frequency distribution of log-transformed time to discharge (n = 1169). Mean 4.79, standard deviation 1.395.

FIGURE 32.

Log-transformed time to discharge by calendar month.

The times to discharge appeared to be broadly similar in the two groups.

A trend was fitted over time:

• annual change in log (time to discharge) = –0.19 (95% CI –0.30 to –0.07).

Adding a step change in January 2011 across all sites:

• annual change in log (time to discharge) = –0.26 (95% CI –0.46 to –0.05)

• step change post January 2011 = 0.11 (95% CI –0.18 to 0.40).

Fitting different changes in the intervention and control localities:

• annual change in log (time to discharge) = –0.26 (95% CI –0.47 to –0.06)

• step change post January 2011 in the control localities = –0.00 (95% CI –0.30 to 0.30)

• step change post January 2011 in the intervention localities = 0.33 (95% CI –0.03 to 0.69).

The estimated impact of the RPIW was given by the difference between the changes in the control and intervention practices (0.33, 95% CI 0.01 to 0.65). This would suggest that the impact of the RPIW was to increase time to discharge in intervention localities. In reality, time to discharge was actually getting smaller in intervention localities (we were looking at changes in both control and intervention localities while allowing for a downwards trend over time). However, the rate of decline in intervention localities was much lower than the rate of decline in time to discharge in control localities. Thus the apparent impact of the RPIW was to increase time to discharge.

Summary

A consistent feature of these analyses was that large changes in most of the key variables were observed in the control localities. Further, the direction of these changes tended to be the same as that of the effects hypothesised in the intervention groups as a result of the RPIWs. Consequently, when compared against the control group, none of the changes in the intervention group was significant in a positive direction. Indeed, this comparison led to an estimated negative impact of the RPIWs on a number of variables. Had the analysis been undertaken without the control groups, in a number of cases it would have been concluded that there had been no impact (instead of a negative impact) or a positive impact (instead of no impact) of the RPIW.

It was very difficult to explain the observed improvements in the control groups (and, as noted earlier, what was happening in the control sites was not explored as part of this study). This was a non-randomised comparison and thus systematic differences between groups were to be expected.

Factors that promoted or inhibited the adoption, implementation and sustainability of the North East Transformation System

Transformational change programmes are usually characterised by four distinct phases: (1) acceptance of the need for change; (2) adoption of the means by which change will be enacted; (3) initial implementation of the change programme; and (4) embedding sustainable practices, such that the programme continues in the medium to long term. In the case of the NETS, the first phase was successfully completed; nearly all North East NHS organisations agreed that there was an urgent case for continuous, region-wide, large-scale QI. This section presents findings in relation to the factors that promoted or inhibited the NETS through early adoption, implementation of the programme and its subsequent sustainability. These factors can be conveniently clustered within five of the areas in the Pettigrew et al. ‘receptive contexts for change’ model125 and the rest of this section is structured around these five areas.

Share-and-spread activities

It is important that lessons can be learned from any activities that have been undertaken, regardless of whether they are positive or not. Sharing such experiences allows other individuals to see what improvements were made and whether or not they could be adopted elsewhere in the organisation. However, this ‘sharing’ or transfer of best practice was not without its difficulties, as this interviewee explained:

I think the more difficult point is often if you then try and translate those changes to other areas where staff haven’t been directly involved in improvement events there, that is certainly much more difficult. Even if to be frank there aren’t those sort of negative consequences, because you’re just trying to get people engaged in making changes when they can’t always see for themselves why they should bother is a problem, regardless of whether there are job losses or job changes, just there is still some resistance. So we usually sort of share and spread events to try and overcome that.

Site 10, HR manager

Another interviewee linked the share-and-spread and replication stages back to the issue of standardisation and standard operations of work:

Even if you’re doing something similar oh well we’ve refined it very slightly, we’ve done it slightly differently. As soon as you put it into, like you say in the information here, as soon as you put it into an organisation of 9000 people it’s never going to be replicated. It doesn’t matter what you do, it doesn’t matter how rigid you are, it will never, ever be replicated like for like. So that ability then to reflect and to learn from that implementation is really key.

Site 07, organisation development manager

In other cases, the share-and-spread events worked quite well. A service development lead explained:

We’ve done spread and share events so an example of that would be we did antipsychotic monitoring, an event in Stockton with a psychosis team. And we then rolled it out to every psychosis team within the trust. So we did, rather than do a 5-day event, we did a reduced 4-day event, and spread out the learning. So we looked at the standard work we’d produced, we looked at the visual control boards we’d produced, we allowed some local variation but they had to have the same outcomes. So we’ve actually started doing share-and-spread events.

Site 10, service development lead

A particular challenge was how a share-and-spread event should be conducted. One interviewee provided an example of the approach used in her organisation:

Well, as an organisation the organisation develop half-day awareness sessions so that staff at every level could have an awareness just about what does the VMPS actually mean, what does NETS mean and to give them an idea about some of the basic concepts of 5S and the waste wheel. That was a half day and then they also put on full days which included how to use different tools and then there’s a week course which is the 5 days looking at how to actually put those tools in place. Now the 5 days would be what we kind of envisage our senior staff getting involved in and our modernisation facility; it’s people who take a lead in trying to, you know, have continuous improvement in things.

Site 11–13, senior nurse business manager

Share-and-spread activities had not been limited to activity within the trusts. They were undertaken with other trusts as well as with external partners, as this interviewee described:

I think there’s the Leadership Awards and the Bright Ideas Awards are coming up as well. And the thing is there’ll be people in different departments, it’s about filtering it down, because someone’s going to have an idea, but they won’t know how to share it. And I know you get your payslips and you get the Bright Ideas application and stuff like that, but I think the more people involved in that the better. We’re very good at talking about sharing, very good at oh I’ll share what we did with this RPIW. We’ve produced a case study template that was last year and getting organisations, and we’ve had a workshop as well with someone external, well she used to work in the NHS, someone [name] knew, came in, delivered a morning session, but basically it was a pilot workshop to test it out, and that went quite well, went very well actually.

Site 09, NETS Coalition co-ordinator

Cost-effectiveness

Although the cost-effectiveness of the NETS programme did not form part of the research objectives, the issue arose frequently and spontaneously in interviews and focus group discussions, and was therefore included as part of our inductive analysis of transcripts. Around one-third of the transcripts (n = 19) provided commentary on this topic.

The question of whether or not the NETS programme was cost-effective has two components: the cost-effectiveness of the NETS training, as compared with training in other methodologies for improving quality and reducing waste, and the cost-effectiveness of the NETS activities in the study sites (RPIWs, 5S and 3P events, compact development, share-and-spread workshops) compared with other ways of achieving the same, or similar, results.

It proved impossible to make a definitive comparison between the cost-effectiveness of NETS training and that of other available QI programmes, mainly because data were either unavailable to the research team or ambiguous. Some of the study sites (01 and 14, for example) were using a mix of different improvement methods, and such data as were available on the costs of training did not provide sufficient detail to understand how much of the total was devoted to NETS-specific or VMPS-specific training. A search of the documentary materials, however, revealed some data on the costs of providing VMMC training for ‘wave 3’ of the NETS in 2010, as brokered by the central NETS support team at NHS NE. This showed that, if all of the proposed VMMC-supported training, plus staff visits to Seattle, WA, and Japan, had taken place in that year, the total costs borne by NHS NE and the relevant trusts would have been comfortably in six figures. This sum only included the direct costs of payments to VMMC, and did not include the costs to the trusts of staff time to organise and administer the training, the costs of providing cover for staff engaged on the training programmes or the costs of materials and travel in the region. It is clear from just this limited set of data that the VMMC training, as part of the overall NETS programme, was not a cheap option or insignificant budget item; but this tells little about whether or not it was cost-effective and value for money. To make this judgement the following evidence would be required:

• whether or not the NETS organisations would have organised other types of training, if not engaged in VMMC activities

• the success of VMMC training compared with other training programmes, in terms of delivering and sustaining training objectives

• the congruence of VMMC training with the overall aims of the NETS, again compared with other possible programmes.

The above information was not available, except in anecdotal and subjective form, so a proper assessment of the cost-effectiveness of the NETS training (and, in particular, the VMMC element of this) cannot be made.

A similar picture emerged when the cost-effectiveness of the NETS activities was considered. Many of those staff who commented on the cost-effectiveness of carrying out NETS activities spoke of the need to perform cost–benefit analyses, to understand the value-for-money aspect of the NETS programme, or simply to develop better metrics that would allow such comparative study. Sometimes the cost-effectiveness of the NETS programme was unequivocal:

Now we’re able to point to things like I’ve told you . . . I’ve got an empty surgical ward following RPIWs that have happened there . . . So the board now . . . sees the financial payback, as well as the fact that in the last set of annual health-check ratings we were double excellent, so it clearly did something in terms of quality and safety as well.

Study site 09, senior director

However, the link between NETS activities and the bottom line was often not so clear-cut. A senior director at a commissioning organisation was asked about the impact of RPIWs financially, and considered that this was not well understood:

. . . that’s one of the areas which I think we . . . do not focus sufficiently on . . . and I think . . . that’s one of the key things that we need given the economic circumstances . . . there’s not a great deal of focus I would say on the pure financial.

Study site 11–13, senior director

Perhaps this is not surprising; the driving force behind the NETS was improvements to patient safety and the quality of care, and although financial benefits were expected to flow from this focus, they were deliberately not positioned as the central driver of the programme.

The data revealed very little about the cost-effectiveness of the NETS compared with other QI initiatives. Even where interviewees and focus group members had extensive experience of other lean-based programmes, or had previously taken part in NHS experiments with Six Sigma, total quality management or quality circles, they did not comment on the comparative cost-effectiveness of the NETS programme. In truth, it would be very difficult to do so. First, the recipients of training programmes (of all kinds) often have little or no idea of the cost of providing the training. Second, those who do know the costs are often unable to put an accurate value on the outcomes of the training, especially if the focus is not directly on cost-saving. Finally, as noted in Chapter 1 of this report, the NETS was a unique, region-wide programme, with expected benefits that were intended to span organisational boundaries. In these circumstances, the cost-effectiveness of the programme would always be difficult to assess directly.

With these caveats in mind, it is worth noting that some of the more senior staff interviewed were passionately convinced that the NETS stood an excellent chance of delivering on its promises and was highly cost-effective, but that this promise was imperilled by the upheaval in NHS structures following the election in 2010. As one medical director commented:

. . . we’ve done a massive own goal getting rid of the SHAs. If I was in the Department of Health I’d be saying oh look what’s happened in the North East, the SHA has managed to galvanise not the whole health-care community but a substantial amount of it, let’s go with that, let’s build on it, let’s spin that fly wheel even more. Let’s get them talking to some of the other SHAs and say look, if you have an SHA-wide service improvement model you can deliver vast amounts of savings. Because I absolutely think that the North East would have continued to deliver more and more savings if we were allowed to carry on. And you can see it in the people’s faces the devastation that we were just on the verge of something really special.

Focus group, medical director

Summary

Patient safety and putting the patient at the centre of all activities is a key message of the NETS. Patient safety was implicit in several of the ‘seven no’s’ that provide the basis of the Vision of NHS NE, especially No avoidable deaths, injury or illness and No avoidable suffering or pain. Emphasising safety was deliberate as patient safety was perceived as the main ‘selling point’ for staff involved in the NETS initiative. The introduction of a Compact was also identified as influential in shifting the emphasis of work on patient safety from a rule-based, process-oriented approach to one that addressed issues of organisation culture. This enabled improvements to be made to procedures that were known to be flawed. It was acknowledged that the planned implementation of improvement programmes was not a key issue. Some of the NETS study sites opted to implement their QI programme vertically (i.e. department by department, or ward by ward) rather than horizontally across the whole organisation.

The NETS focused on making measurable improvements to clinical care as a way to convince initiative-weary doctors and nurses of the value of the principles of the NETS, and to gain their commitment to the programme. Quality and safety were identified as being very closely linked. The lean programmes (e.g. VMPS, Unipart Way or the Productive Ward) taught staff that QI would arise from removing and reducing waste, focusing on adding value and concentrating on seeing processes from the patient’s point of view. The NETS was effective in reinforcing and encouraging a whole-system view of quality.

The NETS encouraged staff to see the world through the eyes of patients, their families and carers, and to focus on making changes to processes only where they were of direct benefit to these ‘customers’. It was a ‘bottom-up’ approach, with senior leadership commitment and resourcing supplied ‘top down’, and it was promoted as achieving transformational change through small-scale but continuous improvements to processes. Some of the non-VMPS study sites also had strong messages about the importance and value of involving patients in the redesign of processes and working practices. However, in these cases the PPI appeared to focus more on ward-level activity, rather than policy setting or influencing strategic objectives.

Successful adoption and implementation of the NETS were found to be closely associated with committed and stable leadership at the highest level in trusts and the SHA. Senior leaders, whether managers or clinicians, were required to take a hands-on approach to driving forward the NETS activities. They were expected to take part in practical, day-to-day improvement events. Evidence of public criticism of a training programme that involved regular overseas travel caused some NHS organisations to have concerns about joining the VMPS version of the NETS. The upheaval in NHS structures that began in 2010 had a significant effect on the NETS as a regional programme, and negatively affected its progress in some of the study sites.

The implementation and sustainability of transformational change in a complex and changing policy and organisational context

The factors that enabled or hindered transformational change were evaluated. Bringing about transformational change in health systems is regarded as particularly challenging even in the most propitious of circumstances, given the multiple complexities involved and the diverse range of stakeholders to be engaged. Health systems are complex in terms of both the challenges facing them and how these can best be addressed. Command and control approaches, though often superficially appealing, rarely succeed and risk oversimplifying the complexity of the problems facing managers and practitioners. In the NHS, complexity arises from numerous sources and factors including the political and regulatory environments, powerful groups operating nationally and locally, and the multiple groups involved in commissioning and providing health and health care.

Kotter145 argued that large-scale change can take years to bring about and goes through a series of eight distinct steps. These are establishing a sense of urgency; forming a powerful guiding coalition; creating a vision; communicating the vision; empowering others to act on the vision; planning for and creating short-term wins; consolidating improvements and producing still more change; and institutionalising new approaches. Virtually all eight steps can be discerned in respect of how the NETS was conceived and progressed. They also complement the eight factors making up the ‘receptive contexts for change’ framework. 125 However, Kotter’s schema does not include the impact of the external environment and, in particular, the political context (although it is included in the Pettigrew et al. framework125), which are among the major reasons why transformation efforts often fail in the NHS. The two factors can have a decisive effect on the direction of transformational change at all levels of the system. It is often the final two steps in Kotter’s list – consolidating improvements and institutionalising new approaches – that fall victim to changing political circumstances and fads and fashions. There was evidence from our study that such factors were at play in determining the NETS’s fate following the announcement of the NHS changes in mid-2010.

As a general point, though it may seem paradoxical, in order to effect successful transformational change there is a need for organisational stability, or a freezing of the organisation, before it can unfreeze and change. Public sector bodies, especially high-profile ones like the NHS, have over the years been subjected to increasingly frequent policy and structural changes which, while giving the semblance of change, tend to leave them remarkably unchanged – what Schön146 terms a case of ‘dynamics without change’. The consequence is that the same problems and challenges tend to recur.

The NHS changes announced unexpectedly in July 2010 were perhaps a good example of this phenomenon. Regardless of whether or not the changes were required or desirable, their impact on the NETS as originally conceived was profound. In particular, it seriously disrupted the approach adopted by NHS NE and necessitated a rethink of how the NETS’s future could be assured in a very different world. The reason for this was because the NETS aimed to achieve transformational change rather than merely apply lean thinking or a set of mechanistic tools. It was an ambitious attempt to change NHS organisations throughout an entire health-care system. To this end, the Vision, Compact and Method were, in combination, a holistic mechanism that aimed to achieve transformational change in a complex setting. This was because these three elements focused on the need for change management in the way that the NHS across the North East conducted its business; the focus was not on particular tools or methods. Overemphasis on the methods could have led to point improvements and a lack of sustainability. They were important, but only after the commitment to the Vision and Compact was in place. Those organisations which achieved this were better placed to weather the disruption unleashed by the 2010 changes. Indeed, as noted on a number of occasions, two organisations in particular stood out in terms of displaying a critical mass of activity which enabled them to achieve a degree of sustainability not found elsewhere in the study sites. That is not to say that a similar degree of embeddedness would not have developed or been possible elsewhere had the NETS been able to continue on its course. However, the immediate impact of the changes announced in mid-2010 did mean a setback in terms of achieving those hopes and aspirations. Consequently, those organisations which were less secure or developed in each of the three domains – Vision, Compact and Method – were more vulnerable to the impact of the changes which served to test the limits of the managerial and clinical commitment to the NETS.

Even so, across most of the study sites, the commitment to deep cultural change sought by the NETS remained fragile and had yet to attain the degree of embeddedness achieved in the VMMC. In particular, the Compact appeared to have received much more attention and investment there than in NHS NE. Indeed, the VMMC implemented the Compact before it became aware of the TPS. It was believed that only when the Compact was in place should the introduction of lean tools begin. In NHS NE, the three elements tended to advance simultaneously. There was a tendency in some quarters to invest more in lean thinking and tools than in the more difficult work arising from the Compact and, possibly to a lesser degree, the Vision.

In particular, the 2010 proposals for major NHS changes, which many commentators regarded as amounting to the biggest bang in the history of the NHS’s many reorganisations, disrupted the combined top-down and bottom-up approach to change which did seem to be working within the NETS study sites. 142,147 Strong and visionary leadership from the SHA was not to everyone’s taste in the region but, where it worked, it provided the momentum required to improve and embed the NETS approach. But this top-down approach could only work in those cases where there was also strong buy-in from those working at or close to the front line in the various NHS organisations. The 2010 changes put paid to the top-down dimension because the SHA’s position could not be sustained when faced with abolition in March 2013.

There were mixed views on the consequences for the NETS of the major upheaval in the NHS announced in July 2010 in the coalition government’s White Paper. 1 Some of those interviewed expressed concerns that the impending demise of familiar structures, notably the SHAs and the PCTs, and the resultant loss of organisational memory from parts of the NHS in the region, would limit progress of the NETS and would divert attention and resources from it. It was acknowledged that organisations could individually pursue the NETS if they wished to. However, the significance of what was happening in the North East as a whole in terms of a collaborative approach would be put in jeopardy without any guarantees that it would survive. But not all staff were persuaded by this view. Some of those interviewed felt that the SHA’s role had been negligible and would not be missed as long as there remained sufficient local commitment. Importantly, this view came more typically from provider organisations in the vanguard of implementing the NETS, who were more confident of their ability to continue to train staff without support from the SHA or elsewhere. In practice, some of these provider organisations did continue to support a small central NETS team to co-ordinate training and support activities, further development of the initiative and promotion of the NETS as a brand that could be offered to other NHS organisations.

The importance of leadership and its effectiveness

The important role of leaders and leadership in enabling transformational change in health-care organisations has been well established in the literature. It was certainly a key factor throughout the NETS at all levels and across all organisations in our case study sites. 148 The Pettigrew et al. 125 ‘receptive contexts for change’ framework includes as one of the eight factors ‘key people leading change’. The words were deliberately chosen to avoid connotations of heroic and individualistic ‘macho managers’. Instead, the model of leadership that is articulated refers to pluralist, distributed leadership styles that encourage team-building, group accountability and diversity of skills. The Vision and Compact were central to the leadership approach adopted; they represented the ends of the NETS project while the Method represented the means of getting there. The Vision, framed around the ‘seven no’s’, proved powerful in motivating staff to think differently about how to improve services.

A number of common themes concerning leadership emerged. First, many of the respondents were strongly of the view that embedding the principles underpinning the NETS required committed, stable, long-term leadership at the highest level. Continuity of key personnel was cited as critical to the success of change projects. Unplanned staff movements were associated with loss of purpose and commitment. In the North East there tended to be less senior staff churn than was evident in other regions, which helps embed learning and sustainability.

Second, it was important for senior leaders to have a hands-on presence. The NETS attracted both scepticism and cynicism that it was just another passing fad – one of the many fashions to sweep through the NHS without achieving much by way of impact or sustainable change. Staff therefore felt that rhetoric and exhortation were insufficient to win commitment and support from the workforce. Senior managers were required to take part in practical, day-to-day improvement activities if they were to be taken seriously. In short, they had to ‘walk the walk’, not merely ‘talk the talk’.

Third, continuity and consistency of purpose were highly valued outcomes of the NETS programme. They provided a framework and created a sense of purpose that enabled organisational change and the building of relations through the Compact, participative workshops and other knowledge-sharing activities. Clinicians were accustomed to working in a largely individualistic, competitive manner rather than in a more collegiate, team-based style, especially when it came to relationships with managers; this could act as a barrier to transformational change, which was addressed by the Compact.

Fourth, the importance of leadership skills was recognised at all levels. However, there was a tension between the type of skills that were considered to be desirable and the requirements of the top-down, target-driven culture of the NHS. The dilemma was captured by Currie and Lockett’s149 reference to ‘a managerialist form of transformational leadership’ that had been ‘promoted through government policy’ and had little connection with its academic conception. Maintaining or improving levels of quality and safety, meeting financial challenges and embedding a culture of continuous improvement were all seen to require different leadership styles from that which was commonly found throughout the NHS. In important respects, these concerns have been reinforced and confirmed by the findings from the Francis Inquiry into the failings at the Mid Staffordshire Hospitals Trust. 111 In contrast to what happened there, those closely involved in the NETS programme viewed leadership as a distributed attribute within their organisation. Leadership was characterised as an ability to empower a team, communicate a vision and be able to effectively delegate responsibility to the staff.

The research confirmed the findings of previous research which had stressed the importance of leadership in effecting successful transformational change in health-care organisations. It was recognised that shaping organisational culture and promoting teamworking were more important in achieving transformational change than the mechanics of implementing lean tools.

Differences between lean in the manufacturing and health sectors respectively, and the nature of complexity in the two sectors

From their origins in the Toyota Motor Corporation, lean production programmes spread through other manufacturing sectors, then into service industries, and more recently to a wide range of public sector organisations, including health care. Within the UK, lean in health care has been focused on single organisations, such as hospitals or smaller units. For example, Radnor et al. 84 looked at four case studies in the UK NHS and reported a number of successes across these cases, including

• reduced waiting times

• improved patient service

• increased direct patient care time

• clearer understanding of care pathways

• removal of duplicated processes

• exposing areas to 5S

• enhanced staff motivation

• removing unnecessary data fields from multiple forms.

This research has also identified similar outcomes. The NETS, however, tried to bring about transformational change in the health-care organisations across a whole region. Pettersen150 categorised four approaches that form the basis of a lean journey:

• toolbox lean: practical and operational

• becoming lean: practical and strategic

• leanness: philosophical and operational

• lean thinking: philosophical and strategic.

In order to achieve the full benefits of lean, an organisation needs to achieve ‘lean thinking’. For a manufactured commodity like a car, the customer is the commissioner, owner and user of the product. Radnor et al. 84 identified that in health care it is not clear who the customer is. The patient is the recipient of the service; the payment is made by the commissioning organisation. In a NHS context, the patient has no knowledge or interest in the price of the service or the cost of its delivery. The notion of value, central to lean thinking, is therefore ambiguous. A further issue is the number of stakeholders involved in a health-care ‘system’ (e.g. government, professional bodies, the local community, friends and family of the patient) who will have different views on what constitutes ‘value’ in this context.

Lean tools can help eliminate waste; make the product flow; standardise best practice; synchronise processes; and introduce a ‘pull’ approach. In manufacturing, it is easier to apply the tools and see the outcomes of the ‘product’ moving through the manufacturing process. In health care, it is likely to be the patient who moves through the ‘production line’, so a different approach and mindset is required for this form of ‘service’ model. Focus needs to be placed on the ‘social’ and ‘organisational’ system, not just the technical system (and the lean tools). The NETS identified these issues, which is why the Vision and Compact were so important. They provided a mechanism for aligning stakeholders’ interests and encouraging teamworking.

Patient characteristics and medical conditions may be highly variable, leading to a requirement for a high level of personalisation of care; this is in sharp contrast to the standardisation of products and processes found in many areas of manufacturing. The health-care ‘servicescape’151 provides the environment in which health professionals work, patients receive treatment and family and friends visit. The servicescape has environmental dimensions (ambient conditions, space/functions and signs, symbols and artefacts). It influences psychological moderators (cognitive, emotional and psychological) which effect stakeholder responses and behaviours (e.g. social interactions). 151 These factors make health services a more complex environment than manufacturing, which make it necessary to adopt a holistic sociotechnical approach. 113

The structure, development and implementation of the NETS has gone some way in trying to tackle these issues through the development of the Vision and Compact. This can be seen as a key difference in approach between manufacturing and health care.

Differences between study sites in terms of their receptiveness to the North East Transformation System and lean thinking, as well as their relative success in implementing the North East Transformation System’s founding principles

The study sites comprised NHS organisations that provided a range of services including acute hospital, mental health and learning disability, community, ambulance and commissioning services. The sites’ complexity varied across different dimensions: the mix of professions; the geographical spread of activities; the scope of interactions with other NHS and non-NHS organisations; and the range and type of internal processes. Apart from the inherently heterogeneous nature of the study sites, they also differed considerably in the type and extent of the external challenges and pressures they faced. Study site 06, for example, had been affected by a national policy of absorbing community service trusts within acute hospital trusts. Its staff transferred into study site 01 in April 2011. Study site 06 no longer existed, and the change also had an effect on the NETS programme within site 01. Former site 06 staff had to ‘buy into’ a different organisational Vision and an alternative Compact, and become familiar with a different set of improvement tools employed as the Method. These difficulties interrupted the rollout of site 01’s improvement programme.

Despite variations in operational priorities and internal structures, the study sites initially displayed a consistently high degree of receptiveness to the principles of the NETS. This was immediately apparent in the observation that most of the NHS organisations in the North East quickly agreed to follow the Vision/Compact/Method route to QI. Of the NHS organisations that were approached to take part in the first round of meetings in late 2007 that set the course for the NETS, only one refused to take part, and one (site 02–05) appeared to reject NETS training at an early stage.

The main reasons for the early ‘buy-in’ to the NETS were as follows.

• NHS NE had identified, through extensive consultation with NHS leaders, a consensus view that a step change in QI required collective as well as individual action. This required changes in culture as well as the application of lean tools.

• The boards and senior executives of NHS organisations were acutely aware that real-terms increases to NHS budgets were due to end after 2008. This was seen as both a pressing reason to find ways to remove waste from the health-care system’s processes, and an opportunity to set a new course for improvements to care quality and safety; in other words, a way to ‘keep ahead of the game’.

• The early pilot visits to VMMC and Japan had enthused senior clinicians and managers. This positive reaction had been well communicated using existing channels across the North East. There was therefore extensive (although not universal) managerial and clinical agreement that the NETS approach should be given an opportunity to flourish.

As the NETS programme was implemented in the wave 1, and then the wave 2 pathfinder organisations, the degree of receptiveness to the Vision, Compact and Method elements began to differ among the study sites. Some appeared to be more interested in the Method component, seeing lean as a means to make an immediate impact on waste, rework and progress towards external targets such as QIPP. Others took a longer-term view that placed Compact development firmly at the centre of the NETS programme. A small number (sites 07 and 10, and possibly site 09) seemed able to make simultaneous progress with all three elements of the NETS, although, even in those sites, there was a general view that the development of the Vision component had lagged behind the others.

It is not possible to establish a strict causal relationship between receptiveness to the principles underlying the NETS (and success in implementing them) and the managerial and clinical circumstances prevailing in the various study sites. There were simply too many confounding factors to establish cause and effect with any certainty. However, we note some of the broader impressions that were formed while conducting the research (Table 26).

At least four, and possibly five of the 14 study sites remained committed to the NETS approach throughout the study. Other sites that had adopted some of the NETS approach had not bought into the whole package (e.g. not using the VMPS Method). They also remained committed but did not consider that the precise Method chosen was of great importance. For them, demonstrating commitment and being clear about the purpose of the NETS approach were the most important factors. This issue is a manifestation of the loose–tight tension which, for many of those working in the NHS in the North East, characterised the NETS and their feelings towards it. Being permissive about means may be perfectly legitimate as long as the purposes and goals are tight, in terms of their clarity and measurability, in order to demonstrate improvement.

Reflections on the interrupted time series

The findings from the ITS analysis of the five RPIWs were mixed, with a small number of statistically significant improvements observed, some ambiguous results, several where no evidence of an impact of the RPIW could be detected and some counter-expectation findings. Clear improvements included a reduction in time from arrival of patients with abdominal pain in A&E to being X-rayed (surgical pathway RPIW) and a reduction in the length of stay on the ward for women (PIPA RPIW). Counter-expectation findings included an increase in the time to discharge (community psychosis – discharge RPIW). Overall, for 9 out of 19 variables the results tended to be ambiguous, without clear evidence of a positive or negative impact of the RPIWs. However, the ITS findings need to be placed in the context of both the limitations of any quantitative method (including the ITS method) for pragmatic evaluations and the findings of the rest of this study, as illustrated notably in the case studies.

As explained in Chapter 4, Interrupted time series, ITS is the strongest quantitative observational study design for examining the impact over time of short-term interventions, but it is acknowledged that the data requirements of multiple time points (12 for a long series) can be onerous. Nevertheless, the NETS evaluation would have presented a challenge to any prospective study as prior or baseline data are a feature of any pre–post prospective design. The ITS analysis presented within this report could be regarded as a ‘methodological experiment’ in determining whether or not this method is feasible for service evaluations of this kind within the NHS. A key problem was the requirement for multiple data points pre and post intervention, and the trade-off between the desire to measure the clinical outcomes targeted by the RPIW (which would have had to be specially collected prospectively for up to a year before the RPIW, whereas the standard method for doing RPIWs requires only baseline data collection) and the data available retrospectively pre RPIW, which could only be derived from routine information systems and was therefore limited to more administrative variables.

Other constraints included the lack of control by the research team over the construction of the variables to be used in the analysis and the consequent inability to predict data collection requirements and feasibility. Major data processing was also required on receipt of the extracted data sets, with some assumptions having to be made by the research team. It was not always feasible to verify the assumptions with hospital information staff or RPIW clinicians. There were particular challenges for the hospital information staff in constructing the linked data set for the PIPA RPIW, involving linkages over two information systems and through three ward name and/or location changes.

As noted in Chapter 4, Interrupted time series, ITS designs are strengthened by the inclusion of controls. Control data were obtained for five control localities for the community psychosis analysis but, for the reasons explained in Chapter 4, Data and data sources, it was not possible within the time frame of the study to identify controls for the other two RPIWs. It should be noted that obtaining control data is an issue for any controlled design.

A number of factors need to be considered in interpreting the findings observed from the ITS analysis, and more detail on these is found in the case studies. Of particular note for the ITS are the following:

• The surgical pathway RPIW was conducted against a background of a significant rising trend in overall attendances at A&E (thought to be due to the closure of a local ‘walk-in’ centre in October 2010), which was also likely to have resulted in a change in the casemix of those attending, so that the proportion of attendees with abdominal pain who required admission was probably falling. This background would have made it more difficult to detect any significant impacts of the RPIW, as ITS analysis controls for background trends.

• For the PIPA RPIW, we found that a greater proportion of patients were being transferred into the ward (from other wards) after the RPIW. This was particularly true for men, with the proportion of male patients who had been admitted elsewhere rising from 1% prior to the RPIW to 18% after the RPIW. When this factor was taken into account in the analysis, the reduction in length of stay for men also became statistically significant. However, it was not possible to determine whether or not the changes in admission pattern were also due to the RPIW, which makes it difficult to interpret these results.

• The community psychosis RPIWs’ variables were particularly difficult to construct from the data set provided; the process required several phases of discussion with clinicians. Assumptions needed to be made in some cases that may have led to data quality issues for the more complex variables. Several variables also had to be abandoned as they could not be reliably constructed, limiting the range of impacts that could be evaluated using ITS.

In summary, although the findings from the ITS analysis are mixed, with the data acquisition problems identified, it would be possible to draw conclusions in either direction regarding evidence for the effectiveness of the RPIW interventions based on the ITS alone. The inability to obtain data on the range of clinical outcomes targeted by the RPIWs (which have, by necessity, used mainly administrative variables) may mean that the ITS missed significant impacts in the other key outcomes. The findings from the ITS should therefore be placed in the context of the remainder of the study. In the context of wishing to use quantitative analyses to make causal inferences within this study, it should also be noted that, although undertaking an ITS was problematic in the context of this study, failing to use this more robust quantitative design would leave researchers using designs considered to be at greater risk of bias (such as uncontrolled after designs). The ITS analysis has, however, provided sufficient findings of interest that it would be worth considering the development of a well-designed prospective study to evaluate the effectiveness of RPIW-type interventions.

General messages from the study

Given the complexities and ever-shifting nature of some aspects of the NETS, it is difficult to come to any firm or final conclusions about its success (or otherwise) or its impact on either services or the public’s health status in the North East. Even where there may be evidence of change and improvement, attributing these either wholly or partially to the NETS, rather than to other factors or policy drivers operating at the time, is virtually impossible. A mixed-methods approach can help to try and cover all the bases but, even then, establishing causal links as distinct from strong correlations and associations from the data and their subsequent analysis is risky, and has to be approached with considerable caution and some humility. Indeed, adopting such a stance may mean rethinking what we understand by complexity and how we seek to research it, a topic considered in the final chapter.

From the interviews, observations and documentary analysis, a range of specific issues were identified:

• The choice of RPIWs was based upon the receptiveness to change rather than maximising return.

• The RPIWs were possibly too standardised and not tailored to the local context.

• The quality of follow up after 30, 60 and 90 days following a RPIW was variable, and on at least one occasion was not evident at all.

• Share-and-spread events and report-outs served as an effective means of dissemination and creating a regional community of practice.

• The NETS served as an enabling, ‘challenging up’ factor which gave staff permission to question and challenge existing practice and to come up with workable solutions.

• The NETS ethos changed over time following the NHS changes announced in 2010. From having been seen as a QIS, it came to be seen by the Coalition Board as a potential source of revenue. The original region-wide focus based upon the notion of ‘we are all in it together’ was undermined to a degree by the changing context which potentially sanctioned fragmentation and competition in place of collaboration and integration.

The real challenge facing us as researchers has been what to make of such findings in the totality of what constituted the NETS and its impact. We reflect on such matters in the next and concluding chapter.

Potential benefits of the research for the wider NHS

Berwick103 argued that complex, ever-changing systems such as health care should be learning organisations. They need to adapt and change direction as evidence becomes available which demonstrates what is working well and what may be defective and require modification or reform. Berwick makes the point in his report on the failings at Mid Staffordshire: ‘The NHS should become a learning organisation. Its leaders should create and support the capability for learning, and therefore change, at scale within the NHS’ [© Crown copyright 2013, contains public sector information licensed under the Open Government Licence v2.0 (www.nationalarchives.gov.uk/doc/open-government-licence/version/2)]. 104 At the same time, it is also claimed that organisations like the NHS are not good at spreading and sharing learning. 14 This is even the case when examples of good practice have been documented and published. Organisations tend to be slow to adapt and hang on to outmoded procedures or ways of operating. This is because they are regarded as familiar and/or they are defended by powerful vested interests, protective of what they regard as valuable and worth preserving.

The NETS was an example of a transformational change initiative that affected an entire health region comprising numerous organisations engaged in the commissioning and provision of health and health services. Within the confines of this community that served a population of 2.4 million people, the NETS was itself an example of spreading and sharing at work. The SHA saw part of its role as ensuring that internal learning was effective throughout the region. Indeed, it was keen to commission research into the impact of the NETS as part of this process. The manner in which the study was conceived was a good example of a NHS organisation working with the NIHR Health Services and Delivery Research (formerly SDO) programme to produce the research brief which resulted in the study reported here being established.

The NETS attracted much interest from other parts of the NHS, with site visits from groups of managers and practitioners a regular feature at some of the study sites. Just as the NETS was informed by developments at the VMMC, adopting and subsequently adapting the VMPS, so the early leaders of what was to have become the North East Production System felt compelled to spread the word beyond NHS NE.

Challenges of conducting research in complex systems

The NETS is an example par excellence of a complex intervention occurring within a complex system. This poses particular challenges for research and researchers, especially with regard to the generalisability and reproducibility of findings. Arguably, the richer the appreciation of complexity, the more difficult it becomes to generalise the findings and apply them elsewhere.

It is simplistic to assume that researchers can isolate the system (in this case the NETS) from its environment, as would be usual in simple randomised controlled trials (RCTs) as a route to understanding causality. More complex, pragmatic RCTs with embedded process analyses are potentially capable of addressing these concerns and are used to do so in the areas of education, policing and the criminal justice system. However, such trials are complex and logistically complicated, but in principle they can be designed and run. Built on sound observational methods to identify a priori hypothesised covariates and important dimensions of a process analysis, they can have important advantages including the identification of causal links that can be applied elsewhere. However, the number of covariates and dimensions of process analyses will always be limited. Moreover, not everyone is convinced that it is possible to use RCTs in this way to fully comprehend complex systems and to apply the lessons elsewhere. It is the context that shapes the intervention and can even determine whether it succeeds or not. In this view, reducing phenomena to their constituent variables is not especially productive, even if it were possible to do so. Thus, key properties of the interactions are emergent rather than determined and contingent upon one another, while significant elements remain uncertain and unknown. Research can aid understanding of how such systems operate while acknowledging that it may never be possible to completely comprehend how that system functions or what the precise mix of factors might be.

The adoption of an ecological approach to complexity views it as an open system that is both dynamic and gives rise to a series of interacting processes, the outcome of which informs and shapes that complexity. 152

The research team shares the view that engaging with complexity is not simply a case of developing more carefully contructed efforts to capture additional data. Nor is it a case of developing more sophisticated models to establish causation or those factors which might account for complexity. 152 The research team has wrestled with these issues. It is not enough to acknowledge the existence of complexity in health care, but it is necessary to find ways of engaging with its dynamic variability and constantly changing context. Such issues were thrown into sharp relief when the current NHS changes were announced in mid-2010, as these had the effect of instantly shifting the ground beneath the NETS. Life was simply not the same after the NHS White Paper appeared in July 2010. 1 In short, as Cohn et al. 152 put it, ‘the challenge is how to go about studying complexity without fully unravelling it’.

Complex health interventions like the NETS will always be highly variable because of the inherent dynamic character of their constituent parts, and as a result of the inevitable adaptations that emerge from their implementation in particular contexts. These are influenced and shaped by numerous local and national factors which cannot always be foreseen or predicted. Health-care systems will therefore remain complicated to study, especially when unpredictable events occur. Moreover, evaluation methods all have their limitations.

Key implications

Given the conclusions in the previous section, it would be both presumptuous and inappropriate to list a set of firm recommendations which, if adopted, would allow the NETS to be replicated and to succeed elsewhere. The study did not follow a linear journey from the identification of a problem, through the adoption of an intervention, to final impact on quality of care. The journey itself is still ongoing, as was recognised from the outset by one of the NETS’s architects, because striving for continuous improvement, by definition, never ends. The best we can do, therefore, is to offer a set of key implications arising from the research, identifying those influences and factors that were encountered on the journey that seemed to be especially critical in bringing about sustainable change. Of course, these influences and factors may not work in quite the same way in any given situation or context. Furthermore, a recognition and acknowledgement from the outset that they cannot be applied in a simplistic ‘lift and shift’ manner ought to go some way to reducing any sense of disappointment should failure occur. The architects of the NETS, as well as the NHS organisations engaged in its implementation, also struggled with this dilemma.

The key implications arising from the research are in no particular order and are all, to a degree, inter-related.

Implications for future research

In the course of our evaluation, a number of further areas for research have come to light.

First, in light of the nature of complex systems and the challenges these pose for research, there is a need for adopting new and different methods to understand how change occurs, or fails to occur, in the NHS. A mixed-methods approach was adopted and it seems that the ITS component raised as many questions as it sought to address. The ITS may be a useful analytical tool under certain circumstances and when the appropriate data exist. In this context of assessing the quality of care, the requisite data were not always available, which is a comment on the (non-)availability of data that are regarded as important and the utility (for this purpose) of the data that are routinely collected. As things stand, qualitative methods would appear to offer greater potential by way of offering illuminating insights into how change occurs and with what impact. Evaluative frameworks are needed which are able to establish how causal relationships emerge.

Second, the research was wide-ranging in its sweep across a number of organisations involved in the NETS. This was intentional and there was merit in attempting to capture the totality of what the NETS sought to achieve, as the integrated approach it sought to pursue was intrinsic to its purpose and design. But although breadth was important, it inevitably meant some sacrifice in terms of depth. There is a case for exploring each of the organisations engaged in the NETS in greater depth, employing a range of methods including observational studies. Limited observational work was undertaken but this could usefully be expanded to include all levels of the organisation.

Third, the importance of leadership has been highlighted. Further exploration of its nature, and those particular features that are especially critical in driving and sustaining transformational change in complex settings, is required.

Fourth, as noted in Chapter 8, Reflections on the interrupted time series, there were limitations arising from the ITS part of the study, particularly with regard to getting access to data retrospectively. There might therefore be merit in considering a well-designed prospective study to evaluate the effectiveness of RPIW-type interventions.

Conclusion

At one level, it will never be known if the NETS would have succeeded in its ambition as it was not allowed to run its course as originally conceived. Given that the NETS embarked on a journey that had no specific end point, it is difficult to know what would have been an optimum length of time to be able to say with reasonable confidence whether it had succeeded or not. Moreover, the venture was arguably high risk in a context such as the NHS where continuous, and often disruptive, organisational change seemed to be a feature of life. Assuming that the NETS could survive intact within such a context was arguably naive. Therefore, given the numerous changes that have occurred in an already complex environment, evaluating the impact of the NETS has not been straightforward, nor has it given rise to clear outcomes which can be tracked back to the role played by the Vision, Compact and Method. The research has been able to document much of the story of the NETS as well as identify ways in which it was perceived to have had an impact on staff and services.

After 2010 the NETS continued, although in a very different form from that envisaged at the outset. However, the research captured many examples of transformational change at a local level that led to improved patient safety, increased quality of care, improved service delivery and efficiency savings. In these ways, the NETS has undoubtedly been a success, despite the fact that it did not achieve the system-wide transformational change that was its central mission and the desire of its founders. In two of the sites the NETS continues to be influential. This might not have occurred without NHS NE’s drive and enthusiasm for a new approach to transformational change inspired by VMMC and TPS. These organisations have many best practices which can be disseminated. Perhaps, in such a complex system, securing that legacy in itself is no mean feat. Moreover, it is hoped that the evaluation of the NETS has revealed a number of lessons and insights which will have a continuing resonance and value when it comes to future transformational change initiatives in the NHS. If there is one certainty, it is that such initiatives will continue to be necessary.

Contribution of authors

David J Hunter was the principal investigator and was involved in the study design, collection of data and data analysis, and contributed to the writing of the final report.

Jonathan Erskine was project manager for the study, and was involved in the study design, collection of data and data analysis, and contributed to the writing of the final report.

Chris Hicks was involved in the study design, collection of data and data analysis, and contributed to the writing of the final report.

Tom McGovern was involved in the study design, collection of data and data analysis, and contributed to the writing of the final report.

Adrian Small was involved in the collection of data and data analysis, and contributed to the writing of the final report.

Ed Lugsden was involved in the collection of data and data analysis, and contributed to the writing of the final report.

Paula Whitty was involved in the study design, collection of ITS data and data analysis, and contributed to the writing of the final report.

Ian Nick Steen was involved in the collection of ITS data and data analysis, and contributed to the writing of the final report.

Martin Eccles acted as advisor to the study throughout its duration, commenting on data analysis and draft reports.

Publication

Erskine J, Hunter DJ, Small A, Hicks C, McGovern T, Lugsden E, et al. Leadership and transformational change in healthcare organisations: a qualitative analysis of the North East Transformation System. Health Serv Manage Res 2013;26:29–37.

Disclaimers

This report presents independent research funded by the National Institute for Health Research (NIHR). The views and opinions expressed by authors in this publication are those of the authors and do not necessarily reflect those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health. If there are verbatim quotations included in this publication the views and opinions expressed by the interviewees are those of the interviewees and do not necessarily reflect those of the authors, those of the NHS, the NIHR, NETSCC, the HS&DR programme or the Department of Health.

Aim

The NETS EAG will, as required by the research team and as set out in the original proposal submitted to the NIHR SDO programme, review and provide independent comment on, and support in regard to, the progress of the evaluation of the NETS at all stages.

Objectives

The EAG will draw on its members’ expertise to

• offer advice on the progress of the research throughout its duration

• review and comment on working papers as they emerge and perhaps, on occasion, and where deemed desirable or helpful, on papers in preparation for publication

• suggest ways in which the products from the research might be disseminated throughout the NHS and other organisations to maximise their value and impact.

Method of working

The research started on 1 December 2009 and will end on 30 November 2012. The EAG will meet twice each year for the duration of the research study. These face-to-face meetings will be supplemented as necessary, and by prior agreement, by e-mail contact with some or all members of the EAG in order to seek opinion and/or advice on particular issues.

Every effort will be made to ensure that the demands made of the members of the EAG are kept to a minimum given that they are giving freely of their time and have other commitments.

All reasonable travel and associated subsistence expenses will be reimbursed to EAG members.

The contribution made by EAG members will be appropriately acknowledged in any outputs from the study.

Research study: An evaluation of transformational change in NHS North East

Chief investigator: Professor David J Hunter

Further information: topic guides for interviews with staff

Interviews – confidentiality of responses reinforced

Introduction