A web-based self-management programme for people with type 2 diabetes: the HeLP-Diabetes research programme including RCT

Potential benefits of web-based diabetes mellitus self-management education

The web appeared to have considerable potential to improve access to, and uptake of, DSME. Internet access in the UK in 2011 was 73%,26 having increased year-on-year with an expectation of continued growth; indeed, latest estimates suggest that 86% of households had internet access in 2015. 27

Computer and web-based interventions were known to have specific advantages, including convenience (accessible at any time), anonymity (important to people with a stigmatised condition such as T2DM) and easy updating. A particular potential benefit appeared to be the ability to provide the entire range of support needed for a person’s illness journey, since diagnosis to end-stage disease. Good design could ensure that users accessed only the information and services needed at that time but, as needs evolved, they could find resources to match. The processing power and connectivity of desktop, laptop or handheld computers and smartphones allowed for interactive, tailored interventions that could respond to data entered by users with personalised information and advice. Such interventions could provide support for behaviour change,28–31 improve mental health and emotional distress32–35 and offer peer support. 36 There was evidence to support their use in long-term conditions (LTCs),37 including in diabetes mellitus,38 asthma39 and hypertension. 40

Potential pitfalls of web-based diabetes mellitus self-management education

Even at the time that this grant was conceived (i.e. 2009–10), it was clear that the potential benefits of web-based self-management interventions were hard to achieve and that there were a number of pitfalls. Challenges that were particularly evident were:

1. low rates of uptake and usage of such interventions41,42

2. very variable clinical effectiveness between different interventions, with no real understanding of the causes for this43,44

3. significant problems with implementation of digital interventions, with, at that time, almost no examples of their successful integration into routine health care45

4. the digital divide – the divide between those who did and did not use digital technologies. 46

This programme of work was undertaken with these challenges in mind. The process of developing the self-management programme was designed to optimise uptake and use, while maximising the likelihood of clinical effectiveness and future implementation into routine NHS services.

Low uptake and usage

We tackled the problem of low uptake and low usage in several ways. First, we followed the principles of participatory design and heavily involved members of the target user population in the development of the intervention. The aim of participatory design is to ensure that the intervention meets user requirements and is appealing and easy to use.

As the intervention was designed to be used by people with T2DM, in partnership with their HCPs, we defined the target population as people with T2DM and the HCPs with responsibility for them, including general practitioners (GPs), practice nurses, specialist diabetes nurses, consultants in diabetes medicine and dietitians.

This involvement was a three-stage process: (1) we undertook qualitative work with the patients and HCPs to identify the ‘wants and needs’ for such a programme; (2) we recruited patients and professionals to contribute on an ongoing basis to the intervention’s development, including seeking input on decisions for content, look and feel, tone, navigation and functionality; and (3) we undertook user testing of the intervention, asking patients and HCPs to review the programme and identify errors in content, bugs or glitches in functionality and problems with design or navigation. This process is described in detail in Chapters 4–6.

Second, we thought that uptake and use would be enhanced by ensuring that the self-management programme was integrated into people’s routine health care and seen by patients and HCPs as an integral part of the total care package. Previous data have shown that patient-centred clinician communication improves diabetes mellitus self-management behaviours. 47,48 Educational and self-help programmes that are actively supported by clinicians can improve health outcomes for people with LTCs,49 and in people with low health literacy the impact that written material has is increased by verbal recommendations from the HCP. 50 Thus, we conceived of the intervention as a web-based programme together with interactions between HCPs and patients around the programme (Figure 2).

FIGURE 2.

The intervention: not just a computer program.

We thought there were at least three ways that HCPs could promote the programme to patients: first, by introducing them to the programme, explaining how it could help them with managing their diabetes mellitus and achieve better health status and quality of life and providing some initial training. Second, we thought that telephone calls would help to encourage the uptake and use of our self-management programme, particularly when patients were new to it and might need a bit of encouragement or help. This was based on experience from the Improving Access to Psychological Therapies programme, which facilitated access to computerised cognitive–behavioural therapy (CBT) through graduate mental health workers who first introduced patients to specific computerised CBT programmes and then telephoned them regularly to encourage adherence to the programme. Third, we hoped that HCPs would refer to our self-management programme in consultations, for example when discussing care plans with patients, or by reviewing any self-monitoring data the patient had entered into the programme.

One inference of this approach of integrating our programme into routine health care was that some linkage between the self-management programme and the patient’s EMR would be useful, and such a linkage formed part of our original grant application.

Variable effectiveness and problems with implementation

At the time we wrote the grant and developed the intervention, relatively few data existed to explain why some web-based interventions were effective and others were not. 43,44 What few data existed supported the expectation that interventions that were based on theory were more likely to work than those that were not. In the light of this, and following Medical Research Council (MRC)’s guidance for development of complex interventions (Figure 3),51–53 we adopted a strong theoretical framework to guide the development of the intervention. We also followed best practice in using theory to guide us in considering implementation from the outset, aiming to ensure that the intervention was maximally ‘implementable’ and would fit easily into existing NHS structures and workflows.

FIGURE 3.

The MRC framework. 51 Reproduced from Developing and evaluating complex interventions: the new Medical Research Council guidance, Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance, 337, a1655,51 Copyright © 2008, with permission from BMJ Publishing Group Limited.

Reproduced from Developing and evaluating complex interventions: the new Medical Research Council guidance, Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance, 337, a1655,51 Copyright © 2008, with permission from BMJ Publishing Group Limited.

We used three theoretical frameworks or approaches to guide us:

1. Corbin and Strauss’ model54 of the work of managing a LTC

2. Abraham and Michie’s taxonomy of behaviour change techniques (BCTs)55

3. normalisation process theory (NPT). 56

We worked within the paradigm of evidence-based medicine, identifying and applying the best-available evidence for treatment of diabetes mellitus and for any decisions made during development (e.g. around maximising acceptability, uptake, usage and clinical effectiveness).

Normalisation process theory

The NPT56 is a mid-range sociological theory that explains why interventions do or do not ‘normalise’, that is become integrated into routine practice. It focuses on the work of implementation, integration and embedding new practices, ways of working or other interventions. It has four main constructs: coherence, cognitive participation, collective action and reflexive monitoring (Figure 4).

FIGURE 4.

Normalisation process theory.

Coherence refers to the ease with which the intervention can be described, understood and distinguished from other interventions or practices. This construct also includes an understanding of the problem that the intervention is designed to address, how the intervention could benefit its target population and what work will be needed for these potential benefits to be realised. Cognitive participation is about the decision whether or not to participate in the intervention. This will include an assessment of the relative benefits of participation (to patients, professionals or the health-care system) weighed against the costs of participation, in particular the expected impact on workload. Collective action is about the impact that the intervention has on the work undertaken by professionals within the organisation and reflexive monitoring is the process of considering whether or not the benefits of the intervention are worth the effort required to implement it. Within this construct is the possibility of altering or adapting the intervention to make it easier to implement in any given setting or organisation.

In turn, collective action has four subsidiary constructs: interactional workability, relational integration, skillset workability and contextual integration. Interactional workability refers to the degree to which the intervention facilitates or impedes the work of professional–patient interactions (consultations). Relational integration is the degree to which the intervention promotes or hinders good communication and relationships between different professional groups, including the degree to which accountability and responsibility are aligned. Skillset workability refers to the degree to which the intervention fits with existing skillsets or roles and the amount of training required to use the intervention. Finally, contextual integration is about the degree to which the intervention fits with existing policies, priorities and practices within the organisation.

The NPT56 predicts that interventions that improve consultations, promote good relationships between professionals, with accountability and responsibility well aligned, that need little training and that fit well with organisational priorities are more likely to normalise than those that do not.

Applying NPT to the development process meant that we were mindful of the following needs for the final self-management programme:

1. it should be easily described, easy to differentiate from other programmes and have clear benefits

2. it should fit with organisational and professional priorities, including enabling people with T2DM to self-manage care, reducing demand on professional time, adhering to NICE guidance and being accessible to a wide range of people

3. it should fit easily into existing working practices and be compatible with existing technology

4. it should make consultations between HCPs and patients easier and more productive and should be very easy to use.

Further details about how we applied NPT to the development of the intervention, and how NPT informed our implementation strategy, can be found in Chapters 5, 6 and 8.

The digital divide

The term ‘digital divide’ refers to the gap between those who do and do not have access to, or make use of, information and communication technologies. 46 At the time that this grant started (i.e. 2011), there were marked inequalities in internet access, with age, income, educational status and health status all being associated with access. Figures from the 2011 Oxford Internet Institute survey26 showed that, although about 85% of people who were at prime working age (25–55 years) used the internet regularly, only 33% of those aged ≥ 65 years did. Some 99% of households with total annual household income of ≥ £40,000 had internet access, but only 43% of households with an annual income of ≤ £12,500 did. Internet use was around 95% among people with degree-level education, but only 54% in those with a basic or secondary school education. The level of internet use in people with a disability was 41%; however, among people without a disability it was 78%. 26

There was evidence that the digital divide could be overcome with appropriate infrastructure, resources, training and design of interventions. Infrastructure requirements included provision of access to the internet, and the UK has benefited from a policy environment that promoted universal access. For example, most public libraries provide up to 1 hour per day per user of free access to an online computer and many local authorities fund community cluster rooms with associated training opportunities. 63 Provision of appropriate resources and training has been shown to enable diverse disadvantaged populations to make meaningful use of internet resources, including homeless drug users,64 vulnerable elderly people,65 parents of children attending early learning centres66 and people with cardiovascular disease67 or human immunodeficiency virus (HIV) infection or acquired immune deficiency syndrome (AIDS)68 from underserved communities.

We took a multipronged approach to attempting to narrow the digital divide and to ensure that our proposed intervention would be used by people from diverse socioeconomic backgrounds. We thought that people would be more likely to use the programme if it were recommended by trusted HCPs and integrated into routine care. Our previous research had shown that even short training sessions could enable some people who were not used to using computers to be able to use well-designed web-based interventions,69 although some people needed ongoing support. We thought that ongoing engagement would be promoted by HCPs referring to the programme in their consultations with the patient. Finally, we aimed to ensure that the intervention could be easily used by people with low literacy or computer literacy skills, by aiming for intuitive navigation, having liberal use of graphics, using text written for a reading age of 12 years and ensuring that key information was provided in video format, as well as text.

Health and Social Care Act 2012

There were a number of significant changes in context between the time that funding for our programme of research was confirmed (in 2010) and its completion (in 2016). By far the biggest of these was the Health and Social Care Act 2012,70 which was described as the ‘biggest single reorganisation’71 and the ‘longest and most complex piece of legislation’ in the history of the NHS. 72

Among its many components was the abolition of primary care trusts (PCTs) and strategic health authorities, which had been responsible for commissioning services. PCTs were replaced by clinical commissioning groups (CCGs), which were intended to control around 60% of the NHS budget, be led by GPs supported by other clinicians and managers and were tasked with meeting the needs of their populations.

This reorganisation resulted in huge workloads for those involved as they struggled to come to terms with new priorities and responsibilities, evolving structures and changes in personnel. This was often accompanied by uncertainty about roles and responsibilities, loss of existing staff with relevant expertise and loss of organisational memory. 73–75

This massive reorganisation occurred at the same time as the NHS entered a period of significant financial austerity. After a period of year-on-year growth in budget, the NHS was charged with making financial savings of £20B over 5 years from 2011–15 while maintaining (or improving) the quality of the service. 76 This required efficiency savings of around 4% per year, compared with maximum previous efficiency savings of around 2% per year. It was recognised that meeting this ‘unprecedented challenge’ would require new ways of working, with an emphasis on reducing hospital admissions for people with LTCs, as well as there being a pay freeze for NHS staff. 76

Not entirely coincidentally, English general practice was, at the same time, entering a period of ‘crisis’,77 with a rapidly rising workload because of increased numbers and complexity of consultations but no concomitant rise in HCP numbers and static or falling incomes. 78,79 As a result, general practices were under enormous pressure. Many were unable to fill vacant clinical posts (both doctors and nurses were hard to recruit), leading to excessive workloads for remaining clinicians. 80 This was reflected in long waiting times for appointments and many GPs reporting low morale, burn out and resistance to change. 80 Although some practices showed considerable resilience, others went into a spiral of decline. 77

This turbulent background had a considerable impact on our research, particularly when it came to implementing the self-management programme into routine care (see Chapter 9).

Closure of the Medical Research Council’s General Practice Research Framework

Our original application included an individually randomised controlled trial of the intervention to be run in primary care. At the time the proposal was submitted, we had a close collaboration with the MRC’s General Practice Research Framework (GPRF) and had designed the trial with the resources and expertise of the GPRF in mind. GPRF practices had trained research nurses available to participate in studies and, therefore, we designed a study that was predicated on unblinded practice nurses undertaking clinical tasks and delivery of the intervention and blinded research nurses undertaking research tasks, including data collection.

However, in 2012, the GPRF was subsumed into the wider National Institute for Health Research (NIHR)’s Primary Care Research Network (PCRN). The PCRN operated a different model of supporting primary care research, with resources focused on promoting recruitment. Most PCRN hubs did not have the resources available to support data collection. This had quite an impact on the feasibility of the trial and, although in the end we were able to recruit sufficient practices and exceed our planned sample size, there were a number of challenges en route that required us to show agility and adaptability (see Chapter 7).

Changes to the Quality and Outcomes Framework

In 2013, the Quality and Outcomes Framework (QOF) was changed to include an incentive for GPs to refer people newly diagnosed with diabetes mellitus to structured education within 9 months of diagnosis. This had an immediate impact on referral rates, which rose from 15% in 2012 to 75% in 2014. Our programme was not designed as ‘structured education’ for newly diagnosed people, but rather as ongoing self-management support for people throughout their illness journey. This mismatch between policy and our intervention affected our research and required us to adapt some of our original ideas. In particular, it led us to develop an additional structured component to our intervention, aimed at newly diagnosed people (see Chapter 3).

Development

1. Determine patients’ perspectives of the essential and desirable features of the intervention (wants and needs).

2. Determine HCPs’ perspectives of the essential and desirable features of the intervention that would encourage uptake and use in the NHS.

3. Determine the overall content and function of the intervention.

4. Determine the optimal facilitation required to encourage use of the intervention.

5. Determine feasibility, acceptability and short-term effects of facilitated access to the intervention in a naive population.

Evaluation

1. Determine the effect of the intervention on clinical outcomes and health-related quality of life (HRQoL) in people with T2DM.

2. Determine the incremental cost-effectiveness of the intervention compared with usual care from the perspectives of health and personal social services and wider public sector resources.

Implementation

1. Implement the intervention in two PCTs (later redesignated as CCGs).

2. Determine the uptake, use and effects of the intervention in an unselected population in routine care.

3. Determine factors that inhibit or facilitate integration into existing services and use of the intervention.

4. Determine the resources needed for effective implementation.

The development and formative evaluation of a cardiovascular risk calculator for people with type 2 diabetes mellitus83

Dates: 2012–13.

Lead: Dr Tom Nolan (GP academic registrar).

Collaborators (additional to programme grant co-investigators and UCL team): Professor David Spiegelhalter (Winton Professor of Public Understanding of Risk, Statistical Laboratory, Centre for Mathematical Sciences, University of Cambridge) and Mike Pearson (Statistical Laboratory, Centre for Mathematical Sciences, University of Cambridge).

Additional funding: none.

An evaluation of the impact that the Healthy Living for People with type 2 Diabetes programme had on the psychological well-being of patients with type 2 diabetes mellitus: a mixed-methods cohort study84

Dates: 2012–14.

Lead: Megan Hoffman (Doctor of Clinical Psychology student). The doctorate was awarded in 2014.

Collaborator (additional to programme grant co-investigators and UCL team): Professor Chris Barker (Professor of Clinical Psychology, Department of Clinical, Educational and Health Psychology, UCL).

Additional funding: none.

A systematic review of technological prompts to improve engagement with digital health interventions85,86

Dates: 2013–15.

Lead: Ghadah Alkhaldi (PhD student). PhD awarded in 2016.

Collaborator (additional to programme grant co-investigators and UCL team): Dr Fiona Hamilton (NIHR Lecturer in Primary Care, Research Department of Primary Care and Population Health, UCL).

Additional funding: PhD studentship funded by the Saudi Arabian Cultural Bureau.

Two randomised controlled trials of the use of e-mail and short message services to improve engagement with the Healthy Living for People with type 2 Diabetes programme81

Dates: 2014–16.

Lead: Ghadah Alkhaldi (PhD student). PhD awarded in 2016.

Collaborator (additional to programme grant co-investigators and UCL team): Dr Fiona Hamilton (NIHR Lecturer in Primary Care, Research Department of Primary Care and Population Health, UCL).

Additional funding: PhD studentship funded by the Saudi Arabian Cultural Bureau.

A systematic review of the implementation of eHealth interventions: update of a review of reviews82

Dates: 2014–16.

Lead: Jamie Ross [research associate (RA) and PhD student], supervised by Professor Elizabeth Murray and Dr Fiona Stevenson. PhD awarded in 2016.

Collaborator (additional to programme grant co-investigators and UCL team): Rosa Lau (PhD student).

Additional funding: none.

The development and formative evaluation of a structured education programme for newly diagnosed patients with type 2 diabetes: The Healthy Living for People with type 2 Diabetes programme – Starting Out (papers not yet published)

Dates: 2014–16.

Lead: Shoba Poduval (academic GP fellow).

Collaborators (additional to programme grant co-investigators and UCL team): Helen Gibson and Rebecca Owen (diabetes specialist nurses).

Additional funding: the National School of Primary Care Research funding round 9.

The development of a digital diabetes prevention programme: HeLP Stop Diabetes (papers not yet published)

Dates: 2016–18.

Lead: Marie-Laure Morelli (academic GP trainee).

Collaborator (additional to programme grant co-investigators): Paulina Bondaronek (PhD student).

Additional funding: the National School of Primary Care Research funding round 11.

Rationale for study objectives

Establishing user requirements for any proposed intervention is a necessary first step. 87 We conceptualised these requirements as ‘wants and needs’, for which ‘wants’ are features that users actively desire and would make them want to use the intervention and ‘needs’ are features that evidence suggested would improve health outcomes.

We postulated that combining wants with needs should combine the attractiveness and appeal of many commercial digital interventions (including games) with the clinical effectiveness of face-to-face interventions. As an example, there was evidence from the literature that forums, where users could interact online, were associated with increased use of an intervention, as people wanted the opportunity to interact with others in similar situations (a ‘want’). 88 However, nothing suggested that this improved health outcomes. 89 In contrast, improving medication adherence is likely to improve health outcomes (a ‘need’), but including such a facility in an intervention is unlikely to promote engagement with, and use of, the intervention. Our goal was to understand both wants and needs with a view to creating an intervention that met both and, hence, was both useful and used.

We were also aware that patients and HCPs used different criteria for assessing web-based self-management interventions90 and that HCPs’ perceptions of patient requirements were often inaccurate. 91 For example, there is evidence that HCPs overestimate patients’ concerns about complications and underestimate their concerns about dietary restrictions. 92 Hence, we were clear that we needed separate, parallel studies to determine patients’ and HCPs’ perceptions of the essential and desirable features of such an intervention.

Previous research by Kerr et al. 90 had established some generic patient requirements for self-management interventions for LTCs. Kerr et al. 90 convened 10 focus groups with 40 patients and carers who generated detailed quality criteria relating to information content, presentation, interactive components and trustworthiness. Participants in that study stated that information needed to be detailed, specific and of practical use. They also advised that long-term use required increasing the depth of information as self-management experience increased, as well as providing new and up-to-date information. Participants wanted information about their condition and the treatments available, practicalities around day-to-day living (holidays, travel and eating out), local services and resources, new research and areas of scientific uncertainty and other people’s experiences and information for family members. They felt that it was important for users to control how much information they accessed at any time and the topic of the information. This was particularly important for ‘bad news’, which they did not want forced on them.

These criteria focused on the need for excellent website design, easy navigation giving intuitive and speedy access to relevant content and an attractive appearance, using colours, graphics, videos, animations, photographs and text broken up into small sections. The tone should be straightforward but not patronising; medical terms and jargon should be explained but not avoided. The criteria for interactive components were that they should be optional, use online assessments to provide tailored advice and monitoring and include an ‘Ask the Expert’ facility and online forum. Finally, trustworthiness was vital and could be established and maintained by information being accurate and regularly updated, the intervention having no commercial links or advertisements and the website being authored or sponsored by a known and trusted organisation, such as the NHS or a local hospital or well-known university or charity. 90

However, technology and website use had changed considerably since the above research was published. Website use had become much more interactive and peer to peer (the so-called Web 2.0). 93 Moreover, although this research had included people with T2DM, it had not specifically focused on diabetes mellitus; rather, it had looked to draw out criteria that were transferable across a range of LTCs. There were no studies in the literature that we could identify that specifically explored the wants and needs of people with T2DM for web-based self-management programmes.

As described in Chapter 2, well-recognised limitations of web-based interventions are non-use and high rates of attrition. 41 In our original grant application, we postulated that engagement with the intervention was likely to be enhanced by HCPs recommending and endorsing both initial and ongoing use. Evidence from internet cognitive–behavioural therapy (ICBT) suggested that facilitated or supported use of ICBT was more effective than unsupported use. 94 However, we also recognised that it is hard to change professional behaviour,95 and that HCP time is a scarce (and expensive) resource. Therefore, we wanted to explore with people the type of facilitation they would find useful and what they thought was realistic.

We were also specifically interested in the possibility of linking the self-management programme with patients’ EMRs. While the work for WP A was being done, there was considerable discussion about the relative advantages and disadvantages of such access. Some pilot studies had suggested that it was feasible, did not lead to excess workload or patient anxiety but did lead to correction of inaccurate data in the record, better-informed patients and more productive consultations. 96–98 However, mainstream opinion, as represented by, for example, the British Medical Association, was unconvinced, citing concerns about privacy, confidentiality, causing unnecessary anxiety among patients and about it resulting in additional workload for doctors. 99 We wanted to explore with patients what they thought about having access to their EMR and, in particular, what information they wanted to see and why. We planned to see whether or not it was possible to find a solution that addressed patients’ wants and needs while respecting clinicians’ concerns.

As an example of the changing context during our research programme, shortly after this study was completed, the Department of Health and Social Care issued new policy, mandating that all patients should have access to their EMR. 100

Rationale for study methods

Two key methodological decisions were to use focus groups for data collection, and, as part of the focus groups, to show participants three existing examples of self-management websites for people with T2DM. These decisions were based on our previous experience with this methodology,90 which had proven feasible and yielded rich data. This work was in turn based on a study by Coulter et al. ,101 which determined quality criteria for traditional paper-based information materials for users.

The use of qualitative methodology allowed participants to explore issues of importance to them, in their own words, generating their own questions and focusing on their own priorities; this was particularly important when exploring self-management and interventions to empower patients. We opted for focus groups, rather than individual interviews, as we wanted participants to explore their underlying reasons for differing perspectives. Group discussions can yield rich data when participants explore and clarify their views through interactions with each other. 102 Group dynamics also involve many different forms of day-to-day communication, such as jokes, anecdotes, teasing and arguing. These natural interactions may allow a more nuanced appreciation of what people know or experience and can reveal ‘shared truths’. 103 A disadvantage of focus groups is that some participants may find it difficult to discuss sensitive or potentially embarrassing topics, such as sexual dysfunction, which can be a common complication of diabetes mellitus. Therefore, we planned to use focus groups as our primary method of data collection, while reserving the option of individual interviews if it appeared that some topics were not being adequately addressed.

The decision to show participants three websites at the start of each focus group was based on the observation that it is much easier to critique existing interventions than to think in the abstract about what one might want. Moreover, the experience of using such interventions allowed users to think about features they looked for, even if they were not present. Our experience from our earlier work had confirmed that participants could review three websites in the time available that, with careful selection, it was possible to ensure that most of the features of interest were included in the websites presented and that there was sufficient difference between the websites to enable participants to compare and contrast them in a fashion that generated useful data. 90

Methods

Results

The burden of diabetes mellitus

Although the topic guide did not directly address the question of what it was like to be diagnosed with diabetes mellitus and the impact that this had on participants’ lives, it was clear that participants were acutely aware of the severe and constant burden that they experienced as a result of the diagnosis. This burden was described most commonly in terms of the emotional impact, but participants also consistently referred to the difficulties their diabetes mellitus caused with their social life, relationships and work. As such, the data resonated strongly with Corbin and Strauss’ model54 of the work of living with a LTC, described in Chapter 2 (Table 3).

The following series of exchanges illustrate the data:

[The diagnosis of diabetes mellitus] was like a bereavement . . . It was just before my 50th birthday.

A lot of people say that, because you are, in a sense; you’re grieving for the loss of your . . .?

Liberty. Freedom.

Is that what it is? Right.

Health.

Yes.

Well, it’s more to do with the mortality business, isn’t it? Without wanting to sound grim, but . . .

Patient 5, male, aged 55 years, white British, 5 years since diabetes mellitus diagnosis and Patient 10, male, aged 70 years, white British, 6 months since diabetes mellitus diagnosis (focus group 1)

Many participants described how the demands of an illness that required them to take medication and eat regularly made it impossible for them to carry on with the work they had previously been doing:

I’m a project manager so I work all hours, different hours, and it was very difficult to manage my diabetes because if I’m going to a meeting at 8 o’clock at night, or going to a meeting at 6 o’clock, whatever, or . . . and breaking up . . . it was very difficult to manage it. Employers don’t really care about you’re diabetic, to be honest with you. . . . [Y]ou do forget to eat when you’re busy and you’ve got a meeting at 1 o’clock or 2 o’clock in the afternoon, or 12 o’clock in between your lunch break, and you can’t eat in the meeting because you’ve got about 20 people sitting there, or whatever the case may be. . . . I’ve just given up work about 4 weeks ago, so it’s just really hard to tell you the truth.

Patient 11, female, aged 51 years, black Caribbean, 10 years since diabetes mellitus diagnosis (focus group 2)

They took me off shifts, because I couldn’t remember if I’d taken my pills one week, you know? One week, I’m working early and next week, I’m working lates, and then I’m working nights, and I used to go, I can’t remember if I’ve taken them or not. My manager said, that’s no use, is it? We can’t have you falling over, you know?

Patient 17, male, aged 54 years, white British, 8 years since diabetes mellitus diagnosis (focus group 3)

The impact that dietary changes had on participants’ social and family lives was also difficult to cope with. Some participants found that it was too hard to keep to their planned diet when out with friends or family and would simply try to manage the consequences, whereas others would try to adhere to the changes they’d instituted but reported negative reactions from their families:

My wife was the only person that said anything that was reasonably sensible; she said, we’re going to this person’s house to eat, they need to know you’re a diabetic, otherwise they’re going to give you the same portions that you’ve always had, the same five pints of beer that you always have over lunch.

Patient 16, male, aged 58 years, white British, 4 months since diabetes mellitus diagnosis (focus group 4)

Because we’re talking about food; I mean, I go to my family, and when I say I can’t eat that food, they usually think that’s disrespecting them, so you’ve got all that as well to deal with.

Patient 11, female, aged 51 years, black Caribbean, 10 years since diabetes mellitus diagnosis (focus group 2)

Not surprisingly, given the impact that diabetes mellitus was having on their lives, participants reported strong negative emotions, which they found difficult to manage. Depression, anger, frustration and guilt were commonly felt:

Low, angry, frustrated. Everything. Because, you know, sometimes you’re frustrated because the doctor hasn’t told you what you want to hear. . . . Or you’re angry with the world, and you take it out on your children, your partners, everybody. And then you’ve got the depression that takes you down, because you’re just thinking one thing after another. So there is emotional change.

Patient 11, female, aged 51 years, black Caribbean, 10 years since diabetes mellitus diagnosis (focus group 2)

Stress and depression.

Yes, depression.

And, things related to that, because I know that depression is related to that, which is something, and when you are into that depression, you think very much negative about life and all that, you know? You say that, my sugar levels are not going to get controlled anytime, whatever I do.

Patient 18, male, aged 36 years, Asian Indian, 8 years since diabetes mellitus diagnosis and patient 15, male, aged 54 years, white British, 7 years since diabetes mellitus diagnosis (focus group 3)

Contributing to the difficulties experienced by participants was the perception that other people blamed them for their illness and that the relationship between lifestyle and T2DM led to stigmatisation:

Usually people say, oh, you must have had a bad lifestyle, something or whatever, which may be true sometimes, but it’s not the only reason . . .

Patient 19, female, aged 64 years, white British, 36 years since diabetes mellitus diagnosis (focus group 3)

Problems with existing health services, leading to unmet need

Although some participants were very appreciative of the care they had received from the health service, this was not a universal experience, with many participants reporting difficulties with access to HCPs, lack of interest or expertise among staff and an increasing sense of a ‘tick-box’ culture, in which problems were recorded but not addressed. Even participants who were positive about their care reported a feeling of never having enough time in consultations:

I’ve been very fortunate with my practice in [location] because they’ve given me a huge amount of support actually in terms of information gathering. But I understand that you’ve only got to be a couple of miles down the road and you get nothing at all. And even if you ask the questions, the doctors feel that you’re taking up their time, and in fact that’s true of all doctors, I appreciate that.

Patient 10, male, aged 70 years, white British, 6 months since diabetes mellitus diagnosis (focus group 1)

I found out far more information from those people [a local support group] than through from my local surgery. And it’s still the case now.

Patient 5, male, aged 55 years, white British, 5 years since diabetes mellitus diagnosis (focus group 1)

I’m asked that, once a year, that question, do you feel depressed? Yes. Next question. It’s not like, what are you going to do about it? . . . And when I see the nurse, every 6 months, she just says, are you exercising? And she ticks a little box if I say yes or no. And that’s it.

Patient 6, male, aged 55 years, black African, 10 years since diabetes mellitus diagnosis (focus group 1)

How a web-based self-management programme could meet this need

Participants reacted positively to our endeavour to create a web-based self-management programme for people with T2DM. They were clear that the programme should address all aspects of living with diabetes mellitus, including diet, physical activity, taking medicines, working with HCPs, managing difficult emotions and handling interactions at work, in social occasions and with friends and family.

They wanted information about diabetes mellitus, including how it is caused and how it affects the body; available treatments, including goals of treatment, pros and cons of each treatment and potential side effects; and access to a suite of resources. The participants wanted the programme to be a ‘one-stop shop’ that they could turn to at times of need. There were clear tensions between wanting or needing information and not wanting to be overwhelmed with ‘bad news’. It was vital that the information was presented in a cheerful and upbeat fashion, with an emphasis on what can be done to prevent complications, while acknowledging that diabetes mellitus can be hard to control and avoiding ‘victim blaming’ when things do go wrong:

I suppose I’d want something that was a bit, kind of, an A-to-Z of one’s life.

Patient 20, female, aged 41 years, white British, 5 years since diabetes mellitus diagnosis (pilot interview)

The mainstream medical opinion seems to be all doom and gloom . . . If you just put that diabetes is such-and-such but can be controlled or managed or whatever word you want to use, through very simple means, I think that’s a huge relief to people.

Patient 10, male, aged 70 years, white British, 6 months since diabetes mellitus diagnosis (focus group 1)

I think you’ve always got to look at the positive side of your illness. But yes, you’re always going to have a negative side, and sometimes you’ve got to have a funny side . . .

Patient 8, female, aged 46 years, white British, 16 years since diabetes mellitus diagnosis (focus group 1)

Tables 4 and 5 summarise participants’ views on desirable and undesirable content and presentation.

Views about health-care practitioner facilitation

Participants wanted any self-management support programme to be integrated into their general medical care, rather than as a stand-alone resource. They wanted to work with their health professionals to obtain good health outcomes, although there was a sense that health professionals were not always interested in their efforts:

Yes, I got told off last week for not taking mine [blood glucose diary] in, but part of the reason is, well, you never look at it. What’s the point of me taking it in? . . . It’s like a little token. You know, look, I have been recording, but he doesn’t go, so, what’s happening here?

Patient 15, male, aged 54 years, white British, 7 years since diabetes mellitus diagnosis (focus group 3)

Although, in principle, participants were strongly in favour of HCPs helping patients to register on the programme, showing them how to use it and discussing their use of the programme in diabetes mellitus-related consultations, they were sceptical whether or not this would be possible:

I think it would be a good thing but I can’t see many people doing it. . . . To even ring your GP surgery normally to make an appointment can be very tiresome for a lot of people. You can’t get through.

Patient 8, female, 46 years, white British, 16 years since diabetes mellitus diagnosis (focus group 1)

Access to electronic medical record

In line with their emphasis on the need for self-management to be integrated with HCP management, participants were keen to have access to their GP EMR. Perceived benefits included the ability to correct information in their records and to share correct information with the multiple health-care providers that they encountered, such as opticians, podiatrists, dentists and emergency doctors. Participants were keen to have control over their information, and to decide what to share with whom:

It might be useful to correct things, if you find anything that’s been recorded incorrectly, that at the moment, you’ve got no idea if . . . anything’s wrong or not.

Patient 16, male, aged 58 years, white British, 4 months since diabetes mellitus diagnosis (focus group 4)

It’s not anything to do with diabetes, but the worst thing is, when you go along to A&E [accident and emergency unit], and they say to you . . . ‘Oh, when were you diagnosed?’ You know, and you have to start . . . the whole story from the beginning, and if they’ve got access as well, so, you know . . .

Patient 7, female, aged 65 years, white British, 3 years since diabetes mellitus diagnosis (focus group 4)

Rationale for study objectives

In Chapter 3, we discussed the need to identify user requirements before developing a web-based self-management intervention. We conceptualised our users as belonging to two groups: (1) people living with T2DM and (2) HCPs who were involved in caring for people with T2DM.

As described in Chapter 2, one of our approaches to improving the uptake and use of the web-based self-management programme was to integrate it into routine health care, so that users perceived it as part of the overall package of care provided by the NHS. We believed that it was important that they were introduced to the self-management programme by a HCP, as this would provide credibility and reassure them that doctors and nurses approved of the programme and wanted them to use it. We considered that ongoing use of the programme would be promoted by doctors and nurses referring to it during consultations, asking patients how they were getting on with it and perhaps jointly reviewing personal goals or self-monitoring data (see Figure 2).

However, we were well aware of the difficulties of achieving such professional behaviour change. 95 Hence, the rationale for the objectives of this study included understanding what HCPs would want from such a programme, what they considered would make them want to use it, what the likely barriers to use would be and what (if anything) could be done to address these barriers. Identifying these factors would allow us to build in motivating features while addressing potential barriers.

Our intention was to develop a programme that would be used in primary care, so our main focus was on HCPs working in that sector. However, people with T2DM are cared for by a wide range of HCPs across primary, community and secondary care. The intervention needed to be acceptable to all of these HCPs, and we believed that it was important that specialists in diabetes medicine approved of the intervention. Hence, our sample included a range of HCPs involved in caring for people with diabetes mellitus.

Factors that the literature predicted would influence HCP views on the potential benefits and pitfalls of self-management interventions and web-delivery included age (with younger people often being more open to technology), gender (with women tending to adopt more patient-centred care supporting self-management) and disciplinary background. 111–115

Data from overseas studies of e-mental health interventions and ICBT demonstrated that facilitation by a HCP led to significantly improved use and greater clinical effectiveness. 94 The disciplinary background of the HCP did not seem to be important, with comparable results obtained by therapists and technicians. 116 Similar findings from the UK were obtained with telephone facilitation of a weight loss intervention. 117 Methods of facilitation that had been shown to be effective included e-mail and telephone support, usually at a frequency of once a week or fortnight. However, facilitation takes time, and time is a scarce resource in general practice. We wanted to explore with HCPs what they thought about facilitation, whether or not they thought it would be a legitimate role for general practice staff and, if so, for what type of staff [practice nurse, receptionist, health-care assistant (HCA) or other] and what level of frequency and intensity would be practicable.

Finally, we wanted to explore HCPs’ perceptions of the advantages and disadvantages of allowing patients access to some parts of EMRs. Early pilot studies suggested that the potential benefits of such access may include more accurate records, more empowered patients and a reduction in duplication of tests and investigations. 96,98 Participants in WP A (see Chapter 4) were keen to have access to at least some of their EMR, stating that it would help them with self-monitoring, managing their medication and understanding their health problems and in communicating with other HCPs. They wanted to be able to share their self-monitoring data with their clinicians. They said that such access would be a powerful motivator for using the self-management programme. In the UK, the public had had the right to see their paper medical record or EMR since November 1991, although relatively few had exercised this right. 118,119 At the time our study was undertaken (2011), there was considerable discussion around forthcoming legislation to mandate patients having online access to their EMR by 2015 (the legislation subsequently passed and people now have this right). 120 Many doctor organisations had expressed strong concerns around privacy and confidentiality. 99,121 Therefore, we thought it important to explore HCP views about providing some linkage between the self-management programme and the patient’s EMR and whether, even if they were opposed to full access to the record, HCPs would consider partial or limited access to EMRs acceptable within strictly defined parameters, such as medication and results of investigations. Given that patients wanted to share self-monitoring data with clinicians, we also wanted to explore this possibility. As our proposed intervention was intended to be used in primary care, and the majority of patients would be cared for in primary care, the possibility of linking with hospital records was not relevant and not pursued.

Rationale for study methods

The main methods considered for this study were focus groups and individual interviews. Focus groups have the advantage that areas of disagreement can be discussed, allowing for illumination and explication of reasons underlying such disagreement and the potential generation of consensual solutions acceptable to all participants. 102 An advantage of individual interviews is the ability to go into specific areas in greater depth, responding to the interviewee’s interests and enthusiasms. Interviews can also be helpful if there are sensitive issues to explore, such as the perceptions of one group of HCPs about the roles, strengths and weaknesses of other groups of HCPs, which participants may be unwilling to share in a less confidential setting. 122 Moreover, focus groups are substantially more difficult than individual interviews to arrange, as they require bringing participants together at a mutually convenient time and location. The challenges of recruiting HCPs to research are well known and were well understood by the research team. 123,124 In view of this, we opted for a combination of individual interviews and focus groups.

As HCP time is at a premium and participants were unable to allocate us more than 30–60 minutes of their time, we did not show them examples of existing self-management programmes as we had the patients. We considered that professionals had sufficient understanding of the type of intervention that we were trying to develop from other digital interventions already in use as part of EMRs or on referral (e.g. ICBT) and that they would know the features that would promote use or act as barriers.

Design

Qualitative study using individual interviews and focus groups for data collection.

Ethics

Ethics approval was provided by the North West London Local Research Ethics Committee on behalf of the National Research Ethics Service (reference number 10/H0722/86).

Setting and participants

Our recruitment strategy was designed to reflect our objectives by targeting a range of HCPs who are involved in caring for people with T2DM, with a focus on primary care. Hence, we sought to recruit GPs, practice nurses, diabetes specialist nurses, consultants in diabetes medicine and dietitians. We used a ‘snowball’ method of recruitment, starting with clinicians with whom we had personal contact and following up with contacts of interviewees. This was augmented by advertisements and flyers at primary care conferences and sending e-mails to HCPs delivering structured education in north London. We sought a sample that varied in its age, gender, ethnicity, disciplinary background and seniority and work setting (primary, community or secondary care). Recruitment continued until theoretical saturation was reached and no new data emerged in subsequent interviews or focus groups.

Data collection

A topic guide was designed that reflected our objectives and built on findings from the existing literature. The guide was informed by NPT56 and consisted of open-ended questions designed to encourage participants to think widely around the issues discussed during the interviews and to enable participants to bring up ideas that were important to them, rather than focusing specifically on issues predetermined by the research team.

The topic guide was piloted in an individual interview, and after revisions, repiloted in a focus group. This revised version was used for subsequent focus groups and interviews. The areas covered by the guide included challenges in managing patients with T2DM, potential benefits and pitfalls of using a web-based self-management programme, the impact on consultations and workload, facilitators of and barriers to implementation, strategies to combat attrition and views on patient access to their EMR.

Interviews were held at a time and location convenient for the interviewee, usually their place of work, and often in their lunch hour or after work. Focus groups were held either in participants’ practices or a convenient, quiet meeting room in an academic or NHS venue and were facilitated by a least two researchers. The lead facilitator chaired the discussion, while the second observed, contributing only when needed to clarify points of discussion. The observer kept extensive field notes, including real-time reflections on the data emerging.

Interviews and focus groups were audio-taped and transcribed by a professional transcription company. Transcripts were checked for accuracy by the interviewer or focus group facilitators.

Participants were also asked to complete a brief demographic questionnaire, providing information about age, gender, ethnicity, professional role and level of seniority.

Data collection continued until we reached theoretical saturation with no new data emerging.

Data analysis

A multidisciplinary team, comprising GPs, psychologists, sociologists and health service researchers, analysed the data. Transcripts were read by each member of the research team and then discussed in group meetings or data clinics. Emerging themes were agreed on and transcripts subsequently coded to these themes. 125 Codes and themes were iteratively discussed until agreement was reached by the whole team. Themes were subsequently mapped onto NPT as described in Chapter 2. This two-stage approach had the advantage of maintaining our theoretical perspective, while ensuring that we did not ignore or discard data that did not ‘fit’ with NPT. 126

ATLAS.Ti was used to manage the transcripts, coding and facilitate the final data analysis.

Illustrative extracts of the data are presented in Results, with identification by participant number and professional background.

Main themes

All the data that emerged mapped easily onto the main constructs of NPT. The following subsections are structured by NPT construct: Coherence, Cognitive participation, Collective action and Reflexive monitoring. Each subsection presents data, for the main proposed programme, for facilitated access and then, finally, for access to EMRs.

Coherence

Coherence refers to the ease with which the intervention can be described, understood and distinguished from other interventions or practices. This includes an understanding of how an intervention or practice differs from other interventions and an understanding at the individual and group levels of what the aims and expected benefits of the intervention will be and what individuals will have to do to realise those benefits.

For our participants, the concept of a web-based self-management programme had immediate coherence. Many of our participants were already convinced of the importance of self-management by people with T2DM, stating that patients played a key role in determining health outcomes:

Anything that I as a clinician will do is predicated on the other individual in the relationship actually playing their role, taking part, taking active control of their own lives. And the more information and understanding they have of what is going on, the more they’ll understand why I’m seeking to help them modify their behaviour.

Participant 5, GP partner (individual interview)

Participants were aware that people with T2DM often needed help and support to engage in self-management but that the dominant model of group-based education was not suitable for many:

Well, we don’t have that many people who are in full-time employment in [town] really . . . but they do have other priorities and, you know, their health may not be one of them, sadly. You know, social deprivation, poverty. You know, often we put on education at times that suit us, as health-care professionals, [but] they’ve got children to pick up from school or they’re looking after elderly parents or, you know, they’ve got children with special needs, they’ve got other illnesses themselves.

Participant 12, diabetes nurse consultant (focus group 3)

Other problems with group-based education included long waiting times and the fact that, in most areas, education was available only at the time point after diagnosis, whereas people with T2DM often had an ongoing need for education and support:

We get very, very good feedback about DESMOND, but the wait is just too long for them to go on it, you know. And plus once you’ve had that . . . initial training and education, it doesn’t mean that you’ve not got further training needs, and there’s a massive gap that we encounter for that extra bit of help for patients.

Participant 15, practice nurse (focus group 4)

Participants considered that a web-based programme would have a number of advantages over group-based education, including being available at the time when a patient was ready to engage, being able to be revisited frequently so that users can learn at a speed that suited them and that video and graphics can be used, which would help people with low literacy skills or poor English. Many commented on the volume and complexity of the knowledge required to self-manage effectively:

[It] is quite difficult . . . for the individual, understanding this cruel nature of type 2 diabetes. I think people understand the sort of sugary side of things, but actually understanding the impact on their kidney disease, their heart disease risk, all these other areas where long-term diabetes is important. It’s quite complicated [even] for us. You know, I think for people who perhaps don’t have that kind of medical understanding, it must be really quite daunting to take all that information on at the start.

Participant 6, GP partner (individual interview)

Clinicians were very aware of the emotional burden that diabetes mellitus placed on patients and were keen to see that addressed, recognising that this was a prerequisite before people with T2DM could start the work of taking control of their health:

One is often tempted to say, well it’s the educated people who get the whole thing quickly and they therefore deal with it. But I remember, certainly one patient of mine, a young . . . a very bright young girl, who just did not want to come to terms with her diabetes, and she was all over the place. So it wasn’t to do with her intellect, it was actually the emotional coming to terms with it.

Participant 0, salaried GP (individual interview)

Although the main concept of a web-based self-management programme had good coherence with our participants, there was less agreement about providing some form of facilitation. Although some participants had experience of telephone facilitation for computerised CBT through the Improving Access to Psychological Therapies programme, others were unsure how such facilitation could be offered. Even those who understood the rationale for offering facilitation, in terms of integrating the self-management programme into clinical care and encouraging uptake and use by patients, were uncertain of the benefits:

One of the ways we thought that might help is . . . having someone maybe ring up every so often and say how are you getting on with the program?

I guess that’s possible. I don’t know if that would help, though.

Participant 14, diabetes specialist nurse (individual interview)

Similarly, there were uncertainties about what patient access to their EMR would entail, in terms of the sort of information patients could access, controls over privacy and the impact that this might have on patients and on HCP workload:

I’m not sure about that really . . . I don’t know whether they . . . should see the complete record or whether it, someone should go through it before they have a look, I’m not sure . . . I think if they say perhaps a summary or just maybe the key parts of the past medical history or drug history, which I think is important for a patient to have, then I think patients should have access to that generally, because they present in A&E [accident and emergency unit] or after hours; that is really handy information. So I think that would be a good thing generally. I don’t know how much detail a patient really needs, though, to read; maybe some individual consultations, I’m not quite sure if that would be very helpful.

Participant 11, trainee GP (individual interview)

Cognitive participation

Cognitive participation is about the decision whether or not to participate in an intervention. One of the components of cognitive participation is the concept of ‘legitimation’, or the extent to which the work required by the intervention is a legitimate part of the role of the various HCPs involved.

The HCPs in our study felt that our proposed programme was likely to be widely supported by professionals, who would see its advantages, including recognition of the importance and difficulties of enabling people with T2DM to self-manage and the benefits that accrue when they do self-manage. Ensuring that the programme was trustworthy, by containing up-to-date evidence-based information in formats that were accessible to patients and HCP, would also aid cognitive participation. Trust could be enhanced by the intervention being:

. . . professional, academic, something, you know, not commercially run.

Participant 6, GP partner (individual interview)

Endorsement by relevant bodies, including the Department of Health and Social Care, the Royal College of General Practitioners, the Royal College of Nursing, Diabetes UK and local diabetes consultants or other local opinion leaders would promote uptake and use of the programme by HCPs.

There was also a recognition that patient experience had considerable influence on clinician behaviour: if patients were calling for the intervention to be available, then this was likely to increase HCP buy-in. Similarly, if the programme was used by only a small minority, participants felt that this would lead to HCPs being much less interested in using and promoting it:

You probably want to get the Department of Health [and Social Care] to say that everyone should commission it as part of their care package.

Participant 9, salaried GP (focus group 2)

Getting good feedback from the patients, that’s going to make me recommend it.

Participant 16, practice nurse (focus group 4)

However, the overwhelming factor that would affect use and uptake was considered to be the impact on workload. Any intervention that increased workload was highly unlikely to be adopted:

If it is time-consuming, I think, [that] would be a big barrier, definitely, because as you know, consultations are really time pressured, aren’t they?

Participant 11, trainee GP (individual interview)

Conversely, if the intervention was found to decrease workload and result in more effective consultations, this was likely to promote adoption.

Hence, the prospect of practice staff facilitating access to the programme had low cognitive participation. The HCPs in our study felt that any potential benefits of facilitation were unlikely to justify the increased workload:

Do you think it’s viable to ask practice nurses maybe to ring up patients and say how you getting on with the intervention, are you using it . . .?

No. . . . We’ve got a crisis in practice nursing in London. Expecting . . . partners or clinicians to say ‘Yes, you can . . . my [practice] nurse can take half a day out to ring a whole lot of diabetics and encourage them to use this’, will probably not go down very well.

. . . Receptionists?

Maybe, but again I’d say this is another . . . activity unresourced. Until clinicians see the utility of it and see significant benefit from it for themselves and for their practices and for their patients, I can’t see clinicians spending a lot of resource and time.

Participant 5, GP partner (individual interview)

Just as the issue of patient access to their EMR had low coherence, it also had low cognitive participation. Even participants who could see its potential to empower patients had serious concerns about issues such as privacy and confidentiality, clinical governance and workload:

For this group of people, it’s really important. It’s their information, and they have control of it and understanding of it. So I like the idea, generally, of them owning the information. Pragmatically, I worry about people having dial-in access to our system, being able to hack in and around.

Participant 6, GP partner (individual interview)

Like all of these things, it’s probably fine for most people but there will be a few people who maybe have other problems like anxiety or whatever and then it actually . . . might just make things worse.

Participant 9, salaried GP (focus group 2)

Participants in WP A had discussed the possibility of being able to enter self-monitoring data directly into their clinical record, but this was considered entirely unacceptable by the HCPs in WP B for medicolegal reasons:

And what happens if the individual puts on that their blood sugar is 29 and they’ve got ketones? Am I meant to go and respond and take medical legal responsibility for that?

Participant 6, GP partner (individual interview)

Theoretical framework

The rationale for developing a web-based self-management programme for people with T2DM was described in Chapter 2, which also set out the theoretical framework underpinning its development, including:

1. Corbin and Strauss’ model54 of the work of living with a LTC, which we used to define the overall content of the programme

2. NPT,56 used to help us consider issues pertaining to implementation throughout the development process

3. a taxonomy of BCTs, which helped us select appropriate techniques within the programme.

We worked within the paradigm of evidence-based medicine, identifying and applying the best-available evidence for treatment of diabetes mellitus and for any decisions made during development (e.g. around maximising acceptability, uptake, usage and effectiveness).

Users

We identified our potential users as people with T2DM, and HCP involved in their care. The work undertaken to identify users’ ‘wants and needs’ was described in Chapters 4 and 5.

Participatory design

Participatory design is a broad term that covers a range of motivation, methods and levels of user involvement. There are two main rationales for adopting a process of participatory design. The first is pragmatic: interventions developed this way are more likely to be acceptable, have better uptake and be effective. 127 Working together in an iterative fashion, developers and users can learn together and optimise system functionality and service quality. Designers are responsible for pointing out technical options, while users provide information about their practices and how they will use the system. 128 The second rationale is a political or moral imperative: those who will be affected by health information technology (IT) systems should have the opportunity to influence their design and implementation. 129 From this perspective, design of health IT systems can be a way of empowering prospective users. 128 We were motivated by both rationales.

Intervention mapping

Intervention mapping is an explicit six-step process for developing complex interventions,87 which was originally developed in the public health context but since has been shown to be appropriate for a wide range of interventions,130–133 including those online. 134,135 It has been successfully integrated with participatory design136 and enables researchers to incorporate multiple sources of data into a single intervention in a transparent and reproducible fashion. 87 The six steps are:

1. needs assessment

2. detailed mapping of programme objectives and their behavioural and environmental determinants

3. selecting theory-based methods, techniques and strategies to modify the determinants of behaviour and the environment

4. producing intervention components and materials

5. planning for adoption, implementation and sustainability

6. creating evaluation plans and instruments.

Intervention mapping helped us think through the stages of development in an orderly fashion. For each stage, we considered how best to combine the principles generated by our theoretical frameworks, the data from our preliminary qualitative work and the input from our participatory design groups. The process of combining these different inputs was by frequent discussions among the core team followed by periods of drafting text and content, which were then discussed with the software engineers, to ensure appropriate functionality. This was an iterative process, with sections of the intervention being drafted, incorporated into appropriate functionality and reviewed by the participatory design panels. The panels also generated ideas for new content and functionality. Once the main outlines of the content and functionality were broadly agreed, we worked with the designer to ensure an aesthetic experience with intuitive navigation. These components are all described in more detail in Methods.

Recruitment of participatory design panels

As in our preliminary qualitative work (see Chapters 4 and 5), we conceptualised our users as HCPs caring for people with T2DM and people with T2DM. Thus, we recruited two complementary participatory design panels: one of HCPs and one of people with T2DM. Both groups were recruited by advertising, which was supplemented by word of mouth. We advertised for the HCP panel through personal contacts, by contacting participants in WP B and by using e-mail lists to contact local clinicians (e.g. the sessional GP contact list, the local locum contact list and the performers list of local CCGs). For the patient panel, we advertised in GP practices, when we had participant identification centre approval, community centres and Diabetes UK groups and sent direct invitations to participants in WP A. Respondents were sent details of what was required, a person specification and details of remuneration and invited to submit an application and attend an interview (for health professionals) or attend a recruitment day (for patients).

Recruitment of software engineers and website designers

Our previous experience had underlined the importance of selecting software engineers and web designers with whom it was possible to build a productive and constructive working relationship, including open, respectful communication, mutual responsiveness to issues arising during development and flexibility. We were keen to use an ‘agile’ rather than a ‘waterfall’ approach. Agile development relies on high levels of technical skills, a close working relationship between the software developers and the client, substantial user input and the use of simple designs to deliver working software to users at regular intervals. Agile developers accept and welcome changes to the brief that emerged during the development phase. 137 In addition, we required the software engineers and website designers to have a good understanding (and preferably experience of) the challenges of developing online tools for the use of patients in the NHS. Being mindful of issues of sustainability and long-term use, we were reluctant to be tied into bespoke software that could only be maintained or adapted by one company and, hence, had a preference for a company using open-source software. We approached the process of recruiting suitable partners in the following manner.

Determining and developing content

Using the Corbin and Strauss model of living with a LTC sensitised us to the importance of addressing tasks that are relating to emotional and role management, as well as those pertaining to medical management. We summarised the data on wants and needs from our qualitative work with people with T2DM and HCPs (Table 9).

We combined these data with the Corbin and Strauss framework to provide an overall list of required content (Table 10), which we mapped against the Corbin and Strauss framework and user data to ensure we had addressed all key areas.

Web design

We contracted a web designer to make the intervention attractive, user friendly and easy to navigate and to ensure that it satisfied the Royal National Institute for the Blind guidance for accessibility.

Specific objectives were to restyle the default template to improve attractiveness and readability, design an attractive and simple-to-use home page that provided easy access to all of the functionality of the website, ensure the menu structure and navigation tools were simple and easy to use and to create custom artwork (text boxes, images, animation) for relevant articles.

The web designer was also responsible for sourcing and creating images for the website. These were sourced from Shutterstock (Shutterstock Inc., New York, NY, USA) and Getty Images (Getty Images International, Seattle, WA, USA) or taken by the designer. The web designer adhered to the principles laid out in the style guide and also contributed to this style guide.

One technique used in HCI and web design to improve the usability of a digital intervention is the ‘user journey’. 158 This involves working with either real users or prototypical personae, identifying how they will use the proposed intervention and ensuring that they can find what they are looking for quickly and easily. Together with our web designer, we created seven personae who, between them, represented a diverse spectrum of patient characteristics (based on age, gender, ethnicity, duration of diabetes mellitus since diagnosis, health literacy, computer skills, motivation, diabetes treatment and social circumstances) to help illustrate the range of user needs. These user journeys helped guide the preliminary development of the navigation and site layout, which was then reviewed and refined by our participatory design panels.

Technical specifications and information governance

As set out under Methods, one important objective was to ensure that the programme met the appropriate information governance and technical requirements for implementation within the NHS. This governed many of our choices for the website software and platform.

The HeLP-Diabetes programme was created using the Joomla! (version 2.5, Open Source Matters, New York, NY, USA) CMS. The criteria for selecting a CMS included that:

1. the interface for the content management be intuitive so that the UCL team could be trained to use it

2. the creation of open-source content in the system was widely supported, to minimise risks – that is, development could be taken over by another software company if needed at the end of the contract or if the contract needed to be terminated early, which also supported the requirement for UCL to own the IP and code created as part of the project rather than simply owning a licence

3. there were high levels of security that would enable the final intervention to meet information security standard International Organization for Standardization (ISO) 270001, as required by the NHS and similar bodies

4. the CMS could be used to create attractive, function-rich and easy-to-use websites; a vibrant and stable development community that ensured the CMS would improve and evolve over time and be regularly updated and patched to deal with new security vulnerabilities or other technical anomalies.

The final intervention was hosted in an ISO 27001-certified environment within England and security audits were done, as required by the NHS Information Governance Toolkit. 159 The intervention was run on a LAMP (Linux, APACHE, MijoSQL and PHP) stack outside the NHS N3 national broadband network. Hosting the intervention within the N3 national broadband network environment would have significantly increased development time and costs, which would not have been feasible within our timelines and budget.

Content

The overall content was broken down into eight sections:

1. understanding diabetes (145 web pages; information about the nature and causes of diabetes mellitus and how it affects the body)

2. staying healthy (107 web pages; motivational material about how to maintain optimal physical and emotional health and the importance of self-management; new behaviour change modules and previously validated programmes for diet, weight loss, physical activity, smoking cessation, moderating alcohol intake and taking medicines)

3. treating diabetes (70 web pages; information about medications used in diabetes mellitus, including information about indications, side effects and monitoring; importance of managing cardiovascular risk factors as well as glycaemic levels; importance of regular monitoring to prevent retinopathy, neuropathy and nephropathy; and the types and roles of different HCPs in caring for people with diabetes mellitus)

4. living and working with diabetes mellitus (87 web pages; focus on managing social and work situations, such as shift work, parties or holidays; the impact diabetes mellitus has on relationships, including sexual relationships; and possible impact diabetes mellitus can have on emotions and feelings of self-worth)

5. managing my feelings (61 web pages; self-assessment tools for identifying low mood, CBT modules and mindfulness-based approaches)

6. my health record (45 web pages; opportunity to record appointments with HCPs and results of tests or self-monitoring, with opportunities for graphical displays and feedback)

7. news and research (16 web pages; updates about diabetes treatment, in-depth articles about seminal research papers; and information for HCPs, such as NICE guidelines)

8. forum and help (28 web pages; moderated forum; videos of personal stories about diabetes mellitus used with licence from healthtalk.org; and additional resources, including local resources tailored to the CCG).

Each section combined written information with videos and other graphics. There were interactive and tailored components in each section, apart from news and research. Behaviour change and emotional management sections included opportunities for self-assessment, in the form of validated questionnaires with automated feedback that contained recommendations for action. Users could set the programme to send them automated texts or e-mail reminders and alerts when new entries were posted in the forum. They could opt out of the regular engagement e-mails and newsletters described in Encouraging engagement.

More details of the website’s content, with screenshots, are provided in Appendix 1, with a site map in Appendix 2.

Registration and facilitation

Our grant proposal had emphasised that the intervention would be more than just the web-based programme. We conceptualised it as comprising interactions between people with T2DM and their HCP and the web programme. We thought that it was important that patients were introduced to the programme by a HCP, as we thought this would demonstrate that the programme was part of the overall care offered by the NHS and that their HCP felt that they would benefit from using the programme; HCP input could also help overcome the digital divide.

In our initial application, we described three ways in which HCPs could support use of the intervention: (1) initial registration and introduction to the programme; (2) follow-up phone calls to promote use and address any technical difficulties the user was experiencing; and (3) making reference to the programme within consultations, for example by reviewing progress against personal goals, discussing self-monitoring data or suggesting that a patient could benefit from a particular section.

Our qualitative work described in Chapters 4 and 5 showed that we needed to revise this model. Although people with T2DM welcomed HCP input, both they and HCPs felt that our suggestions were unrealistic. Thus, we refined our model for the trial (see Chapter 7), limiting the facilitation to registration and an initial demonstration of the site (undertaken in one 20-minute appointment). For the implementation study (see Chapter 9), we had to refine the model still further, reducing HCP input to a minimum with a 5-minute registration process; even this proved overly burdensome for some practices, leading to the development of a patient self-registration model.

Encouraging engagement

Further encouragement for people with T2DM to engage with the programme came through regular e-mail or (later) text message prompts. E-mails started in November 2013 and consisted of either a short two- or three-line text with a single message and link to the relevant section of the website; or a longer newsletter (of one A4 page), which summarised recent research or news, explained what this meant for people with T2DM and contained a paragraph or two on a specific health promotion or feature of the programme, all with relevant links.

In October 2014, we added text message prompts. These were very brief, with one clear message and a link to the programme.

Examples of the shorter e-mails included a seasonal reminder of the importance of influenza vaccinations for people with T2DM, with a link to the HeLP-Diabetes programme website explaining how and why they were beneficial and reminding them that GPs offered free influenza vaccinations. We sent around three prompts per month – usually two short e-mails or a text message plus one longer newsletter. Users could opt out of receiving prompts.

The procedure for writing newsletters is described above (under Maintenance and updating). The development of the shorter e-mails and text message followed a similar process, with the multidisciplinary team coming up with ideas that were then prioritised in discussion with PPI members. E-mails or text messages were then drafted by one team member and reviewed by the rest of the team and at least two PPI members.

We undertook some formative evaluation of the engagement e-mails, newsletters and text messages, exploring quantitative data on which e-mails or newsletters were opened and whether or not users who opened these subsequently visited the intervention website. These data suggested a small positive effect on engagement. We also undertook usability testing and qualitative interviews exploring which features of the e-mails or newsletters were perceived as interesting or attractive and what features would promote opening them and subsequently visiting the intervention.

These interviews showed that engagement prompts should be short, easy to understand, contain plenty of links to the main intervention, provide non-directive advice, be personalised and contain news and updates around topics of interest to users. The use of bullet points, pictures and bold colours encouraged users to read the prompts and follow the links to the intervention website. Examples of e-mails and newsletters sent can be found in Appendices 3 and 4.

Design

The study was a multicentre, two-arm individually randomised controlled trial in primary care.

Setting

The setting was general practices in England.

Participants

Participants were adults, aged ≥ 18 years, with T2DM, registered with participating general practices. In order to maximise the ability to generalise from the results of this pragmatic trial, exclusion criteria were kept to a minimum. We excluded people who were unable to provide informed consent, for example, because of psychosis, dementia or severe learning difficulties; terminally ill patients with < 12 months’ life expectancy; those unable to use a computer as a result of severe mental or physical impairment; those unable to use the intervention because of insufficient mastery of spoken or written English; and those who were currently participating in a trial of an alternative self-management programme. Participants did not need home internet access or prior experience of using the internet to take part. Those with previous or current experience of self-management education were eligible to participate.

Recruitment

Recruitment took place in two stages. First, we recruited practices through research networks, including the PCRN and the North Central London Research Consortium. Second, once a practice had agreed to participate and completed set up procedures, participant recruitment started.

Patient recruitment followed standard opt-in procedures. Each practice had a register of patients with T2DM, as required by the QOF. A nurse or other qualified health professional reviewed the EMR of each patient on this register to screen out those who were ineligible. All of those who were remaining were sent a letter from their GP inviting them to participate in the study. A participant information sheet, consent form, expression of interest and stamped addressed envelope were included. Patients who were interested in participating were asked to return the expression of interest form to the trial manager.

On receipt of expressions of interest, the trial manager contacted the practice research nurse who then offered the patient an appointment at the practice. This provided patients with an opportunity to discuss the pros and cons of participation and, if they wished to proceed, sign the consent form. Baseline clinical data were obtained either at this appointment or a subsequent one. After signing the consent form, patients were asked to complete the self-report baseline data and patients were randomised only once all baseline forms were completed. Randomisation marked the point of study entry.

Randomisation

Randomisation was performed centrally using a web-based randomisation system provided by Sealed Envelope^TM (London, UK; www.sealedenvelope.com). Randomisation was at the level of the individual participant and conducted using random permuted blocks of sizes 2, 4 and 6, stratified by recruitment centre. The practice nurse was informed which arm the participant had been randomised to so that those in the intervention arm could be offered the training appointment.

Intervention

The intervention consisted of facilitated and supported access to the HeLP-Diabetes programme. There were three components to the supported access: (1) an introductory training session, (2) supportive follow-up telephone calls and (3) ongoing discussion of patient’s self-management goals in routine appointments for diabetes mellitus-related matters.

In the training session, practice nurses registered the patient on either the intervention or the comparator website and gave the patient a booklet containing the unique resource locator (URL) for the programme, the participant’s log-in details and information about the content of the website and how best to use it (a copy of the booklet for the intervention site is available in Appendix 5; a similar booklet was provided for the comparator site). Nurses showed the patient how to access the website and introduced them to the main content areas. The nurse was asked to discuss with the patient what their most pressing needs were and use this to guide them towards certain sections, such as improving diet, being more physically active or managing emotions. Follow-up telephone calls were offered to support the patient in the use of the programme. Nurses and doctors in participating practices were asked to refer to the programme in consultations with participating patients and to integrate information from the programme into management plans.

The HeLP-Diabetes programme was a theoretically informed web-based programme for which the overall goals were to improve health outcomes and reduce diabetes mellitus-related distress. 165 Overall, content was guided by the Corbin and Strauss model of managing a LTC, which posits that patients must undertake medical, emotional and role management. 54 It was developed using participatory design principles, with substantial input from users, defined as people with T2DM and HCPs caring for such patients. All content was evidence based, drew on evidence on management of diabetes mellitus and promoted behaviour change and emotional well-being.

The HeLP-Diabetes website contained information sections on diabetes mellitus, how diabetes mellitus is treated, possible complications of diabetes mellitus, possible impacts that diabetes mellitus can have on relationships at home and at work, dealing with unusual situations such as parties, holidays, travelling or shift work, and what lifestyle modifications will improve health. Further sections addressed skills and behaviour change, including modules on eating healthily, losing weight, being more physically active, smoking cessation, moderating alcohol consumption, managing medicines, glycaemic control and blood pressure control. Users could set the programme to send themselves reminder text messages or e-mails and could specify the content and frequency of such reminders.

The third strand of components focused on emotional well-being, with self-help tools based on CBT and mindfulness. There were multiple personal stories (used with the licence from HTO, now known as healthtalk.org, www.healthtalk.org), and a moderated forum. Participants were free to use the programme as much or as little as they chose. Engagement was promoted through regular newsletters, e-mails and text messages containing updates on latest diabetes mellitus-related research or practice, seasonally relevant advice (e.g. on fasting during Ramadan, benefits of influenza vaccinations), and links to specific relevant parts of the programme. Two or three prompts were sent each month, although users could opt out of receiving them. For more information about the HeLP-Diabetes programme, e-mail and text messages prompts see Chapter 6.

Primary outcomes

The outcomes to be measured reflected our aims of improving clinical outcomes and HRQoL. We selected two primary outcomes: HbA_1c levels and diabetes mellitus-related distress, as measured by the PAID scale. 167,168 The PAID scale has 20 items focusing on areas that cause difficulty for people living with diabetes mellitus, including social situations, food, friends and family, diabetes mellitus treatment, relationships with HCPs and social support. It has been the subject of a number of reviews169–173 comparing available quality-of-life measures for diabetes mellitus. Eigenmann et al. 170 assessed available measures against criteria of reliability – content, face, construct, criterion and convergent validity; responsiveness to change; interpretability; response burden; acceptability and availability – and concluded that the PAID scale was one of three measures that met all criteria. It is sensitive to change and has been widely used to evaluate self-management programmes for people with T2DM, including in the influential DESMOND trial. 15

Secondary outcomes

Secondary outcomes were selected to reflect the proposed pathway of action of our intervention and allow for health economic analysis. They can be categorised as clinical, patient-reported or economic outcomes.

The clinical outcomes were:

• systolic blood pressure (SBP) and diastolic blood pressure (DBP)

• body mass index (BMI)

• total cholesterol and high-density lipoprotein cholesterol (HDL-C) (not fasting)

• completion of ‘nine essential processes’ (weight, blood pressure, smoking status, measurement of serum creatinine, cholesterol and HbA_1c levels, urinary albumin and assessment of eyes and feet) – data were obtained from notes at the 12-month follow-up point for data before randomisation (12 months prior) and after randomisation (12 months after).

The patient-reported outcomes were:

• depression and anxiety, as measured using HADS13

• diabetes mellitus-related self-efficacy measured using DMSES174

• satisfaction with treatment, measures using the status and change versions of the Diabetes Satisfaction with Treatment Questionnaire (DTSQ). 15

The economic outcomes were the EuroQol-5 Dimensions (EQ-5D) and health service use.

In addition, we used automated software to automatically record each participant’s use of the intervention (date and time of log ins and pages visited).

Data collection

Each practice provided information on the number of invitation letters sent out and the age and gender of the patients invited. Baseline data collection covered patient demographic, clinical and other descriptive data. Demographic data consisted of age, gender, highest educational attainment, ethnicity, current employment status, presence or absence of home internet access, level of expertise in computer use and current or previous participation in DSME.

Baseline clinical data obtained from the medical record included:

• the date of diagnosis of diabetes mellitus

• HbA_1c levels, blood pressure, total cholesterol levels, HDL-C and smoking status at time of diagnosis

• the presence or absence and date of diagnosis of complications of diabetes mellitus, including ischaemic heart disease, myocardial infarction, congestive cardiac failure, atrial fibrillation, peripheral vascular disease, amputation, cerebrovascular disease, retinopathy, renal failure and neuropathy

• a list of current medications.

Additional clinical data on height (cm), weight (kg), SBP and DBP, current smoking status and current levels of HbA_1c, total cholesterol and HDL-C were obtained during the baseline visit.

Baseline patient-reported outcomes were PAID score, HADS score, DTSQ score, DMSES score and the EuroQol-5 Dimensions, three-level version (EQ-5D-3L) score.

The collection of follow-up data was at 3 and 12 months, with 12 months as the primary outcome point. The window for 3-month data was 60–120 days post randomisation (90 ± 30 days) and for the 12-month data was 305–425 days (365 ± 60 days).

Health service use was recorded for the past 6 months at baseline, the past 3 months at the 3-month follow-up and the past 9 months at the 12-month follow-up. At the 12-month follow-up point, we extracted the data on completion of the nine essential processes for the 12 months before to randomisation and the 12 months after randomisation.

Standard operating procedures (SOPs) covered every aspect of data collection and nurses were trained in these procedures. Adherence to the SOPs was monitored. Participants completed self-reported questionnaires (demographics, the PAID scale, HADS, DMSES, DTSQ and EQ-5D-3L) online, prior to the nurse recording clinical outcomes and taking blood for HbA_1c levels and lipids to be tested for. The nurse entered all clinical data directly into the online database and extracted health service use data from clinical records.

Concealment of allocation and protection against bias

After baseline data had been obtained, randomisation was performed centrally. Allocation was not revealed to the participants, who were informed only that the trial would compare two forms of web-based education for diabetes mellitus; in order to maintain blinding of participants, they were not given details of the differences between the two websites. Practice nurses were provided with similar-looking booklets for the comparator and intervention websites. Potential contamination was monitored by recording participants with similar family names and identifying those with the same addresses. When this occurred, it was dealt with in the analysis by reporting the extent and undertaking a sensitivity analysis excluding these individuals.

The risk of bias in the collection of follow-up data was minimised by using standardised data collection instruments, with participants completing self-assessment questionnaires before seeing the nurse to record clinical data. Nurses collecting the clinical data were trained to adhere to detailed SOPs developed in collaboration with UCL’s PRIMENT Clinical Trials Unit. Blood pressure was recorded using automated electronic sphygmomanometers. At the 12-month follow-up point, we extracted the data on completion of the nine essential processes for the 12 months before to randomisation and the 12 months after randomisation to avoid triggering behaviour change among the study nurses. Estimations of HbA_1c and blood lipid levels were undertaken by the local hospital laboratory used by each participating practice.

Blinding of staff involved in data collection was maintained by having two nurses at each participating practice. One nurse (usually a practice nurse) was not blinded and was responsible for introducing participants in the intervention group to the intervention website. The other nurse (a different practice nurse or a specialised research nurse) was responsible for data collection at follow-up, with a blinded researcher collecting follow-up data by telephone from persistent non-responders.

Adherence and loss to follow-up

Fidelity of the intervention was promoted and monitored through automatic recording of the use of the intervention and comparator websites. Practice nurses were trained in the procedures for facilitating access and provided with laminated booklets to remind them of each stage of the process.

Every effort was made to promote follow-up, which was co-ordinated by the trial manager centrally. For the 12-month patient-reported data, an automated e-mail was sent to each participant from the trial website containing a hyperlink that took the participant directly through to a questionnaire on the data collection website, with data entered using this link being automatically connected to the participant identification number. This link remained active for 120 days from when first sent. Up to three automated reminders were sent at 15-day intervals if the participant had not completed the questionnaires. Fifteen days after the third automated reminder, persistent non-responders were sent a personal e-mail from the trial manager’s e-mail account. This e-mail also contained the hyperlink that would take participants through to the data collection website, linked to their participant identification number. After a further 7 days, non-responders were sent a letter through the post, referring to the link sent the previous week and also containing a paper copy of the primary outcome measure (the PAID scale), with a letter encouraging online completion of all the measures but, failing that, asking for completion of the paper version of the PAID scale, with a stamped addressed envelope for return to the trial manager. If there was no response after a further 10 days, the trial manager called the participant and obtained the PAID score over the telephone.

The procedures for the patient-reported data at 3 months were similar but less rigorous. Only two automated reminders were sent, followed by a personal e-mail and a letter through the post.

Practice or research nurses, blinded to allocation, were asked to collect the clinical data within the window period, and were reminded to do so by the central trial team.

When a participant withdrew from the trial before the final follow-up, the date of withdrawal and reason for withdrawal were recorded.

Sample size

We hypothesised that use of the intervention would improve both PAID scores and HbA_1c levels. The analyses gained power through adjustment for baseline levels, which accounts for the correlation between baseline and follow-up levels. We back-calculated the relevant effective SDs from a previous trial as SD 0.676% for HbA_1c level and SD 10.75 points for the PAID score,175 which were substantially lower than the SDs of cross-sectional measures of around 1.4% and 16 points, respectively, because of the correlation between baseline and subsequent measures. We intended to recruit 350 participants; with attrition of up to 15%, we expected to recruit at least 300 participants for the primary analysis, a sample size that would have given us 90% power to detect an average difference in the PAID score of 4.0 points and 90% power to detect a difference of 0.25% in HbA_1c level. These are both small effect sizes; therefore, the trial management group decided that, as HbA_1c levels and PAID scores are co-primary outcomes measuring very different aspects of the T2DM condition, both should be tested at a 5% significance level.

Analysis

The analysis followed a prespecified analysis plan, based on comparing the groups as randomised (intention to treat). The analysis plan was approved by the trial steering committee before unblinding and uploaded to the trial website. Only HbA_1c levels and PAID scores measured within a 10- to 14-month window following randomisation were used in the primary analysis. Missing 12-month outcomes were imputed using multiple imputation and baseline and other outcome data (e.g. 3-month data and final follow-up data collected outside the 10- to 14-month window). Descriptive statistics of the baseline and 3-month outcomes were summarised by randomised group, with a two-sample t-test used to test differences in mean outcomes at 3 months for those individuals in whom the outcome was measured.

A linear mixed-effects model with random centre effects was used to analyse each of the 12-month primary outcomes separately, adjusting for the baseline level of the outcome, age, gender, previous participation in self-management programmes, pre-existing cardiovascular disease and time since diagnosis of diabetes mellitus. Secondary outcome measures were analysed similarly using generalised linear mixed models. Effect modification analysis for the co-primary outcomes was undertaken by baseline glycaemic control (HbA_1c level outcome only), baseline PAID score (PAID outcome only) and duration of diabetes mellitus since diagnosis, treating all potential effect modifiers as continuous. The interaction between randomised group and each effect modifier was included in the model separately and assessed using a test of linear trend. The magnitude of effect modification was illustrated by splitting baseline HbA_1c level into two subgroups, < 7.5% and ≥ 7.5%, and by splitting baseline PAID score and the duration of diabetes mellitus since diagnosis above and below the median.

Missing data

Multiple imputation using chained equations was used as the primary method to account for missing data in both baseline and follow-up data. 176 A set of imputation models was specified, one for each variable with missing data. Each variable was then regressed on all other variables, including completely recorded baseline and follow-up variables, and stratified by randomised group. Imputations were performed using predictive mean matching, using the five nearest neighbours to the prediction as a set to draw from. The full list of variables considered in the multiple imputation using the chained equations approach is shown in Table 11, together with the number of missing values for each variable and time period.

As only measurements within a 10–14 month window were used within the main analyses of HbA_1c levels and PAID scores, the following imputation procedure was implemented for these two co-primary outcomes. The 12-month measurements were divided into those measured within 10–14 months (the primary outcome variable) and those that were measured outside 10–14 months (a variable used for imputing only). For HbA_1c level, two additional variables were created for use within the imputation model:

1. the time in days from randomisation that the 12-month HbA_1c level measurement was actually taken for values inside of the window (and set to 365 days for measurements taken outside the window period)

2. the time in days from randomisation that the 12-month HbA_1c level measurement was actually taken for values outside the window period (and set to missing for measurements taken within the window period).

The first of these variables gives the desired time for imputing HbA_1c level measurements when they are missing, whereas the second gives information on how far outside the window the actual measurements were taken. Corresponding variables were created for the 12-month PAID score measurement. Finally, two additional variables were created defining the time in days at which HbA_1c level and the PAID score were measured at 3 months. All variables were included within the chained equations and imputed when necessary.

Forty imputed data sets were created, the analysis models were fitted to each imputed data set separately and the estimates were pooled using Rubin’s rules.

Causal analyses

The intention-to-treat analysis was supplemented by a complier-average causal effect (CACE) analysis,176,177 in an attempt to investigate how the clinical effectiveness of the intervention was mediated through frequency of website usage. In particular, it was important to understand whether or not prolonged usage of the website modified the efficacy of the intervention. As website usage was measured post randomisation, a naive analysis of correlating usage with outcomes in the intervention group may give biased and misleading results, as there may be unmeasured confounders that are also correlated with the outcomes that distinguish the motivated users who regularly log in from the less motivated ones. Therefore, causal analyses using instrumental variables were used to determine the effect of website usage on outcomes. This approach preserves randomisation (i.e. provides a comparison independent of observed and unobserved confounders). ‘Usage’ was defined as the proportion of follow-up (rescaled as the number of days in a year) that the HeLP-Diabetes website was accessed. Usage of the intervention website in the control group was set to zero.

This main underlying assumption of the causal analysis is that the effect of randomisation to the HeLP-Diabetes intervention on the 12-month outcomes occurs only through use of the website (Figure 5, in which WU is website usage, Y is 12-month outcome, e.g. for HbA_1c level or the PAID score, and Z is the randomised intervention). This relies on the ‘exclusion restriction’ assumption that the HeLP-Diabetes intervention has no effect when usage is zero (i.e. for individuals who never log in). Hence, randomisation is assumed to be an instrumental variable.

FIGURE 5.

Causal model for CACE analysis. WU, website usage; Y, 12-month outcome (HbA_1c level or the PAID score); Z, the randomised intervention.

Causal model assuming no direct effect of intervention (complete mediation through website usage). In this causal model, randomisation (Z) is the instrumental variable, which acts on the outcome (Y) exclusively through website usage (WU).

Another assumption in the CACE analysis was that the control website was unlikely to be clinically effective and, therefore, usage in the control group was ignored in the model. A structural mean model was fitted using the observed level of compliance and treating randomisation as an instrument (i.e. assuming it is independent of both observed and unobserved confounders and only affects the outcomes through its effect on website usage).

Potential contamination was monitored by recording participants with similar family names and identifying those with the same addresses. When this occurred, it was dealt with in the analysis by reporting the extent and undertaking a sensitivity analysis excluding these individuals.

A number of other sensitivity analyses were performed to assess the robustness of the primary analyses: (1) performing two complete-case analyses disregarding outcomes measured outside 10–14 months and 11–13 months post randomisation; (2) repeating the analysis using multiple imputation of baseline covariates only and (3) fitting linear models excluding centre random effects.

Health economic outcomes

The components of the analysis were the costs of the active and control intervention, health-care and social services use and health outcomes, including a diabetes mellitus-specific outcome (diabetes mellitus-related distress) and a generic quality-of-life outcome (EQ-5D-3L).

Analysis

The analysis followed a prespecified plan, based on comparing the groups as randomised (intention to treat). The analysis plan was approved by the trial steering committee before unblinding and was uploaded to the trial website [see the project web page: www.journalslibrary.nihr.ac.uk/programmes/pgfar/RP-PG-0609-10135/#/ (accessed August 2018)]. Only PAID scale scores and EQ-5D-3L data obtained within a 10–14 month period following randomisation were used in the primary analysis. Missing 12-month outcomes were imputed using multiple imputation and baseline and other outcome data (e.g. 3-month data and final follow-up data collected outside the 10- to 14-month window). The EQ-5D-3L index score was presented and then converted to QALYs to enable a cost–utility analysis, as per NICE’s guidance. 184 The QALYs were calculated over the duration of the trial using the area under the curve from the baseline and 3- and 12-month follow-ups. 195

A linear mixed-effects model was fitted with the 12-month outcome as the dependent variable, adjusting for the baseline variables of age, sex, presence of pre-existing cardiovascular disease, duration of diabetes mellitus since diagnosis and smoking status, as well as the corresponding baseline outcome (costs, PAID score and EQ-5D-3L index score, respectively). Centre effects were included as random effects in the analysis. The differences in mean 12-month costs and outcomes were estimated based on the model. As the HeLP-Diabetes programme was dominant over the comparator (being less costly and more effective), the incremental cost per point of PAID score improvement and incremental cost per QALY gained were not calculated.

The non-parametric bootstrap technique was employed to explore the uncertainty of calculated difference in mean 12-month costs and outcomes from multiple-imputed data sets. In total, 5000 bootstrapped data sets were created and the total costs and outcome were estimated for each. The results from bootstrap resampling were used to construct 95% CIs for incremental costs, incremental PAID score and incremental QALYs and to plot the cost-effectiveness plane and cost-effectiveness acceptability curve (CEAC) to show the uncertainty surrounding the conclusion. 196

Sensitivity analyses

Two sensitivity analyses were undertaken: first, a complete-case analysis to assess the effect of imputing missing data; and, second, a one-way sensitivity analysis examining the uncertainty around the cost of the intervention.

The one-way sensitivity analysis was undertaken to explore the impact that increasing the costs associated with the intervention, (e.g. increasing the amount of facilitation, which would lead to additional cost per user) or by reducing the number of users (which would lead to an increased cost per user, as the costs of delivery and maintenance were largely fixed) would have.

Assuming that the effect on health resources and HRQoL remained the same once the participant took up the intervention, the additional intervention cost per person had a linear relationship with the incremental cost-effectiveness ratio (ICER):

A corresponding ICER was not calculated for the incremental cost-per-unit improvement of PAID score, as there is no national willingness-to-pay (WTP) threshold to compare it with.

Costs

Primary outcomes

The outcomes and the incremental costs and effectiveness based on imputed data sets are presented in Tables 27–30. The mean costs of health resources used in months 1–3 were higher in the intervention group than in the control group (£580 vs. £544) (see Table 27). In months 4–12, this position reversed, with the mean costs of health resources used being higher in the control group than in the intervention group (£1231 vs. £1525). Overall, during the 12-month trial period, the costs of health resources used were higher in the control group than in the intervention group. The unadjusted difference in total cost (including intervention cost) was £210, with a lower value in the intervention group. The difference was smaller (£91) when adjusted for baseline health resource costs only, but still higher in the control group. After adjusting for the costs in the 12 months prior to recruitment and baseline variables, the incremental mean cost for the intervention group compared with the control group was –£111.

The mean PAID score was higher in the control group than in the intervention group at baseline and at 3 months and 12 months (Table 28). The unadjusted difference in PAID score at 12 months was 3.1 points, with the intervention group scored lower. This difference was reduced to 1.9 points after adjusting for baseline PAID score, which was similar to the score when the adjustment for other baseline variables was also made.

The mean EQ-5D-3L index score in both groups increased from baseline to 3 months (see Table 29). At 12 months, the mean index score dropped in both groups, but the reduction was sharper in the control group. QALYs were calculated based on the EQ-5D-3L index score at three time points. The unadjusted value of QALY during the 12 months trial period was 0.802 in the intervention group and 0.764 in the control group, giving a difference of 0.038. The difference was reduced by half after adjusting for baseline EQ-5D-3L index score. After adjusting for EQ-5D-3L index score at baseline and other baseline variables, the incremental QALY was 0.020, comparing the intervention group with the control group.

For both health outcomes, the intervention was less costly and more effective, suggesting dominance by the intervention over the control (Table 30). However, the 95% CIs of all three outcomes included zero, indicating the uncertainty of this conclusion.

Figure 12 shows the cost-effectiveness plane describing the uncertainty surrounding the conclusion of the intervention’s dominance. The dashed lines show the national WTP thresholds of £20,000 and £30,000 per QALY gained. As the WTP threshold increases, the slope of the line will increase. There is considerable uncertainty surrounding the point estimates of the incremental costs and incremental QALYs from the primary analysis, but the majority of the distribution is scattered in the south-east quadrant of the plane, supporting the primary conclusion of the intervention being less costly but more effective. When the distribution falls into the north-east quadrant of the plane (more costly and more effective), most of the data points fall below the WTP thresholds. This indicates that even if the intervention becomes more costly instead of cost-saving, it is still likely to be cost-effective.

FIGURE 12.

The cost-effectiveness plane comparing the intervention group with the control group.

The CEAC constructed using the bootstrapped data shows that the probability of the intervention being cost-effective reaches 95% at a WTP of £6900 (Figure 13). Under the existing WTP threshold of £20,000 to £30,000, the probability of the intervention being cost-effective, compared with the control, was over 97%.

FIGURE 13.

The CEAC for the intervention.

Sensitivity analysis

Complete-case analysis

For the three outcomes used in the primary analysis, the extent of completeness varied (Table 31). The completeness of PAID score at 12 months was the highest among the three. For costs, only the participants who had complete cost data (i.e. heath services use recorded by nurses, health services use reported by participants and medication information) at 3 months and 12 months were considered complete cases, which gave the lowest number of complete cases. In order to calculate QALYs, the EQ-5D-3L must be completed at all three data collecting points to be considered complete.

TABLE 31 - Number of complete cases for costs, PAID score and QALY, and their mean values (SD), by group

Outcomes	Treatment group
	Intervention		Control
	n	Mean (SD)	n	Mean (SD)
Total costs (£) during 12-month trial period	98	1785 (1511)	103	1719 (1525)
PAID score (points) at 12 months	153	14.1 (14.8)	168	16.5 (18.0)
QALYs	126	0.822 (0.185)	128	0.808 (0.242)

Taking into account the completeness of the baseline variables required for adjustment, there were 96 participants in the intervention group and 101 participants in the control group who had complete information at all required time points on every required variable. Therefore, the complete-case analysis was performed on these 197 people.

In the complete-case analysis, the mean cost during the 12-month trial period, including intervention and health resource use costs, was £1736 in the intervention group and £1721 in the control group (Table 32). Using the same adjustment methods as in the primary analysis, the intervention group showed a £15 higher mean cost when unadjusted, a £58 lower mean cost when adjusted for baseline health resource use costs and a £37 lower mean cost than the control group after adjusting for all baseline variables.

TABLE 32 - Costs estimated from complete cases, by group

n/a, not applicable.

Calculation based on assumption that medications were for chronic use and prescribed monthly.

Baseline variables included age, sex, history of cardiovascular diseases, smoking status and time since diabetes mellitus diagnosis.

Costs	Treatment group, cost (£)
	Intervention (n = 96)	Control (n = 101)
Intervention cost	41	0
Time point, months, mean cost (SD)
1–3
Health services use (from nurses)	332 (503)	267 (327)
Health services use (from participants)	24 (64)	27 (68)
Medication for 1-month period	57 (66)	52 (55)
4–12
Health services use (from nurses)	566 (692)	693 (963)
Health services use (from participants)	51 (143)	65 (186)
Medication for 1-month period	61 (73)	57 (69)
Health resources use in the 12-month trial period,a mean (SD)	1695 (1404)	1721 (1539)
Total costs in the 12-month trial period,a mean (SD)	1736 (1404)	1721 (1539)
Unadjusted incremental cost for the intervention group	15	n/a
Incremental cost for the intervention group, adjusted for health resources use cost at baseline only, mean (SD)	–58 (163)	n/a
Incremental cost for the intervention group, adjusted for health resources use cost at baseline and other baseline variablesb (95% CI)	–37 (–367 to 282)	n/a

The mean PAID score at 12 months was 14.6 points in the intervention group and 15.9 points in the control group (Table 33). Although PAID scores continued to reduce in the intervention group, the scores remained the same between 3 and 12 months in the control group. The adjusted difference between groups was 1.6 points at 12 months, with a lower value in the intervention group.

TABLE 33 - The PAID score estimated from complete cases, by group

n/a, not applicable.

Other baseline variables included age, sex, history of cardiovascular diseases, smoking status and time since diabetes mellitus diagnosis.

Time point	Treatment group
	Intervention (n = 96)	Control (n = 101)
Time point, mean PAID score (points) (SD)
Baseline	18.8 (16.8)	19.0 (16.5)
3 months	16.9 (16.1)	15.9 (15.2)
12 months	14.6 (15.5)	15.9 (16.7)
Unadjusted incremental PAID score (points) at 12 months for the intervention group	–1.3	n/a
Incremental PAID score (points) at 12 months for the intervention group, adjusted for baseline PAID score only, mean (SD)	–1.2 (1.6)	n/a
Incremental PAID score (points) at 12 months for the intervention group, adjusted for baseline PAID score and other baseline variablesa (95% CI)	–1.6 (–5.1 to 1.4)	n/a

The mean QALYs during the 12-month period were 0.816 in the intervention group and 0.833 in the control group (Table 34). The unadjusted difference shows a higher QALY in the control group than in the intervention group. After adjusting for baseline variables, the mean QALYs in the intervention group were 0.010 higher than in the control group.

TABLE 34 - The EQ-5D-3L and QALYs estimated from complete cases, by group

n/a, not applicable.

Other baseline variables included age, sex, history of cardiovascular diseases, smoking status and time since diabetes mellitus diagnosis.

EQ-5D-3L and QALYs	Treatment group
	Intervention (n = 96)	Control (n = 101)
EQ-5D-3L index, mean (SD)
Baseline	0.792 (0.232)	0.829 (0.207)
3 months	0.824 (0.186)	0.840 (0.229)
12 months	0.814 (0.218)	0.825 (0.250)
QALYs
QALYs in the 12 months’ trial period, mean (SD)	0.816 (0.183)	0.833 (0.215)
Unadjusted incremental QALYs for the intervention group	–0.017	n/a
Incremental QALYs for the intervention group, adjusted for baseline EQ-5D-3L index score only, mean (SD)	0.026 (0.015)	n/a
Incremental QALYs for the intervention group, adjusted for baseline EQ-5D-3L index score and other baseline variablesa (95% CI)	0.010 (–0.018 to 0.044)	n/a

Combining all three outcome measures, among complete cases, the intervention group still showed a lower mean cost and better effectiveness on both PAID score and QALYs (Table 35). Compared with the results from the primary analysis (see Table 30), the complete-case analysis results generally showed a larger SD but a smaller difference between groups.

TABLE 35 - Incremental cost-effectiveness estimated from complete cases

Baseline variables included age, sex, history of cardiovascular diseases, smoking status and time since diabetes mellitus diagnosis.

Incremental cost-effectiveness	Mean (95% CI)
Incremental cost for the intervention group, adjusted for health resources use cost at baseline and other baseline variablesa (95% CI), (£)	–37 (–367 to 282)
Incremental PAID score at 12 months for the intervention group, adjusted for baseline PAID score and other baseline variablesa (95% CI)	–1.6 (–5.1 to 1.4)
Incremental QALYs for the intervention group, adjusted for baseline EQ-5D-3L index score and other baseline variablesa (95% CI)	0.010 (–0.018 to 0.044)
ΔCost/ΔPAID score	Intervention dominating control, as it is less costly but more effective
ΔCost/ΔQALY	Intervention dominating control, as it is less costly but more effective

Figures 14 and 15 present the cost-effectiveness plane and CEAC plotted using the same technique as in the primary analysis but using data only from the 197 complete cases. From the distribution of the dots on the plane (see Figure 14), it is evident that only about a quarter of them fall in the south-east quadrant, where the intervention would be less costly but more effective. A considerable proportion of dots fall in the left side of the plane, where the intervention would be less effective.

FIGURE 14.

Cost-effectiveness plane comparing the intervention group to the control group (among complete cases).

FIGURE 15.

The CEAC for the intervention (among complete cases).

Accordingly, it is not surprising that the probability of the intervention being cost-effective was much lower than estimated from imputed data. In contrast to the high probability (> 97%) of cost-effectiveness in the primary analysis, the complete-case analysis showed a probability of under 70% of the intervention being cost-effective. Between £20,000 and £30,000, the probability only changed 0.2% (from 68.7% to 68.9%). This trend is consistent with the primary analysis. Marginal effects reach near zero quickly. Additional investment in the intervention did not appear to have a big impact.

Comparing the results from the complete-case analysis to the primary analysis, neither showed any apparent change in health resource use costs within groups (Figure 16). The cost of health resources used in the 12 months prior to recruitment in the control group was lower among complete cases than that in the imputed data set, although there was little difference between the two data sets in the intervention group. Given that the baseline data were almost complete, this difference in the control group suggested that the participants who had higher usage of health services were more likely to be lost to follow-up than those with lower usage. The mean costs during the 12-month trial period in both groups were lower in the complete-case data set than in the imputed data set, with the difference in the control group more prominent. This was because the multiple imputation took the higher health resource use cost in the 12 months prior to recruitment as one of the factors for its imputation of missing values. That, in turn, leads to a bigger difference in costs in the imputed data set than in the complete cases (–£111 vs. –£37).

FIGURE 16.

Comparison of the costs of health resources use between imputed data and complete cases.

As the imputation was performed on an aggregated level, the individual contribution of the cost of each type of health resource used during the 12-month trial period was not presented in the primary analysis. For complete cases, the biggest drivers in costs were hospital inpatient stays and hospital outpatient appointments (Figure 17), followed by GP consultations in surgery. However, if all forms of GP consultation were aggregated, the costs would approximate to those of hospital outpatient appointments and inpatient stays. It should be noted that Figure 17 does not include medication costs. If our assumption of one prescription item per month was correct, medication costs would be higher than all other costs (£718 in the intervention group and £671 in the control group).

FIGURE 17.

Cost of each health care and social services use during the 12-month trial period among complete cases, by group.

Complete cases showed a similar pattern in change of PAID score in the control group across three time points as in the imputed data set but with lower scores (i.e. less distress) (Figure 18). Meanwhile, the mean PAID score was slightly higher at baseline and at 3 months among complete cases than in the imputed data set, but at 12 months was similar in the complete cases and the imputed data set in the intervention group. This suggested that there were possible opposite missing patterns in the groups. In the intervention group, participants who scored lower on PAID score at baseline were more likely to be lost to follow-up; however, in the control group, those who scored higher were more likely to be lost to follow-up. The difference of PAID score at 12 months was much more prominent in the control group than in the intervention group. After adjusting for baseline PAID score and other variables, this leads to a smaller reduction in PAID score (–1.6 vs. –1.9 points).

FIGURE 18.

Mean PAID score (points) of imputed data set and complete cases at three time points, by group.

In the complete-case analysis, the general pattern of the mean EQ-5D-3L index score increasing at 3 months and decreasing at 12 months remained, but the mean index scores in the intervention group were higher in the complete cases than in the imputed data set and did not reduce at 12 months as much as in the imputed data set (Figure 19). In the control group, the baseline level of EQ-5D-3L index score was much higher among complete cases than in the imputed data set and remained so at all time points. The mean EQ-5D-3L index scores in the control group increased at 3 months and reduced below the baseline level in both complete cases and imputed data set. The drop of EQ-5D-3L index score level was more gradual and to a lesser extent among the complete cases. Among complete cases, the mean EQ-5D-3L index score in the control group was higher than in the intervention at all time points, opposite to the situation for the imputed data set. This leads to a higher QALYs in the control group when unadjusted. This pattern of change is consistent with the pattern in PAID score where in the control group, participants who had lower EQ-5D-3L index score at baseline were more likely to be lost to follow-up later on. Because the baseline EQ-5D-3L was near complete and taken as a factor in the imputation model, the imputed data set estimated lower values for missing data at 3 and 12 months. After adjusting for all relevant baseline variables, the intervention group still shows a QALYs gain, compared with the control group. This QALYs gain was also smaller than that of the imputed data set (0.010 versus 0.020).

FIGURE 19.

Mean EQ-5D-3L index score of imputed data set and complete cases at three time points, by group.

One-way sensitivity analysis for cost of intervention

The intervention cost was estimated at £41 per person for the primary analysis. Using Equation 1, a linear line was drawn, assuming incremental cost per person remained –£111 and incremental QALYs per person remained 0.020 (Figure 20). As the additional intervention cost per person increased, the ICER for the intervention gradually increased. At a £20,000 WTP threshold, the additional intervention cost per person was £511. At a £30,000 WTP threshold, the additional intervention cost per person was £711. That means, on top of what was included in the primary analysis for the intervention, another £511–711 per person could be spent on facilitating activities or running costs of the intervention, and the intervention might still be considered cost-effective.

FIGURE 20.

One-way sensitivity analysis for intervention cost per person, from imputed data set and complete cases. CCA, complete-case analysis; MI, multiple imputation; WTP1, willingness to pay threshold 1 (£20,000); WTP2, willingness to pay threshold 2 (£30,000).

In the analysis, the website operation cost was allocated to 10,000 users, as this was the reported capacity of the server. When it was allocated to the 185 participants in the intervention group, the intervention cost would increase by £222. The ICER, in turn, would be £5550 per QALY gained. Although not less costly any more, the intervention could still be considered cost-effective under the national WTP thresholds, in comparison to the control.

The complete-case analysis demonstrated a more rapid increase in incremental cost per QALY gained than the analysis using imputed data. An increase of £237 in intervention cost per user led to the ICER reaching the lower bound of the current national WTP threshold (£20,000 per QALY). Another £100 additional intervention cost per user led the ICER up to the upper bound of current national WTP threshold (£30,000 per QALY). In other words, in the complete-case analysis, cost-effectiveness was more sensitive to increased costs of the intervention.

Main results

In this within-trial health economic analysis, from a NHS and personal and social services perspective, the HeLP-Diabetes programme was found to dominate the comparator, that is, total costs were lower (–£111, 95% CI –£384 to £136) and health outcomes better (incremental QALYs 0.020, 95% CI 0.000 to 0.044 QALYs) in the group allocated to the HeLP-Diabetes programme than in the group allocated to the comparator. The cost-effectiveness plane and CEAC illustrated that, despite considerable uncertainty around the point estimates, this overall conclusion is likely to be robust. The complete-case analysis demonstrates more uncertainty than the primary analysis, which is based on imputed data. Although the total costs were still lower (–£37, 95% CI –£367 to –£282) and health outcomes better (incremental QALYs 0.010, 95% CI –0.018 to 0.044 QALYs) in the HeLP-Diabetes programme group than in the comparator group, the extent of the difference was smaller than in the primary analysis. Consequently, the cost-effectiveness plane showed the cluster of the distribution moving towards centre and widely scattered in all four quadrants. The probability of cost-effectiveness was reduced from 97% in the primary analysis to just under 70% in the complete-case analysis.

This difference in results between the primary and complete-case analyses may be because of the differential retention rates of participants with worse health. People with worse health were more likely to use health-care services, which resulted in higher costs. The comparison of costs and quality of life was consistent with this suggestion, as participants in the complete-case analysis demonstrated better quality of life and lower health resource use than those in the primary analysis. Whether it was the impact of worse health or the burden of completing trial outcome questionnaires on top of their ill health that caused this differential dropout is unclear.

Although both the primary and complete-case analyses suggested that the HeLP-Diabetes programme was highly likely to be dominant over the comparator condition, the uncertainties surrounding both the change in costs and in quality of life, and the fact that these changes were not statistically significant, require this conclusion to be treated with caution. Moreover, as the trial lasted only 12 months, it is possible that some of the drivers of health resource use by participants pre-dated the trial, for example hospital inpatient admissions may have been planned before the start of the trial. Moreover, although it was apparent that the main cost drivers were medications, hospital inpatient stays, hospital outpatient appointments and GP consultations, it was unclear which one in particular or combination of services caused the overall health-care costs to be lower in the intervention group.

As a large proportion of the costs are fixed costs, the average user cost falls and the cost-effectiveness increases as the number of registered users grows. This is particularly important in a highly prevalent condition such as T2DM. Current estimates suggest there are around 4 million people in the UK with T2DM, making the need to identify a cost-effective method of improving health outcomes in these people imperative.

Methodological strengths and weaknesses

The analysis was undertaken following NICE’s guidance. We collected detailed data on health and social service usage. The majority of service use data reported by nurses was extracted from participants’ EMRs, which was a more accurate and reliable method than participants’ self-reported data. Although there may have been under-recording, it is unlikely to have introduced systematic bias. Data on service use obtained directly from participants may have been subject to recall bias and be less reliable but, again, it is hard to see how this could have introduced systematic bias. Prescribed medications were extracted by nurses from participants’ EMRs and, hence, were more likely to be accurate than patient recall. However, it would have been better if we had asked nurses to record defined daily doses, as without these data we relied on the assumption that each prescription item was issued per month (as per national guidelines). Although this added to the uncertainty around total medication costs, it too was unlikely to lead to systematic bias, as it was applied to both groups.

We took the methodological decision to exclude the costs of development of the HeLP-Diabetes programme and include only the costs of maintenance and delivery. The rationale for this decision was that it is these maintenance and delivery costs that the NHS will have to cover if the HeLP-Diabetes programme is implemented nationally and the development costs are true sunk costs, in that they cannot be recovered. However, we have presented estimates of the development costs, for those interested in developing similar interventions for other LTCs.

Owing to the length of the trial (12 months), any future updates were not taken into account in the analysis. With the fast pace of development of technology, it is not certain how frequently major updates would be required or what they would involve or cost.

Our original intention had been to undertake a modelling exercise, using the UK Prospective Diabetes Study model, which simulates the likely occurrence of major complications and costs and quality of life associated with them by four prognostic risk factors: HbA_1c level, SBP, the ratio of total cholesterol to HDL-C and smoking status. 198 The main trial analysis reported a significantly better outcome in HbA_1c level and SBP in the intervention group than in the control group and no significant difference in the total cholesterol-to-HDL-C ratio or smoking status. Hence, any modelling would be likely to demonstrate reduced complication rates in the intervention group, with associated improved quality of life and reduced health-care costs. We did not wish to overstate our findings and, as modelling would increase the overall uncertainty, we decided not to undertake this modelling.

Fit with existing literature

One of the main drivers for developing digital health interventions is the expectation that they can help health-care systems meet the twin challenges of delivering more and better care at reduced cost. Despite this, there has been a marked lack of formal health economic analyses. Authors of systematic reviews of digital health interventions have repeatedly reported a lack of economic data, preventing them from drawing any conclusions about cost-effectiveness. 29,30,37,199–201 Fortunately, this situation is now improving, particularly in the field of mental health, in which a number of cost-effectiveness studies have been undertaken, enabling a systematic review published in 2015202 to include 16 studies and report the cost-effectiveness of guided internet interventions for depression, anxiety, smoking cessation and alcohol moderation. Recently, there have been a number of reports of health economic analysis of digital health interventions;203–211 overall, these reports confirm that, when effective, digital health interventions, such as the HeLP-Diabetes programme, tend to be cost-effective. However, the current report is the first in which we have been able to identify that a digital health intervention dominates the comparator.

Conclusions

The HeLP-Diabetes programme appears to be a highly cost-effective intervention, which, if adopted nationally, has the potential to be even more cost-effective.

Rationale for undertaking an implementation study

Implementation research is a relatively new field, concerned with ‘the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice’. 212 Our reasons for including an implementation study alongside a trial in this programme grant were threefold:

1. Whether or not the HeLP-Diabetes programme itself proved clinically effective in a trial, the policy and financial imperatives for introducing eHealth interventions to help patients improve their health are overwhelming; hence, producing generalisable knowledge about how best to implement such interventions would be useful for the NHS and other health-care systems internationally.

2. Evidence of clinical effectiveness that is derived from RCTs does not always transfer to ‘real-world’ use. By their nature, trials involve tightly controlled procedures and selected participants whose characteristics may differ substantially from non-trial participants. Trial methodology emphasises the importance of fidelity to the intervention and trials of complex interventions will often devote considerable resources to ensuring that the intervention is delivered as intended. However, once the intervention becomes approved for routine use, these resources are often not available and the emphasis often changes from ensuring that there is fidelity to enabling adaptions that will improve local uptake and use. These changes in user population and implementation mean the effects seen in trials may not be replicated in routine use. Therefore, the effects obtained in routine use should be viewed alongside those seen in trials to inform commissioning decisions.

3. One of the overarching aims of the NIHR’s Programme Grants for Applied Health Research programme is for funded work to achieve benefits for the NHS. Hence, it was important to ensure that the programme of research generated all the data required to inform future implementation and use of the HeLP-Diabetes programme throughout the NHS. As part of the planning for this programme grant, we thought about long-term sustainability and maintenance. It was clear that this would require a revenue stream, with NHS commissioning being the most likely source. Commissioners need more information about a service than can be provided from trial data, including information about likely uptake and use and the resources required for effective implementation. This study was designed to address these needs, as well as providing generalisable data of international relevance.

The importance of undertaking implementation research alongside trials is increasingly recognised. 213–215 Broadly speaking, such research addresses two types of question: first, what are the effects of a treatment or intervention in routine use (referred to as Phase IV studies in the MRC framework); and, second, what are the effects of an implementation strategy? Researchers from the US Department of Veterans Affairs (VA) Quality Enhancement Research Initiative (QUERI) have argued for ‘hybrid’ designs that combine both types of research question. 216 They have suggested a taxonomy for such ‘effectiveness-implementation hybrid designs’ in which type 1 hybrid design tests the effect of a clinical intervention on relevant outcomes while observing and gathering information on implementation, type 2 hybrid design tests clinical and implementation interventions and strategies equally and type 3 hybrid design tests an implementation strategy while observing and gathering information on the impact that the clinical intervention has on relevant outcomes. 216 In line with our rationale for undertaking an implementation study, we initially designed the study as a type 3 hybrid design using this taxonomy.

Theoretical underpinning for design of implementation plan

The NPT (see Chapter 2) was used during the development of the HeLP-Diabetes programme to help ensure that the intervention was ‘implementable’ and would fit well with current NHS practice. The NPT emphasises that interventions are more likely to be normalised into routine practice if they are easily understood and described (coherence), target users want to adopt them (cognitive participation) and they fit well with existing workflows (collective action). 56 Areas of particular importance are (1) the impact that the intervention had on consultations between HCPs and patients (interactional workability); (2) the impact the intervention had on relationships between professional groups, including ensuring that accountability and responsibility are aligned (relational integration); (3) ensuring that there was a fit with existing skill sets and role responsibilities (skill set workability); and (4) ensuring that there was a fit with organisational priorities and available infrastructure (contextual integration). Finally, users will engage in a process of reflexive monitoring, in which they consider whether or not the benefits of the intervention are worth the work involved, along with any adaptations or improvements that can be made. This reflection can be based on formal or informal data. These constructs provided an overall framework for us during the development of the implementation plan.

However, although the NPT can be used to sensitise researchers and implementers to challenges and issues that are likely to arise during an implementation process,217 it does not provide a step-by-step guide to developing an implementation plan. Therefore, we augmented our theoretical framework with a model developed by Grol and Wensing218 to assist with the implementation of change in health care. This model provides a step-by-step guide to developing an implementation plan, starting with defining the operational-change objectives and analysing current practice, the target group and the context of the proposed implementation. This then facilitates selection of appropriate implementation strategies, rather than relying on frequently used but potentially inappropriate strategies, such as a generic implementation plan. Crucially, the evaluation of the implementation is included, with a strong emphasis on this happening in parallel with the planned implementation activities, allowing early identification of areas needing attention or strategies that are not working. The importance of an iterative approach whereby implementation and evaluation of the implementation occur in parallel, feeding into each other, has also been stressed by the VA QUERI team. 219,220

Context for the implementation

There are theoretical and empirical grounds for recognising the importance of context in any given implementation. 221–224 In the Consolidated Framework for Implementation Research, Damschroder et al. 221 name two aspects of context: the outer and the inner settings. The outer setting consists of factors such as external (e.g. national) policy and incentives, peer pressure from other organisations and organisational priorities, while the inner setting includes structural characteristics such as the availability of infrastructure, leadership, culture, communication and local or internal incentives and reward structures.

At the time this study commenced (March 2013), the outer setting was dominated by the Health and Social Care Act 2012,70 which was described as the ‘biggest single reorganisation’71 and the ‘longest and most complex piece of legislation’72 in the history of the NHS. Among the many components of the Act were the abolition of PCTs and strategic health authorities, which had been responsible for commissioning services. PCTs were replaced by CCGs, which were intended to control around 60% of the NHS budget, be led by GPs supported by other clinicians and managers and were tasked with meeting the needs of their populations.

This reorganisation resulted in huge workloads for those involved as they struggled to come to terms with new priorities and responsibilities, evolving structures and changes in personnel. This was often accompanied by uncertainties about roles and responsibilities, loss of existing staff with relevant expertise and loss of organisational memory. 74,75

Furthermore, it occurred as the NHS entered a period of significant financial austerity. After a period of year-on-year budget growth, the NHS was charged with making financial savings of £20B over 5 years from 2011–15, while maintaining (or improving) the quality of the service. 76 This required efficiency savings of around 4% per year, compared with maximum previous efficiency savings of around 2% per year. It was recognised that meeting this ‘unprecedented challenge’ would require new ways of working, with an emphasis on reducing hospital admissions for patients with LTCs, as well as a pay freeze for NHS staff. 76

Not entirely coincidentally, at the same time as this reorganisation and economic squeeze, English general practices were entering a period of ‘crisis’,77 with rapidly rising workload as a result of increased numbers and complexity of consultations with no concomitant rise in HCP numbers and static or falling incomes. 78,79 As a result, practices were under enormous pressure. Many were unable to fill vacant clinical posts (both doctors and nurses were hard to recruit), leading to excessive workloads for remaining clinicians. 80 This was reflected in long waiting times for appointments and many GPs reported low morale, burn out and resistance to change. 80 Although some practices showed considerable resilience, others went into a spiral of decline. Over the period of this study, three practices closed in our study CCG.

The inner context for our case study included the features of the two CCGs that we intended to work with. One of the two was a dynamic and largely functional group, with strong clinical leadership and a commitment to innovation. It was one of the first CCGs to be authorised in England,225 going live in April 2013, and had an early goal of ‘cultural change to improve self-management of medical conditions’. 226 Improving prevention and management of LTCs was identified as a key priority, particularly in diabetes mellitus, heart failure and lung disease. PPI was also prioritised, with active patient input to key decision-making committees.

The other CCG also went live in April 2013. Its priorities for its first year were cardiovascular disease, cancer and respiratory disease, with a focus on improving delivery of care through community and primary care services. There was less emphasis on self-management and PPI,227 which are key elements of the HeLP-Diabetes programme.

Developing an implementation plan

Our original implementation plan was devised to be adaptive and iterative, with evaluation occurring in parallel. It involved a batched roll out, which was started by identifying two or three general practices with a reputation for embracing innovation and successfully managing change. We wanted to work closely with these practices to identify what worked well and what needed further attention, before moving on to further batches of practices.

As described in Chapter 2 and this chapter, the NPT stresses the importance of fitting with existing workflows and work patterns, so the implementation plan was designed to mirror existing systems, thus minimising the behaviour changes required by HCPs. One model of change that GPs were well used to was their local CCG (formerly PCT) commissioning a new service and encouraging practices to use it. We judged that having CCG support for the new service would be important for enhancing credibility and, in NPT terms, likely to promote coherence and cognitive participation.

Evaluating the implementation plan

Evaluating the impact that the Healthy Living for People with type 2 Diabetes programme had

The planned methodology for determining the impact that the HeLP-Diabetes programme had on both the primary outcomes selected for the trial (HbA_1c level and PAID score) and on hypothesised key intermediate outcomes was a series of substudies with a single-arm, pre–post design. Participants who consented to take part in the evaluation would be asked to complete additional questionnaires at baseline and follow-up. We planned to start with a substudy that focused on the impact that the intervention had on diabetes mellitus-related distress, as measured by the PAID scale, to generate data directly comparable to the trial data. Once that had been achieved, we hoped to run repeated substudies exploring outcomes of interest. However, as only 36 out of the 205 patients who registered to use the HeLP-Diabetes programme consented to participate in the associated research, we were unable to pursue this aim.

Developing the implementation plan

The original plan, with modifications, has been described in Developing an implementation plan, General practice.

Evaluating the implementation plan

Objective 2a: describe the adoption, uptake and use of the Healthy Living for People with type 2 Diabetes programme by health-care professionals and patients across participating sites

‘Adoption’ was defined as the service deciding to offer the HeLP-Diabetes programme to their patients, while ‘use by services’ was defined as staff within the service offering the HeLP-Diabetes programme to their patients and registering at least one patient. Adoption rates were high, but usage rates were lower (Table 36).

TABLE 36 - Adoption and usage rates by service

Of the 37 practices at the beginning of the study, three closed and could not adopt the HeLP-Diabetes programme.

The second CCG patients were not included in data in this section as, for reasons described in Developing an implementation plan, we were unable to put any resource into promoting or evaluating implementation in this CCG.

Service	Adoption rate (number of sites adopting the programme/number of sites that the programme was offered to; %)	Usage and number of patients registered	Comment
CCGs	2/2; 100	2/2; the first CCG registered 205 patients; and the second CCG registered 12 patientsb	The first CCG was extremely enthusiastic and proactive, so we focused our available resource there
General practices	22/34a; 65	18/22 registered at least one patient. The number of patients registered per practice ranged from 1 to 40, with a median of 3	Of the non-adopters, one site declined because of potential linkage with the EMR, six sites responded positively but we were unable to arrange a practice visit and eight sites never replied to the CCG or research team
Intermediate care service	1/1; 100	1/1; registered 1 patient	The service agreed to adoption of the programme but then experienced a year of substantial staff turnover; the new team again agreed to adopt the programme but only shortly before the end of the study
Secondary care service	1/1; 100	1/1; registered 30 patients	–

Uptake and use by patients

‘Uptake by patients’ was defined as the patient registering to use the HeLP-Diabetes programme. A total of 205 people with T2DM registered during the study period (March 2013 to August 2015). Of these, 143 (70%) were registered by a HCP, with the remainder (n = 62; 30%) self-registering. Use by patients was defined as patients using the intervention at least once after the day of registration. Just over half of those registered (n = 104; 51%) used the HeLP-Diabetes programme by this definition.

These users showed a wide range of usage of the intervention. The number of days with a recorded log-in ranged from 1 to 77 days, with a median of 2 days (interquartile range 1–4 days) of use per person. Users appeared to appreciate the availability of the intervention at all times, with page views recorded for every hour of the day. In total, there were 3221 page views, of which just over half (n = 1651; 51.2%) took place outside normal working hours (09.00–17.00) (Figure 21).

FIGURE 21.

Percentage of page views by time of the day.

However, less use of the intervention was made at weekends than during the week, with only 20% (n = 88) of visits and 13.7% (n = 443) of page views occurring on a Saturday or Sunday.

Content accessed by users

Users viewed 396 different pages out of the 560 available in the intervention. The most frequently viewed pages are displayed in Table 37: apart from the homepage, pages on food, common diabetes questions, care planning and the forum were accessed most often.

TABLE 37 - Most frequently viewed intervention pages

Title of page accessed	Page views
	Frequency	Percentage of total views
Home page	520	16.1
Food	62	1.9
Common diabetes questions	54	1.7
My diabetes care plan	43	1.3
Help Diabetes forum	42	1.3
How my body can be affected	39	1.2
My health profile	39	1.2
Eating and drinking	35	1.1
Exercise videos	32	1.0
My test results	32	1.0
Understanding diabetes	32	1.0
Staying healthy	28	0.9
My appointments	22	0.7
Understanding my moods	22	0.7
How is type 2 diabetes treated?	21	0.7
Living and working with diabetes	20	0.6
Physical activity	20	0.6
Looking after yourself	19	0.6
My health record	19	0.6
Snacks and desserts	19	0.6
Forum	17	0.5

Objective 2d: understand and explain observed variation between sites in adoption, uptake and use by health-care professionals and patients

Health-care professional interview data

Twenty-one members of practice and clinic staff took part in 17 interviews and one focus group (with four staff from the same general practice). The majority of staff (18/21) worked in general practices, with a further two based in hospital clinics and one at the CCG headquarters. Unfortunately, no members of staff from community clinics participated. Most interviewees (19/21) worked in services in which the HeLP-Diabetes programme had been adopted and implemented to some extent. The number of patients registered to the HeLP-Diabetes programme from these services ranged from 1 to 40. One participant worked in a practice in which the adoption of the HeLP-Diabetes programme had been declined and another worked in a GP practice in which the HeLP-Diabetes programme had been adopted (adoption agreed) but not used (no patients were registered) (Table 41).

TABLE 41 - Characteristics of HCPs interviewed (n = 21)

N/A, not applicable.

Identifier	Age range (years)	Gender	Professional role	Service setting	Years in current role	Ethnic background	Internet experience	Practice/clinic implementation status	Number of patients registered
ST1	35–44	Male	GP partner	General practice	11	White British	Experienced	Adopted and implemented	40
ST2	55–64	Female	Diabetes specialist nurse	Hospital	12	White British	Experienced	Adopted and implemented	30
ST3	45–54	Male	Commissioning officer	N/A	2	White British	Expert	Adopted	N/A
ST4	25–34	Female	Practice nurse	General practice	2	White British	Experienced	Adopted and implemented	28
ST5	45–54	Female	HCA	General practice	10	White British	Experienced	Adopted and implemented	40
ST6	35–44	Male	GP partner	General practice	3	Indian	Experienced	Adopted and implemented	8
ST7	35–44	Female	Receptionist/administrator	General practice	9	White British	Experienced	Adopted and implemented	40
ST8	18–24	Male	Receptionist/administrator	General practice	1	White and black Caribbean	Experienced	Adopted and implemented	11
ST9	55–64	Female	Advanced nurse practitioner	General practice	4	White British	Experienced	Adopted and implemented	3
ST10	35–44	Male	Practice manager	General practice	7	White British	Experienced	Not adopted	0
ST11	35–44	Female	GP partner	General practice	17	White British	Experienced	Adopted and implemented	15
ST12	45–54	Female	Practice nurse	General practice	4	White British	Experienced	Adopted and implemented	11
ST13	45–54	Female	Practice nurse	General practice	9	White British	Experienced	Adopted and implemented	11
ST14	35–44	Female	HCA	General practice	< 1	Sri Lankan	Experienced	Adopted and implemented	11
ST15	25–34	Female	Practice nurse	General practice	8	White New Zealand/American	Experienced	Adopted and implemented	11
ST16	55–64	Female	Diabetes specialist nurse	Hospital	1	White British	Experienced	Adopted and implemented	30
ST17	45–54	Female	HCA	General practice	7	White British	Expert	Adopted and implemented	40
ST18	25–34	Female	Receptionist/administrator	General practice	5	White British	Experienced	Adopted and implemented	1
ST19	45–54	Male	Salaried GP	General practice	1	White Irish	Experienced	Adopted and implemented	11
ST20	65–74	Female	Practice nurse	General practice	17	White British	Experienced	Adopted, but not implemented	0
ST21	55–64	Female	Practice nurse	General practice	20	White British	Experienced	Adopted and implemented	3

The themes that emerged from the data mapped easily onto the NPT, with no themes that did not map onto any NPT construct. The NPT provided a useful theoretical lens through which to interpret the qualitative data and explain the observed variation in adoption, uptake and use between sites. This variability was reflected in the variation in degree to which the constructs of NPT operated. Hence, the data from the interviews are presented under the main NPT constructs. As in WP B, interviewees distinguished between what they thought of as the main intervention (the HeLP-Diabetes support programme) and the facilitation we asked practices to provide. Overall, the HeLP-Diabetes programme appeared to have reasonably good levels of coherence, cognitive participation and collective action, whereas offering facilitation had poor levels of coherence and cognitive participation and a negative impact on collective action. As a result, overall uptake and use by patients was lower than HCPs had expected, and this led to HCPs perceiving the HeLP-Diabetes programme to be less beneficial than expected (reflexive monitoring). These findings are presented in more detail in the following sections.

Coherence

The HCPs in our study had a clear understanding of the potential benefits of self-management, both intrinsically or philosophically and as a response to lack of capacity in the health-care system to meet all the needs of patients with LTCs, in general, and diabetes mellitus, in particular. Self-management had the potential to improve health outcomes and quality of life for people with T2DM and to reduce health-care costs. They were also aware that patients needed considerable support to self-manage and varied in their capacity and readiness to self-manage and that there were problems with the current dominant model of support, namely group-based structured education shortly after diagnosis (as discussed in Chapters 2 and 5).

Professionals liked the concept of a web-based programme, were impressed by the holistic nature of the HeLP-Diabetes programme (addressing patients’ emotional and role issues as well as more obviously medical concerns), the way it presented information in an accessible format, including videos, and its user-friendly nature. It fitted well with local and national priorities and policies, including promotion of self-management and use of technology. These factors all helped explain the widescale adoption of the HeLP-Diabetes programme:

What HeLP-Diabetes does is it gives people that information that they can access at any time, because it’s on the computer.

ST1, GP partner

People can tailor it or get the information that they want out of it. In terms of timing they can access it whenever they want, rather than having to miss work, which is obviously quite a big issue for these patients.

ST4, practice nurse

In contrast, HCPs had more variable understanding of the need for supporting patients to use the HeLP-Diabetes programme. Although some felt that patients should simply be given the URL and left to it, others could see the value of registering patients and introducing them to the varied content.

Cognitive participation

Most HCPs felt that making the HeLP-Diabetes programme available to their patients was a legitimate part of their role and something that should be supported within the practice. However, for many practices, the structural constraints of workload and staff shortages made it impossible to take on any additional duties, such as registering patients or introducing them to the programme. This was an example of how implementation of even beneficial and legitimate interventions can be compromised by structural inequalities and shortages in resources:

And I think perhaps the fact the partners invited you [researcher] in to come and present about HeLP-Diabetes gave great weight to it as a programme, and because we obviously respect their judgement and they felt this was a good service and something important to promote in the practice.

ST12, practice nurse

It’s a brilliant website that patients can access really good-quality information at home, and take their time over it. Because when they come in to the doctors, sometimes we give information really rapidly and we’re always under time pressure. So, they can actually take their time.

ST13, practice nurse

Collective action

Some practices were able to incorporate the work of implementing the HeLP-Diabetes programme, including setting up systems for registering patients and introducing them to the programme. In these practices, HCPs reported that the HeLP-Diabetes programme had the potential to improve the quality of consultations and save HCP time, by reducing the number of consultations and making them more efficient. These practices found that HCAs were the appropriate professional group to register patients and facilitate access, as this task fitted well with their skill set and professional role. HCAs were often younger, IT-literate and keen to help patients but aware of their lack of clinical knowledge. Hence, HCPs liked being able to answer patient questions by turning to the HeLP-Diabetes programme and searching for the answer together with the patient. It was not a suitable task for general practice nurses, many of whom were not comfortable with IT and who felt their skills should be used on clinical tasks:

Actually, I would argue it shouldn’t be done by the nurse because, you know, it’s quite a low-key IT thing . . . I would rather our practice nurses spend their time, you know, doing the clinical work.

ST1, GP partner

And the barrier was this notion that we’d got to get the receptionist trained to fill in a form and to go online. That all sounded like a bit of a nightmare.

ST11, GP partner

I would register patients as they, as they came . . . It would be 10 minutes out of my day here and there . . . It wouldn’t be a massive, massive burden, no.

ST8, receptionist/administrator

Reflexive monitoring

In practices that managed to develop systems for registering patients on the HeLP-Diabetes programme, staff found that the work was not onerous and was justified by the benefits. In practices that lacked the capacity to incorporate this additional work, relatively few patients were registered, which led to reduced enthusiasm for the programme and even less appetite to take on the work:

I have asked some of them to get back in touch and let me know how they find it . . . because it would be interesting to know, and then I could say to other patients, well, actually patients have found this really helpful, you know.

ST5, HCA

Patient interview data

Interviews were undertaken with 15 patients between April 2014 and February 2015. Of these 15, 11 were male, nine were white British, five had no or only school-leaver qualifications, three had Advanced levels (A levels) or similar and seven had been educated to degree level or higher. Four described their computer skills as basic and a further five as intermediate. The duration of time since their diagnosis of diabetes mellitus ranged from < 1 year to 10–20 years. Treatment modalities included lifestyle only (n = 5) and lifestyle and tablets (n = 10). No interviewees were treated with insulin or other injectables (Table 42).

TABLE 42 - Characteristics of patient interviewees

CSE, Certificate of Secondary Education; GCSE, General Certificate of Secondary Education; HND, Higher National Diploma; O level, Ordinary level; NVQ, National Vocational Qualification.

Variable	Response options	Sample (n = 15)
Age (years)	Range	43–76
	Mean (SD)	59.3 (8.6)
Gender, n	Male	11
	Female	4
Ethnic group, n	White (English/Welsh/Scottish/Northern Irish/British)	9
	White (other)	2
	Black or black British (African)	1
	Black or black British (Caribbean)	1
	Mixed (other)	1
	Other	1
Highest educational attainment, n	None	3
	School leaver (e.g. CSE, GCSE, O level, NVQ 1–2)	2
	A level or vocational equivalent (e.g. NVQ 3)	3
	Degree or NVQ 4, HND or similar	5
	Postgraduate degree or NVQ 5	2
Duration of diabetes mellitus since diagnosis (years), n	< 1	3
	1–5	6
	> 5–10	3
	> 10–20	3
How diabetes mellitus managed?, n	Lifestyle only (diet and physical activity)	5
	Lifestyle and tablets	10
Internet access, n	Home	13
	Public	1
	No response	1
Computer skills, n	Basic	4
	Intermediate	6
	Advanced	5
Implementation model, n	Patient registered by staff	11
	Patient self-registered	4

The main themes that emerged from the patient interview data related to:

• the experience of diabetes mellitus

• perceptions of diabetes mellitus

• self-management

• self-management education and support

• diabetes mellitus information

• engagement with the HeLP-Diabetes programme

• use of the HeLP-Diabetes programme.

The inter-relationship between these themes is presented in Figure 22. Perceptions and use of the HeLP-Diabetes programme were contingent on users’ overall experience and perceptions of diabetes mellitus, their views on self-management and the available support and their perceptions of their need for information about diabetes mellitus. These themes did not map easily onto the NPT, and, in our view, use of the NPT as a theoretical lens to interpret these data did not enhance either the robustness or the transferability of the findings. Hence, only the results of the thematic analysis are presented.

FIGURE 22.

Themes identified from the analysis and the connections between them.

Patients’ experiences and perceptions of diabetes mellitus

This theme influenced all other themes in the data. Many people had difficulty accepting the diagnosis, particularly if they had no symptoms or if their diabetes mellitus was managed with lifestyle modification. Similarly, treatment with lifestyle modification or oral medication, rather than insulin, was often taken to mean that their diabetes mellitus was not ‘serious’:

I do not feel that I’ve got diabetes. I don’t feel, you know . . . I mean, I don’t take no medication. I don’t take nothing at all. I don’t test myself, because I’m not on medication.

PT9, male, aged 66 years, basic computer skills

In contrast, some people experienced symptoms, including fatigue, erectile dysfunction, poor healing and frequent urination, or complications, including neuropathy, peripheral vascular disease or retinopathy. These patients tended to perceive diabetes mellitus as a serious condition and wanted to urge others to take it seriously from the very beginning.

Self-management

Acceptance of their diagnosis of diabetes mellitus and perceptions of its seriousness influenced participants’ reported engagement with self-management, which is in turn important for implementation of the HeLP-Diabetes programme. A fear of disease progression and a desire to prevent future complications and worsening of health was a main priority for many participants engaged in self-management, whereas for others the motivation was a desire to avoid medication or to have a sense of control over their condition and disease trajectory:

Well, I really don’t want to be a diabetic and I certainly don’t want to reach a level where I . . . have to take medication.

PT5, male, aged 76 years, intermediate computer skills

Participants varied greatly in their engagement with self-management and, if engaged, the degree to which they had made lifestyle modifications. Factors included their readiness to engage, relative prioritisation of self-management vis-à-vis the rest of their lives, perceived responsibility for self-management and self-efficacy. Participants who were not ready to accept that they had diabetes mellitus or that it could have serious repercussions were unlikely to want to engage in self-management:

I don’t deal with that sort of thing otherwise I’ll be . . . I mean, I’m a worrier anyway and that would just be another thing just to . . . I’ve kind of put my head in the sand about the long term.

PT11, male, aged 46 years, advanced computer skills

For many, the demands of their lives, including work and family or caring responsibilities, made self-management difficult. Although most participants perceived self-management as their responsibility, some suggested they would prefer more support from professionals. Some found the burden of changing their diets, activity levels or other lifestyle modifications too great, whereas others found them relatively straightforward.

Self-management education and support

For most participants, the main source of HCP input came through primary care services, specifically appointments with GPs and nurses in their surgeries. The general practice also provided access to hospital services for specific diabetes mellitus-related tests and checks, appointments with dietitians and group-based DSME.

There were strong opposing feelings about the care and support participants were getting from their GP surgeries. Some felt their care was excellent, whereas others were dissatisfied, wanting more input, better access, more continuity of care and more practical support:

How would you describe the care that you receive from the NHS?

From my doctor it is absolutely top notch excellent.

PT6, male, aged 58 years, intermediate computer skills

Participants tended to distinguish between tasks that they perceived as the responsibility of HCPs and tasks they thought of as their own responsibility. In general, HCPs were perceived as responsible for providing information, prescribing, adjusting and monitoring medication and performing medical tests and checks, while patients’ responsibilities included being informed, following advice, attending appointments and making changes to diet and exercise.

Not all participants reported having been offered access to group-based structured education, particularly those who had been diagnosed for some years. Those participants who were offered access to group-based structured education reported several barriers to attending, including the fact that they were held during working hours, making it difficult for those in work to attend. Others felt too busy to attend a course that lasted several hours:

I couldn’t attend [group based education] because I had to go to work at the time there, so I couldn’t do it.

PT13, male, aged 52 years, intermediate computer skills

Among those that did attend, perceptions of the value of the course varied widely. Group-based education was particularly suited to those who reported feeling alone with their diabetes mellitus, as it provided an opportunity to connect with others, helped alleviate feelings of isolation and, for some, provided motivation to engage with lifestyle modifications. However, some participants said that the information provided was basic and insufficiently tailored, that the experience had not proved motivating or that they had felt uncomfortable in the group environment. Reasons for discomfort with the group format included difficulties with hearing, difficulties with understanding rapid spoken English and straightforward dislike of group interactions.

Information about diabetes mellitus

Many participants felt that providing information about diabetes mellitus and its treatment was a core responsibility of HCPs. For some, there was the implication that if their HCP did not tell them something, it could not be important and, therefore, there was no need to attend a course. Others were concerned that some HCPs’ knowledge was not up to date, leading to conflicting advice and uncertainty. Although some participants were keen to learn about their condition, others were reluctant, in case they learned something that would cause them anxiety or distress. This led to a range of perceptions about the degree to which patients had unmet informational needs:

PT1: Well, I don’t feel that the GPs – and I’m not blaming . . . my doctor is a very good doctor, but he’s just not up on diabetes. . . . And I just feel that they’re not really in touch.

PT1, male, aged 58 years, advanced computer skills

You say you haven’t really engaged with any information on diabetes because of the way you’re feeling about it?

I would say the bottom line is that . . . there is a kind of fear of confronting it which is sort of holding me back a bit.

PT12, female, aged 52 years, basic computer skills

Engagement with the Healthy Living for People with type 2 Diabetes programme

A range of factors influenced participants’ uptake and use of the HeLP-Diabetes programme, including acceptance of their condition, views about self-management and the burden of illness, satisfaction with their current care and education and a perception of unmet information needs.

Many patients had cogent reasons for not wanting to engage. Those not yet ready to engage with their diabetes mellitus felt that it might make them more aware of their condition, the possible future complications and the importance of self-management; others already felt that they were well informed or believed that they were managing as well as they could and saw no need for the intervention. Some already felt overburdened by the work of living with, and managing, diabetes mellitus, and found the texts, e-mails and newsletters from the HeLP-Diabetes programme an additional burden:

I have signed up to it but I think that would mean everything becomes a bit more real. At the moment, stabbing myself every so often and doing my long-term count . . . I don’t want to look at something knowing that it’s going to be bad.

PT11, male, aged 46 years, advanced computer skills

Well, the reason I don’t use it, as I explained, I feel I’m sort of self-managing it, so I’m not looking at it because I don’t need advice.

PT2, male, aged 63 years, advanced computer skills

Some people with T2DM, however, said they welcomed the HeLP-Diabetes programme, finding it a convenient, discreet (‘private’) and trustworthy source of information. Many reported finding it a useful resource for supplementing, checking and corroborating advice from their HCP:

[Diabetes mellitus] is a very private thing and . . . because you can go on to the website in the privacy of your own space and look things up.

PT7, female, aged 68 years, intermediate computer skills

Trust arose from the programme’s affiliations to the NHS and UCL, bodies that participants said they trusted and respected and that gave the HeLP-Diabetes programme credibility and authority. The fact that the HeLP-Diabetes programme was offered by their GP practice or diabetes clinic further underlined its credibility and trustworthiness. In fact, this endorsement by their HCP was of central importance in participants’ decisions to register and use the HeLP-Diabetes programme. HCP input into the registration process and any introduction to using the programme were warmly welcomed and clearly made a difference to the perceptions of its importance and value:

That was the only reason I signed up, because it was recommended, plus they had a person there to show me what it was like.

PT14, male, aged 56 years, advanced computer skills

Registration and initial introduction seemed particularly important in overcoming the digital divide; participants with only basic computer skills or with English as a second language said that being shown how to use the programme greatly helped build their interest and confidence. Those who had not had this facilitation spontaneously mentioned how such input could have helped them get more out of the programme.

Strengths

Limitations

Developing an effective digital health intervention is a complex process that takes time, resources and an appropriate multidisciplinary team

This may seem an obvious statement, and indeed, when we wrote the grant, we thought we had allowed a considerable amount of time (2 years) and resources (1.75 full-time equivalent RAs, 0.6 of a project manager, 0.6 of a clinical research fellow and 0.3 of a chief investigator, with additional input from co-investigators and funds for commercial collaborators) and had put together an appropriately multidisciplinary team (clinicians, patients, health psychologists, sociologists, health service managers and health service researchers).

However, in retrospect, we would have done better to include an additional year for piloting and refining not only the digital self-management programme (the HeLP-Diabetes programme) but also the surrounding delivery package (facilitated access). Had we done this, we would have been able to trial a sustainable intervention and delivery package that we had shown was feasible and acceptable in general practice. Other researchers in the field of digital health interventions have had similar insights, and this learning has been extensively shared, including in an international workshop on digital health interventions funded by the MRC from the UK and the National Institutes of Health and Robert Wood Johnson Foundation from the USA. This workshop resulted in a panel of publications. 185,243–246

Although our team was very multidisciplinary, with many investigators and collaborators who had considerable experience of undertaking research into digital health interventions, it did not have a computer scientist or HCI specialist. This had been a deliberate decision, as, at the time of writing the grant, we considered the health service research component to be extremely innovative but wanted the technology to be tried and tested, and so had thought that the project would not interest computer scientists who wanted to develop new knowledge in their field. This was the foundation for our decision to tender for commercial software and web-design companies; we also considered that commercial discipline would contribute to delivering on time and in budget. In retrospect, a computer scientist and a HCI specialist on the team would have helped in managing the technical side of the project and the interface with the academic work.

Problems that this might have alleviated included difficulties in communicating with software designers and understanding when we were requesting something that was straightforward to implement compared with something that required a great deal of programming or a new approach. It took us a long time to learn the relevant language and to express our health-related ideas in a form that made sense to the software engineers, despite them being familiar with working with academics and attempting to meet us halfway. Their commitment to the programme and their enjoyment of a challenge meant that, early on, they did not always explain the workload implications of a particular request; alternatively, it would not be clear even to them whether some requested functionality would require 1 or 10 days to program.

In the end, one member of our team who had a natural aptitude towards IT took on the role of ‘translating’ and became the sole communicator with the software team, and we learned to guillotine the amount of time that the engineers could spend on any one request to keep to required timelines.

We had also thought that by using open-source software we would minimise the risk of becoming entirely reliant on one company. In reality, the HeLP-Diabetes programme was such a large and complex programme that only those involved in its development could easily maintain it.

In the light of our experience, we would suggest that similar research programmes include computer science and/or HCI expertise on the team and specifically allow time for piloting the digital health intervention in routine practice to ensure that the surrounding delivery package is feasible and acceptable.

Working with the commercial sector requires careful planning

Before starting this programme of research, we had considerable experience of working with the commercial sector and had developed in-house guidelines for tendering for software and web-design work. This experience was vital in managing the complex process of tendering. We would advise those planning similar work to:

• ensure that the research team includes an excellent project manager

• work closely with host university support services (contracts, procurement, business advisors) to ensure that the tender is compliant with UK and European Union legislation

• think through the specifications in the tender in great detail

• allow for contingency

• ensure clear communication channels.

Digital health interventions require ongoing maintenance

Our original grant application was carefully costed and did indeed cover the costs of the development process. What we had not considered sufficiently were the costs of ongoing maintenance, in terms of revising content, improving navigation and design and the need for regular software updates to keep up with the broader technological environment, such as browser updates or new releases of the programs used in our intervention. We advise future researchers to budget for these maintenance costs for the duration of the research, not just for the development period.

Implementation of digital health interventions needs careful planning

Our emphasis on long-term implementation and dissemination has been successful to the extent that we set up a CIC, which obtained contracts with seven CCGs to provide the HeLP-Diabetes programme to their populations. However, the work of marketing and contracting with individual CCGs across England is considerable and in a commercial (albeit not-for-profit) model this work has to be paid for through sales of the product. This increases the cost of the HeLP-Diabetes programme, and it is doubtful whether or not it is a good use of resources in the current financial climate. Central commissioning by either NHS England or Public Health England would be more appropriate for effective digital health interventions, for which the cost per additional user tends to zero with increasing numbers.

Undertaking implementation research in parallel with effectiveness studies

Undertaking implementation research in parallel with clinical effectiveness evaluations has advantages in reducing the overall time taken to generate the data needed to inform policy. Moreover, the use of different research designs in parallel allows for added value in terms of total data generated.

Implications for future research

In our view, our findings highlight a number of areas that need further research:

1. further exploration of the reasons why people with T2DM choose not to engage with self-management support and how they can be encouraged to perceive its value

2. in parallel, more work on how to support HCPs in promoting patient engagement with the HeLP-Diabetes programme or similar programmes – this could include further clarification of underlying reasons that HCPs have for not valuing this work and not seeing it as part of their role, as well as identifying factors that would help them see its value and incorporate it into routine practice

3. as part of this, further work on linking the HeLP-Diabetes programme with the GP EMRs would be useful, exploring how to make this acceptable to GPs and useful to patients and whether or not such linkage does improve uptake of and engagement with the intervention

4. given the importance of overcoming the digital divide, further work on how this can be aided by HCP input would be useful

5. although the trial demonstrated the intervention’s efficacy, we lack data confirming its clinical effectiveness in routine use; this is clearly an area for which further research is needed.

Implications for practice and policy

1. Our findings suggest that widespread deployment of the HeLP-Diabetes programme across the NHS could benefit patients and that it has a very high probability of being cost-effective at standard WTP thresholds.

2. Widespread implementation would be best undertaken centrally, as this would improve the overall cost-effectiveness. The alternative model of implementation by individual CCGs increases costs, particularly in marketing and the costs of agreeing and maintaining contracts.

3. Digital health interventions may have a role in enabling the NHS to deliver more and better care for less resource, but only if there is significant investment in developing acceptable, clinically effective and implementable interventions.

4. Along with this investment in developing and evaluating digital health interventions, there is a need to consider how best to commission, implement and maintain such interventions in a cost-effective manner.

The authors wish to thank

Fiona Giles, UCL; Helen Gibson, diabetes specialist nurse; Rebecca Owen, diabetes specialist nurse; Dr Anisoara Jardim, programme manager, NIHR Programme Grants for Applied Research; and Nicolas Barnard, monograph editor.

Additional co-investigators

Dr Stephen Thomas, consultant in diabetes medicine; Charles Gore, PPI and co-investigator; and Mary Jamal, NHS manager, The Whittington Hospital.

Patient representatives

Joni Iniss, PPI and co-investigator; Bindie Wood; Niccy Whitaker; Jonathan Benford; Malcolm Knox; Pamela Ellam; Woolf Van Silver; Charles Prince; Paul Davies; Ros Bacal; Angela Edwards; Caroline Szylit; Robin Healey; Nastir Matabdin; David Forbes; Frank Wood; Andy West; James Munroe; Isidora Walcott; John Mealings (deceased); Tim Saunders; and Trisha Shirt.

The team who worked on the development of the HeLP-Diabetes programme

Joe Palmer, SoftForge; Phil Walton, SoftForge; Christina Lamb, web design; Eleana Papadopoulou, dietitian; Douglas Twenefour, dietitian; Janice Mavroskoufis, dietitian; Adam Cohen, video production; Rupert Charmak, fitness trainer; Angela Nicholas, editing; Julia Bindman, editing; Dr Elsa Gubert, GP academic trainee; and Richard Peacock, information scientist.

The trial steering committee and wider trial team

Professor Frances Mair, University of Glasgow; Professor Peter Hindmarsh, UCL Hospital; Professor Nick Freemantle, UCL; Deborah Haworth, former trial manager; and Kerstin Modrow, data officer, UCL.

The collaborators

Charles Gore, Hepatitis C Trust; Dr John Powell, NHS Choices, editor of the NIHR Journals Library and member of the Health Technology Assessment and Efficacy and Mechanism Evaluation Boards; Professor Irwin Nazareth, UCL Priment; and Dr Tom Aslan, GP, Camden.